Issue
@Alex notified us about the expiration of the SSL/TLS certificate offered by the Mosquitto broker providing MQTT for "swarm.hiveeyes.org":
$ echo "Q" | openssl s_client -host swarm.hiveeyes.org -port 8883 2> /dev/null | openssl x509 -noout -enddate
notAfter=Sep 29 03:52:26 2018 GMT
Thanks Alex, you saved us once again! See TLS Zertifikat für MQTT broker auf swarm.hiveeyes.org ist abgelaufen.
Investigation
After renewing the certificates via Let’s Encrypt last time, the certificate storage path changed from
/etc/letsencrypt/live/swarm.hiveeyes.org to
/etc/letsencrypt/live/swarm.hiveeyes.org-0001 due to the suffixing thing [1,2].
As we didn’t monitor the certificate offered by the MQTT service in an automated way, this slipped through. Bummer!
[1] Explicit specify cert storage location(remove use of automatic -0001 suffixes) · Issue #2128 · certbot/certbot · GitHub
[2] https://community.letsencrypt.org/t/how-to-prevent-creation-of-etc-letsencrypt-live-domain-tld-0001-when-removing-domains-from-a-domain-tld-multidomain-certificate/8135
Solution
We just amended the “cafile”, “certfile” and “keyfile” configuration settings for Mosquitto to account for the updated certificate storage location. From now on, the certificate should be automatically renewed including restarting the Mosquitto daemon. Additionally, we added another sensor to our monitoring system for keeping an eye on the certificate expiration date.
@Alex answered promptly while i haven’t even finished typing this:
Verify return code: 0 (ok)
Danke!
Thank you, @Alex!
Sorry for any inconveniences this might have caused.