Veröffentlicht: Montag, 03. Februar 2020 15:51
Geschrieben von Martin Wilmes
To check if the content of a certain web page changed is not as obvious as it should be in Zabbix but my experience with Zabbix is limited so I needed some time to figure out how to do this.
First of all you need a host, lets say "java-cup.de". On this host you define an item with a key that is constructed like this:
This key just retrieves the content of the web page, including all headers. There are more parameters to web.page.get possible but we do not need them, if we supply a complete URL on the first parameter.
There is also the key web.page.regexp but I prefer to collect the full web page in this step because of being more flexible afterwards.
Then we need a trigger that references the just created item. It uses the function changes to tell if the page was modified since the last retrieval.
With just this setup a change is detected on every retrieval, because of the headers being included. There are always the headers date and expires that contain a date and time of the retrieval/expiry of the page, so that changes on every usage of the item.
To avoid this we have to parse the content from the page that we want to watch. This can easily been done by using the Preprocessing tab on the item. Here we can employ a regexp to extract the data that we want to watch.
Even better, we can do this in multiple stages so that we can do the extraction process in small steps. Of course, it isn't optimal to use regexps on HTML but there is no HTML parser available and the XPath parser that we coud use here instead of the regexp will not work on most HTML pages since they are not well defined XML.
Veröffentlicht: Montag, 20. Januar 2020 09:59
Geschrieben von Martin Wilmes
Recently I had to work on a Ubuntu 18.04 system where a colleague installed letsencrypt and created the initial certificates. There was no problem so far and everything seemed to be ok until the certs had to be renewed. They simply did not and we had to find the reason because the initial certificate neared expiration. Of course we could have just do it manually but you surely agree that this is something that should work automatically.
I started the search and the colleague who initially installed letsencrypt started as well. It turned out, that the reason for the renewal not working was that my colleague used the standalone authentication when initializing the certificates and now when Apache is running, certbot was unable to acquire port 80. So we installed the additional package python-certbot-apache and added the option --apache to the crontab entry in /etc/cron.d/cron. After a manual update we thought we would be done with this.
But two months later it turned out that we were not: Again, the certificate was not renewed. Again, from the logfile (/var/log/letsencrypt/letsencrypt.log) it turned out that the renewal was tried by using the standalone authenticator!
Now I started a deeper search and had a closer look at the line in the crontab:
0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot --apache -q renew
And I noticed, that the renewal command was never executed because of the test command in from of it!
So I had to not e that on systems using systemd this crontab entry never executes because a respective entry in systemd exists!
The service entry can be found in /lib/systemd/system/certbot.service, the scheduling for it in /lib/systemd/system/certbot.timer.
This entry also contained no hint to using apache and I was already going to create an override entry for the cerbot service when I noticed that there is a configuration file for every certificate that shall be renewed by certbot.
It looked like this:
# renew_before_expiry = 30 days
version = 0.27.0
archive_dir = /etc/letsencrypt/archive/xxx.ch
cert = /etc/letsencrypt/live/xxx.ch/cert.pem
privkey = /etc/letsencrypt/live/xxx.ch/privkey.pem
chain = /etc/letsencrypt/live/xxx.ch/chain.pem
fullchain = /etc/letsencrypt/live/xxx.ch/fullchain.pem
# Options used in the renewal process
account = b61bb893aa21cbd202aaaaaaaaaaaaaa
server = https://acme-v02.api.letsencrypt.org/directory
It is not recommended by letsencrypt to modify this file manually but in this case I had to because the configuration of the renewal changed after the first installation. So I added these two lines to the section "renewalparams":
authenticator = apache
renew_hook = systemctl reload apache2
Now, with these parameters switched on the renewal finally worked as expected!