Get me notified please!
Last week my server got down an i only noticed it by accident. So i began to think about asolution to get me informed as soon as one of my services on my server goes down. Because you know connectivity is important
gotify
Gotify is a notification server similar to FCM (Firebase Cloud Messaging). There are alot of clients available for every thinkable platform. One could also easily write one, because there is a REST Api exposed on the Server, so basically thats what i was going to do. I wanted to get notified when a system critical systemd service fails or crashes. So i had a rough plan for my project.
DBus
First thoughts went into the direction, parsing the cmd output of systemctl status ... . But this did not feel right for a serious implemention. So i figured out that systemd works over the DBus interface, and i could get my data over that. I wanted to program the whole thing in C, because i dont want overhead for such a trivial task. So i begun to search for a good dbus lib. There are several for example: dbus-glib, sd-bus. But every lib i found was either bad documented or very outdated.
So i decided to use the lowlewel C api of the official dbus library. They do not recommend implementing software with the lowlevel api, mostly because of complexity, it should mostly be useful for language implementations. But im going to do it anyway, so i have as good as no dependencies, and i also understand the lowlevel code better than the high level ones.
Dbus has a very nasty terminology which is easy confulable. For example:
- object paths: /org/freedesktop/systemd1
- interfaces: org.freedesktop.systemd1.Manager
- Methods / Signals: ListUnitFiles()
Specially object paths and interfaces are very easy confused with each other. I always confused object and interface, so my mnemonic for this is:
- Object path is the endpoint i want to connect.
- Interface is where the "functions(methods)" are located.
- methods are the functions.
systemd
The most work was the dbus implementation, the only thing to do here is to choose the right methods and parameters and parse the data which comes back from the systemd dbus api.The importent parts are:
- Our interface for the dbus call is"org.freedesktop.DBus.Properties"
- Our object path is the service we would like to test in a strange format like
/org/freedesktop/systemd1/unit/NetworkManager_2dwait_2donline_2eservice - Our method we want to use, is called "Get" witch is located at the interface org.freedesktop.DBus.Properties
- Our method wants two arguments, which interface to check and what property we want
- The property is "SubState"
- Parse and type check the returned values
config
I wanted to have a list of services to check for in a config file. So i searched for a config parser lib, but was not happy at all with the available ones, and wrote the parer myself for a simple config file, as it turned out it was not that difficult if it does not has to be generalised or abstracted. I even included Sections in the form of [SECTION] which also supports config flags for the daemon to run.
Final steps
I went on to daemonize the application, so itself can be run as a systemd service, which is quit easy done with a .service file.Is it actually usefull?
Actually im running it on a "productive" server and i think it is quite handy. I tested a few scenarios and i always got the notification onto my android phone that something is not working as it should.
