pinger

pingerd pings every host of the IP plan and says how long the answers took, for Prometheus.

• Whom it pings: every host in /etc/ipplan.db (table host) that has an IPv4 address. The database is read again for every round, so new hosts are picked up without a restart; a host that has left the plan is pinged until the next restart.
• One site only: --site SITE (several times, or --site A,B) keeps it to the hosts of those sites of the plan. A host's site is the part of its network's name before the @ - EVENT in EVENT@TECH-SRV-A; upper or lower case is the same. Without the switch it pings every site's hosts, as it always has.
• Another database: --database FILE.
• How often: once a second (--interval).
• What it says: the histogram icmp_rtt_seconds, one per host (label host), on http://<server>:13110/metrics (--port). A host that does not answer simply gets no new observations: compare icmp_rtt_seconds_count over time.
• Rights: it starts as root and continues as nobody - see "Privileges" below.

The sending and receiving is a small C module, dhmonpinger (src/pinger/dhmonpinger.c); the daemon is src/pinger/pingerd.

Privileges

pingerd must be started as root, and is root only while it starts:

1. as root it opens the one raw ICMP socket, sets that socket's receive buffer (see below) and raises its own priority (nice -15);
2. then it changes to the group nogroup and the user nobody. With that it also loses every capability; the socket, already open, keeps working.

Everything else is done as nobody: reading the plan database, sending the pings, receiving the answers, and the web server on port 13110. /etc/ipplan.db must therefore be readable by everyone.

Two things it does not do:

• It does not clear root's supplementary groups. Started by systemd there are none (a root service carries none on Debian); started by hand from a root shell, it keeps the groups of that shell.
• It has no account of its own: it shares nobody with whatever else runs as that user on the machine.

Every package is tried for this before it is released: the running daemon's user must be nobody (.github/try.sh).

The receive buffer

The answers of a round arrive all at once and wait in the socket's receive buffer until pingerd has read them; what does not fit is lost, and looks like hosts that did not answer. The module therefore asks for a large buffer (3670016 bytes, which the kernel counts as twice that).

• Started as root - as the unit does - pingerd gets that buffer whatever the kernel's limit is (it has CAP_NET_ADMIN and forces the size). Nothing needs to be set.

• Started without CAP_NET_ADMIN (in a container, say), the kernel silently cuts the buffer down to net.core.rmem_max - 212992 on a stock Debian 13, a seventeenth of what is asked for. pingerd says so when it starts (WARNING: the receive buffer is ...). Then raise the limit, in /etc/sysctl.d/:

  net.core.rmem_max = 8388608
  

Running it

A release is a Debian package, built for Debian 13 - nothing is compiled on the server:

pinger_X.Y.Z_amd64.deb
    /usr/sbin/pingerd
    /usr/lib/python3/dist-packages/dhmonpinger.cpython-313-x86_64-linux-gnu.so
    /usr/lib/systemd/system/pingerd.service

apt install ./pinger_X.Y.Z_amd64.deb
systemctl enable --now pingerd

apt fetches what it depends on: Debian's python3-prometheus-client. The package depends on exactly the Python it was built for (Debian 13's, 3.13) - the module is tied to it. Installing it starts nothing (pingerd needs /etc/ipplan.db); an upgrade restarts a pingerd that is running.

Releasing

1. Set the version in src/pinger/setup.py.
2. Push; the ci workflow builds the package and tries it.
3. Tag that commit vX.Y.Z (the same version) and push the tag: the release workflow builds, tries and publishes the package and SHA256SUMS as a GitHub release.

Building is .github/build.sh. Trying is .github/try.sh, in a new Debian 13 container that has no compiler: the package installs with apt, the module pings 127.0.0.1 (tests/loopback.py), pingerd reads its hosts from a plan database and answers on its web port with what it measured, as nobody, and the package is removed again.

Building by hand

docker run --rm -v "$PWD:/src" -w /src debian:13 sh .github/build.sh
docker run --rm --cap-add NET_RAW --cap-add SYS_NICE \
    --cap-add NET_ADMIN -e PINGER_FULL_BUFFER=1 \
    -v "$PWD:/src:ro" -w /src debian:13 sh .github/try.sh

The first leaves the package in dist/. (With rootless podman the second stops where pingerd raises its priority, and the buffer cannot be forced: both need real root.)

The old src/pinger/Makefile (build and install straight onto the machine it runs on, under /opt/pinger) is unchanged, and so is pingerd.service in the repository, which names that path; the package's unit names /usr/sbin/pingerd.