Running the Linux Agent
The commands and their exit codes, what the three systemd units do, how status and doctor fit into monitoring, test in place of fail2ban-regex, and the troubleshooting table: symptom, cause, command.
Commands
| Command | What it does | Exit |
|---|---|---|
REPORTEDIP_KEY=K install [--accept-terms] [--admin-ip IP|CIDR] [--ssh-port N] [--mode log|drop] [--ban|--no-ban] [--notify-email A] [--log-level L] [--expert] | Detects the services, writes the config, the auto-whitelist and the install ID, installs and enables the units, and verifies the key. The key comes from the environment; --key still works and warns that its value is in the process list. Never overwrites an existing config. Root only. --expert also asks for the ban ladder, the thresholds, the limits and an SMTP server. | 0, 2 on a rejected key before anything is written or on a config problem, 1 when the key check after the install fails |
sync | One feed pass: rebuild the whitelist set, fetch each list conditionally, swap what passed the size check, rebuild the chain, restore missing local bans, run the housekeeping, check for an update every six hours. This is what the timer runs and what runs at boot. | 0, 1 degraded, 0 also when another sync holds the lock |
watch | The daemon. Tails every source, counts, reports and bans if that is enabled. Stops on SIGTERM. | 1 when the queue lock is still held after a minute or the daemon stops on an error, 2 on a config problem |
report-queue | Sends the queued reports once. Useful after an API outage, or on a host where the daemon does not run. | 0, 1 degraded |
status | Version, backend, mode, tools, per-list counts and last success, whitelist, chain check, per-source read positions and hit counts, queue and sender, account and licence, the group of this key, the reputation of the address this host reports from, disk, local bans, open health conditions. | 0 healthy, 1 degraded, 2 config |
doctor | What the host offers and what is missing, including a log this host writes that no source reads. Changes nothing, works without a config. | 0, 1 with limits, 2 cannot run |
test <file>... [--type T] [--rule ID] [--lines] | Runs the detectors over real log files. Reports nothing, bans nothing, writes nothing. --rule runs one rule from a rule file on its own. | 0, 2 on a file or type problem |
ban list | add <ip> [minutes] | rm <ip> | Shows the local bans against what the kernel really holds, sets one by hand, lifts one. | 0, 1 on a kernel failure, 2 on a bad argument or while a sync holds the lock; add also on a whitelisted address or when the host has no firewall backend |
unban <ip> | The same as ban rm, under the word an operator types when something is on fire. | as above |
whitelist add <ip|cidr> [comment] | list | rm <ip|cidr> [--auto] | Manages the never-block, never-report list. add writes the file and the kernel set at once. list prints your file, the auto-whitelist and the whitelist of your group, the last on lines marked group: with their notes. --auto removes an entry the installer wrote. | 0, 1 when the file changed but the set did not, 2 on a bad address or while a sync holds the lock |
update [--check] | Checks the distribution point and replaces this binary, then restarts the watch service. --check only reports. This is the one command that reads the config loosely, so it still works on a host whose config is newer than its binary. | 0, 1 when the check or the install failed, 2 when the config cannot be read or the state directory cannot be created |
housekeeping | Removes the agent's own leftovers and prints the numbers. The sync run does the same silently. | 0, 2 on a config or state problem |
rules list | show <id> | check [file...] | export <id> | status | enable <id> | disable <id> | The detection rules: the shipped ones and the files under /etc/reportedip-agent/rules.d that override them or add to them. check tests a file before it goes live, export prints a rule as a starting point, status adds the hits and which rules are young, enable and disable switch one rule on or off. | 0, 1 when a file was skipped or the operator rules are not used, 2 on a bad argument or when check finds a problem |
version | Prints the version and nothing else. | 0 |
help | The command list, with the config and state paths. | 0 |
test, instead of fail2ban-regex
test runs the detector chain over a log file you already have, with the thresholds and
the whitelist of this host, and prints which addresses would have crossed a threshold and where in
the file. It reports nothing, blocks nothing and writes nothing, so it is safe on a production host
and it is the honest way to answer "would this have caught last week's attack".
# A configured source: the type is taken from the config
reportedip-agent test /var/log/auth.log
# Any other file: name the type yourself
reportedip-agent test --type postfix /var/log/mail.log.1
# A rotated log and its successor as one stream, in this order,
# so a window that spans the rotation counts like the daemon saw it.
# A .gz is read inflated.
reportedip-agent test /var/log/mail.log.2.gz /var/log/mail.log.1 /var/log/mail.log
# Every single hit, with line number, address and timestamp
reportedip-agent test --lines /var/log/auth.logThe summary line gives the lines read, how many matched, how many carried no usable timestamp and how many were skipped as overlong. Then one row per address with its hits, its resets, whether it would have been banned and how often, the time of the first hit, and the threshold that was applied. An address that is on your whitelist is shown with the layer that matched and marked as never banned and never reported, which makes this the fastest way to prove a whitelist entry actually works.
Two things test does not do. It does not guess a format: without --type the
file has to be one of the configured source paths, and otherwise it tells you so and lists the valid
types. And it does not apply the ban-history weight, so the daemon needs fewer hits for an address
with previous bans than this replay suggests.
Running it
Three systemd units, written and enabled by install and brought up to the state of the
running version by every sync. Put your own changes into a drop-in under
/etc/systemd/system/<unit>.d/, where they survive an update. A unit you edited
directly is replaced, the previous version is left beside it as <unit>.bak, and
the journal names both.
| Unit | When it runs | Details |
|---|---|---|
reportedip-agent-sync.timer | Every 15 minutes, plus a fixed random offset of up to 2 minutes | It has to fire at the fastest rate any licence can ask for, because it is the only thing that starts a sync. How many of those runs really fetch is decided by the licence, and a run that comes too early for its plan fetches nothing. |
reportedip-agent-sync.service | At boot, and on every timer tick | Enabled for multi-user.target, after network-online.target and after every firewall service, with no random delay. One sync pass: whitelist set, the six lists, the chain, the restore of missing local bans, the housekeeping, and an update check every six hours. |
reportedip-agent.service | Continuously | The watch daemon. Restarts on failure after ten seconds, at most five times in five minutes, and never on exit code 2, so a misconfigured host stops with a readable reason instead of being retried for ever. |
Both units run with NoNewPrivileges, ProtectHome,
PrivateTmp, a locked personality and a restricted set of address families.
| What runs by itself | How often | What it does |
|---|---|---|
| Log rotation | On every write | The agent rotates log_file at log_max_mb and keeps log_keep generations. It does not depend on logrotate. Every line also goes to stderr and so into the journal of the unit. |
housekeeping | In every sync | Old queue files, expired dedup entries, stale read positions, leftover temporary and lock files, expired ban records, surplus generations of the agent's own log, files in the mirror of the shipped rules that the binary no longer carries, and the previous binary once it is a month old. Run it by hand to see the numbers. |
| Self update | Every six hours | A new release is verified against an Ed25519 signature with the public key compiled into the binary before it is installed, so a tampered download fails on your machine rather than being trusted because it arrived over HTTPS. A swap that fails puts the running binary back, so a broken update leaves the host on the version it had. |
| State mail | Per condition, with notify.cooldown_hours | One mail when something is wrong, one when it is fixed. From Professional it can go out through the relay of reportedip.com. |
| The disk floor | On every queue write | Below disk_min_mb the agent queues no new reports and says so, while detection and blocking keep running. |
systemctl list-timers reportedip-agent-sync.timer
systemctl status reportedip-agent.service --no-pager
journalctl -u reportedip-agent.service -n 50 --no-pager
journalctl -u reportedip-agent-sync.service -n 50 --no-pager
tail -n 100 /var/log/reportedip-agent.logExit codes
| Code | Meaning | What to do |
|---|---|---|
0 | Healthy | Nothing to do. |
1 | Degraded, but it will try again | Read the output. A retry may well fix it. An open error condition in the health record colours the exit code even when the run itself went through, because an exit 0 next to a recorded error is no signal for a monitoring system. For status, a watch daemon that stopped is one of these reasons. |
2 | Not repeatable without a human | The config, the file permissions, a missing tool, a lock held by another process. Run doctor. systemd will not retry the watch service on a 2. |
That makes reportedip-agent status usable as a monitoring check directly, with no
wrapper script and no output parsing. status --json prints the same result as a JSON
document; the setup for Zabbix, Nagios, Icinga, Checkmk and Prometheus is under
Monitoring.
The reputation of this host's own address
The sync run asks the server what it thinks of the address this host reports from, at least once a day and on every run while the condition below is open. If that address
is itself listed in the community database, status warns and exits 1. With a confidence of 75 or more
the health condition reputation is open as well, with the confidence, the number of reports and the link to the
delisting page in its text. It is a warning and not an error, because a listed
address is a fact about the network the host lives in and not necessarily about the host: a shared
address, a carrier NAT or a neighbour behind the same proxy can put it there. It is still worth a
look, because a host that reports from a listed address is what a compromised host looks like from
outside. Check what leaves the machine, request the delisting once it is clean, and the condition
clears with the next sync after the delisting; there is nothing to reset on the host. Below 75 no health condition is raised: a handful of old reports is honest data and not something an operator can act on, and no feed serves an address at that score.
Troubleshooting
One symptom, one likely cause, one command. reportedip-agent doctor and
reportedip-agent status between them answer most of these, and they are the first two
things support will ask for.
| Symptom | Cause | Command |
|---|---|---|
| Refuses to start and names its config file | The file is readable by group or others, and it holds your API key. | chmod 0600 /etc/reportedip-agent/config.yaml && chown root:root /etc/reportedip-agent/config.yaml |
| Exit 2 with "field ... not found in type" | An unknown key in the config. The parser is strict on purpose. | Remove or correct the key the message names. The list of valid keys is above. |
| Every command exits 2 after a rollback | The binary is older than the config and cannot parse a newer block. | reportedip-agent update, the one command that reads the file loosely and names the keys it does not know. |
An older binary exits 2 on a config that carries notify.relay | That key is newer than the binary. The same class of trap as the row above, and it has left hosts unprotected twice. | Remove the relay: line from the notify block, or go forward instead of back: reportedip-agent update. |
| The sets exist but are empty | Either the feed was refused or the list failed the size check. | reportedip-agent status says which of the two, and the per-list line carries the last error. |
status says unlicensed | The account has no licence to spare for this host. | Add one under Agent Servers. The list in the kernel keeps working meanwhile. |
| Nothing is ever reported | The source that would have fired was never detected, or its threshold is not reached. | reportedip-agent test /path/to/log, then reportedip-agent status and read the sources section. |
| Nothing is ever blocked, although reports go out | ban.enabled was set to false, or mode to log (by default both block). Two switches. | reportedip-agent ban list says which of the two, in its last lines. |
| No source state at all | The watch daemon never ran. | systemctl enable --now reportedip-agent.service |
A source shows files=0 | The path or glob resolves to nothing on this host. | reportedip-agent doctor names the source and the pattern it tried. |
| A service installed after the agent reports nothing | Detection runs once, at install time, and that is deliberate: a service stopped for maintenance must never switch a source off. The other way round nothing happens by itself either, so a mail server added to a host that had none before is not picked up. | reportedip-agent doctor names the log that service writes, says no source reads it and exits 1. Add the source to config.yaml and restart the watch service. |
| A log source is listed as unreadable | The file is not readable even for root. Usually a control panel that sets mode 000 on rotation. | ls -l on the path, and fix the rotation that produced it. |
| A source has hits but never reports | Its log timestamps are more than a minute from the system clock, so the counting window never fills. | reportedip-agent doctor, the drift lines under sources. Fix the time zone of that log. |
| Your own address got banned | It was not on the whitelist. | reportedip-agent whitelist add <address>. Instant, because the whitelist rule sits before the block rule. |
A ban is in bans.json but not in the kernel | A reboot without a sync, or a firewall reload that took the set with it. | reportedip-agent sync restores it with the time it has left. |
An entry in the kernel with record none | Somebody banned by hand with ipset or nft, or the store was lost. It expires, but no reboot brings it back. | reportedip-agent ban list |
| A set exists with the wrong type, or without a timeout | A leftover from the documented shell script, or from another tool. | ipset destroy <set> and then reportedip-agent sync, which rebuilds it correctly. |
Both tool chains have rip- objects | The backend was changed without a migration. | reportedip-agent sync --migrate-backend, and status prints the exact removal commands for the other side. |
| The firewall is gone after a reboot | A saved ruleset references a rip- set, and iptables-restore discards the whole file on an unknown set. | grep -n rip- /etc/iptables/rules.v4 /etc/iptables/rules.v6 and remove those lines. The agent needs nothing persisted. |
| A removed source is still being read | A systemctl restart is not enough for a removed source. | systemctl stop reportedip-agent.service && systemctl start reportedip-agent.service |
| Reports stop but detection continues | The disk-space floor was reached. | Free space, then reportedip-agent housekeeping. |
| HTTP 429 on reports | The daily report limit of your plan. | The limits are listed under Authentication. |
| The queue keeps growing | The sender is paused after a failure, with a backoff. | reportedip-agent status shows the pause and the reason; while the daemon runs it sends by itself, and reportedip-agent report-queue only sends when the daemon is not running. |
| Every web hit is the same handful of addresses | nginx sees a proxy or Cloudflare and not the visitor. | Configure set_real_ip_from. reportedip-agent install warns about exactly this case. |
| "more addresses than the counter tracks at once" | A scan wider than the per-source counter. Not a fault. | Nothing. A scan that wide is a case for the community list, not for local bans. |
status says the host is behind on versions | The automatic update is not getting through, or auto_update is off. | reportedip-agent update --check, then reportedip-agent update. |
status says group=none | The key of this host is in no group. Not a fault: the set rip-group stays empty. | Create a group under Groups in your account and put the keys of the hosts in it; the next sync fills the set. |
status warns about the reputation of this host's address | The address this host reports from is listed in the community database. | Check what leaves the machine, then request delisting. The condition clears with the next sync after that. |
No rip-scan: lines in the kernel log | No scan source in the config, or the sync that builds the rule has not run since it was added. | Add - type: scan under sources, restart the watch service, then reportedip-agent sync. Whitelisted and already dropped addresses never produce a line. |
Last updated: · Maintained by the ReportedIP team