Detection and rules

What the agent finds in the logs a server already writes: the fourteen source types and their thresholds, how a source the installer did not find is added, what leaves the machine and what never does, and the rule files every detector is made of, including the ones you write yourself.

What the agent finds in your logs

Each source has its own detector and its own threshold, because five failed SSH logins in ten minutes and fifty suspicious web requests in two hours are the same statement about an attacker and not the same number. Rotation, copytruncate and a log that disappears for a while are handled, and an unreadable file produces one warning rather than one per read.

The fourteen source types

web-app is not a source type and cannot be written into sources. It is a second detector that runs on the lines a web source already reads, and it has its own threshold, which is why it appears under its own name in the threshold table above. An agent whose configuration names it refuses to start: sources[0]: type "web-app" is not supported in this version.

Port scans, from the kernel log

Every other source reads a log a service writes. The scan source reads the kernel, and it is the one source that also changes the firewall: with it in the config, the sync run adds a rule behind the lists that logs a SYN to a port this host does not listen on, with the prefix rip-scan: and at most ten lines a minute after a first burst of 20, so a scan of all 65535 ports cannot flood the journal. With the ipset backend that is a chain of its own, rip-blacklist-scan; with nftables it is one rule in the chain. The listening ports are read from /proc on every sync, so a service you start is not a scan target after the next run, and there is nothing to declare. Only a socket something outside could reach counts as listening. A port bound to 127.0.0.1 or ::1 alone stays in the rule, because a packet that arrives there from the network is a probe of a closed port and nothing else. The list from the last sync is not the only check: the rule also asks the kernel when the packet arrives, and a SYN to a port on which a socket is listening at that moment is a connection and is never logged. That covers a service started since the last sync and a daemon that opens a port for a single transfer. The passive port range of a running FTP server (pure-ftpd, proftpd, vsftpd) is read from its configuration and left out of the rule; if the server runs without a configured range, the kernel's ephemeral port range is left out instead. A kernel without the socket match keeps the rule without that check.

The rule stands behind the whitelist and behind the lists. An address on your whitelist returned from the chain before it and produces no line, and an address the lists already drop never reaches it either. In mode: log and mode: off it does what everything else does in those modes.

The source reads the kernel journal, journalctl -k, and falls back to /var/log/kern.log or /var/log/messages where there is no journal; a path you set wins. The shipped rule scan counts ten probes inside ten minutes from one address, reports them under category 14 (Port Scan) as port probes and bans the address like any other find. A client that retries one closed port sends at most six packets, which is why the threshold is ten. install writes the source on a fresh host as the last entry of sources; a host installed earlier keeps its configuration, and adding it is one line, followed by a restart of the watch service and a sync for the rule.

# config.yaml: the source, next to the ones the installer wrote
sources:
  - type: sshd
  - type: scan

# /etc/reportedip-agent/rules.d/50-scan.yaml: ban a scanner on this
# host, but leave the report to hosts that see more of it. Every field
# not named here stays as shipped.
format: 1
rules:
  - id: scan
    action: {ban: true, report: false}

# The same file with enabled: false switches the rule off; the firewall
# rule and its log lines stay. A threshold of your own goes into
# config.yaml and wins over the file:
thresholds:
  scan: {hits: 20, window_minutes: 10}

Adding a source the installer did not find

A source the installer did not find is simply absent from the config, and adding it is one entry with a path or a glob. Because a file with a sources key replaces the default list entirely, add your entry next to the existing ones rather than in a second block.

sources:
  - type: sshd
  - type: web
    path: /var/log/nginx/access.log

  # A second web server on a different path
  - type: web
    path: /var/log/caddy/access.log

  # Every site of a panel host, rescanned every ten minutes.
  # At most five wildcard segments.
  - type: web
    glob: /var/www/clients/*/web*/log/access.log
    exclude:
      # This site reports to the API on its own, through Hive or a
      # honeypot. Without the exclude the host sends every address
      # twice and pays twice out of the daily quota.
      - /var/log/ispconfig/httpd/honeypot.example.com/*

  # Exim on a host the installer saw as a Postfix machine
  - type: exim
    path: /var/log/exim4/mainlog

Two paths that resolve to the same file are not a problem and need no exclude: ISPConfig publishes every access log twice, under /var/www and under /var/log/ispconfig, and the agent reads such a file once. Before you trust a new entry, point reportedip-agent test at the file, then restart the watch service, then check status: the sources section shows hits per source. doctor shows a files= count per source, and a source with a count of zero resolves to nothing at all.

What leaves the machine

A report carries the address, the threat category IDs and a generated sentence. No log line, no request body, no User-Agent, no URL and no user name is ever transmitted, so a report cannot leak your customers, your paths or your credentials even by accident.

One precision, because an absolute promise would be wrong here. Two sources put a name from the log into that generated sentence: with fail2ban it is the name of the jail that banned, with imunify360 the name of the rule that fired. Both are filtered to letters, digits, dot, hyphen and underscore and cut to 32 characters before they leave the machine, because a jail name comes out of a log line and is therefore attacker-adjacent input. No other source sends anything out of a log line at all.

The hostname is sent, for one purpose only: so that you recognise your own machine in the server list instead of comparing random identifiers. It is filtered on the client to A-Za-z0-9._-, with everything else removed rather than replaced, cut to 191 characters, and it never goes out without the install ID next to it.

The identity is still the install ID. The hostname is a label and nothing more: renaming the machine changes nothing about the licence, nothing about which host holds it and nothing the server decides. You can also give a host a label of your own in your account, which is shown ahead of the hostname and is never derived from the machine.

Four addresses are never reported and never blocked, and this is a boundary in the code rather than a setting: private and reserved ranges, the loopback, every address of this server's own interfaces and its default gateway, and everything in your whitelist. The install-time SSH client address sits in the auto-whitelist, which counts as part of that last group.

Rules

A rule says which lines of one source are hits, where the address stands in such a line, what a hit is called, and how many of them inside how many minutes become a ban or a report. Every detector of the agent is a rule, and every rule is a file. Three layers are read and merged by rule id:

The pack wins over the binary, your files win over the pack. An override names a rule by its id and only the fields to change; every field it does not name keeps the value of the layer below. match and examples are replaced as a whole, action field by field. enabled: false switches a rule off. The file name plays no part: a file called 10-sshd.yaml under rules.d does not shadow the shipped file of that name, it is read like any other and merged by the ids it contains, so a release that renames a shipped file cannot turn your override into a duplicate.

The file format

One file holds any number of rules. This is a complete one, taken from the agent's own test suite:

format: 1
rules:
  - id: my-sshd
    source: sshd
    match:
      regex: '(Failed password|Invalid user) .* from '
      anchors: ["Failed password", "Invalid user", "Accepted "]
      reset: 'Accepted (password|publickey) for .* from '
    addr: {after: " from ", occurrence: last}
    categories: [22, 18]
    noun: failed logins
    threshold: {hits: 5, window_minutes: 10}
    examples:
      match:
        - {line: "Sep 24 14:47:38 host sshd[1]: Failed password for root from 45.33.32.156 port 51422 ssh2", addr: 45.33.32.156}
      inject:
        - {line: "Sep 24 14:47:38 host sshd[1]: Invalid user x from 8.8.8.8 port 22 from 45.33.32.156 port 51422", addr: 45.33.32.156}
      nomatch:
        - "Sep 24 14:47:38 host sshd[1]: Connection closed by 45.33.32.156 port 1"

Why the address is not a capture group

fail2ban had this twice, CVE-2013-2178 and CVE-2009-0362, and both times it was the same mistake: the address came out of the pattern, and an attacker had put an address of their choosing into a field the pattern reached. RE2 finds the leftmost match, so in Invalid user x from 8.8.8.8 port 22 from 45.33.32.156 port 51422 a group behind the first from would ban 8.8.8.8, which is the user name the attacker typed. addr: {after: " from ", occurrence: last} takes the last one, the one sshd wrote itself. That is why a rule names a position instead of a group, and why every rule carries an inject line that proves it.

What protects you from your own rule

The examples run every time the rule is loaded, so a pattern that has stopped matching its own lines does not load. A pattern rule your layer adds or overrides bans and does not report for the first 24 hours of its content (a rule with match.builtin never does): a wrong kernel entry expires and shows in ban list, a wrong report cannot be taken back. rules status marks such a rule as young. A daemon restart starts the day again.

Two guards watch a rule while it runs. When three addresses from your whitelist reach its threshold inside ten minutes, the rule reads the wrong field, because real attackers never stand on the whitelist; it stops reporting until its file changes, and keeps banning, which the whitelist covers. When a young rule pushes more than 50 distinct addresses over its threshold within one minute, it matches every visitor and is suspended whole, bans and reports, until its content changes. An established rule is never suspended for a burst: a distributed brute force with hundreds of sources a minute is what an sshd rule exists for. A rule from the pack is watched by the burst guard like a young one, because a pack reaches every host at once.

An entry under thresholds in config.yaml keeps winning over the file, so a host you tuned loses nothing. A file that is not valid YAML, or names a newer format, is skipped alone and the others load. A rule that fails one of the checks above costs your whole layer: the shipped rules and the pack run on, status and doctor say so, and rules check names the file, the line and the id.

The rules commands

reportedip-agent rules list                 # every rule: id, source, event, kind, state, threshold, action
reportedip-agent rules show sshd            # one rule as merged, and whether it is overridden
reportedip-agent rules check                # load rules.d the way the daemon does, and say what is wrong
reportedip-agent rules check /root/my.yaml  # try a file before it is put in place
reportedip-agent rules export sshd          # print the shipped file of a rule, as the template
reportedip-agent rules status               # state, hits since the daemon started, young and suspended rules
reportedip-agent rules disable sshd         # write enabled: false for one id; enable takes it back
reportedip-agent test --rule my-sshd /var/log/auth.log   # run one rule alone over a real log

list, show, check, export and status read and print. check without a file loads your directory exactly as the daemon would; with files it loads those on top of the shipped set instead, so a file can be tried before it lies in rules.d. export prints the shipped file a rule lives in, comments included, as the starting point of an override. enable and disable are the two that write: one generated file, 90-agent-toggles.yaml, root only, rewritten on every call. test --rule runs that one rule and no compiled detector beside it, takes the source type from the rule, and reports and bans nothing, like test always does.

Writing a rule of your own

1. Start from a shipped file. reportedip-agent rules export sshd > /root/60-sshd.yaml prints the file the rule lives in, comments and examples included.
2. Cut it down, or give it a new id. For an override keep the id and only the fields to change; the file below is complete. For a rule of your own give it a new id with source, match, addr, categories and examples, like my-sshd above. A new id is a new event source, and the shipped rule keeps running beside it.
3. Check it. reportedip-agent rules check /root/60-sshd.yaml loads the file on top of the shipped set exactly as the daemon would and prints every problem with file, line and id, or one line that says ok.
4. Put it in place. install -m 644 -o root -g root /root/60-sshd.yaml /etc/reportedip-agent/rules.d/. The directory and every file in it have to belong to root and be writable by nobody else, and a symlink that points out of the directory is refused; otherwise the whole layer is not used.
5. Wait a minute. The daemon picks the change up on its own, without a restart, and the counters start afresh. reportedip-agent rules status shows the rule, marked young for its first day. reportedip-agent test --rule my-sshd /var/log/auth.log shows meanwhile what it would have caught.

# /etc/reportedip-agent/rules.d/60-sshd.yaml: the shipped sshd rule,
# stricter on this host. Everything not named here stays as shipped.
format: 1
rules:
  - id: sshd
    threshold: {hits: 3, window_minutes: 10}

Last updated: 30. September 2026 · Maintained by the ReportedIP team