Torutils

Few tools for a Tor relay.

Block DDoS

The scripts ipv4-rules.sh and ipv6-rules.sh protect a Tor relay against DDoS attacks¹ at the IP network layer, as seen in this metrics:

An older example is here.

¹ see ticket 40636 and 40093 of the Tor project.

Idea

Mark an ip as malicious if its connection attempts over a short time period exceed a given threshold. Block that ip for a much longer time period.

Therefore a simple network rule won't make it. However ipset helps to achieve the goal.

Further considerations:

• never touch established connections
• try to not overblock

Quick start

Install jq, ipset and iptables, e.g. for Ubuntu 22.04:

sudo apt update
sudo apt install -y jq ipset iptables

Download the script

wget -q https://raw.githubusercontent.com/toralf/torutils/main/ipv4-rules.sh -O ipv4-rules.sh
chmod +x ./ipv4-rules.sh

Make a backup of the current iptables filter table:

sudo /usr/sbin/iptables-save > ./rules.v4
sudo /usr/sbin/ip6tables-save > ./rules.v6

Run a quick test

sudo ./ipv4-rules.sh test

Best is to stop the Tor service(s) now. Flush the connection tracking table

sudo /usr/sbin/conntrack -F

and (re-)start the Tor service. Check that your ssh login and other services are still working. Watch the iptables live statistics by:

sudo watch -t ./ipv4-rules.sh

If something failed then restore the backuped state:

sudo ./ipv4-rules.sh stop
sudo /usr/sbin/iptables-restore < ./rules.v4
sudo /usr/sbin/ip6tables-restore < ./rules.v6

Otherwise, if all is fine then run the script again, but with the parameter start:

sudo ./ipv4-rules.sh start

Finally, create cron jobs (via crontab -e) like these:

# DDoS prevention
@reboot /root/ipv4-rules.sh start; /root/ipv6-rules.sh start

# keep ips during reboot
@hourly /root/ipv4-rules.sh save; /root/ipv6-rules.sh save

# update Tor authorities
@daily  /root/ipv4-rules.sh update; /root/ipv6-rules.sh update

Ensure, that the package iptables-persistent is either de-installed or at least disabled.

More hints are in the Installation section. I do appreciate issue reports and GitHub PR.

That's all.

Details

Generic filter rules for the local network, ICMP, ssh, DHCP and additional services are created. Then the following rules are applied:

1. trust connection attempt to any port from trusted Tor authorities/Snowflake servers
2. block the source¹ for 24 hours if the connection attempt rate to the ORPort exceeds > 9/min² within last 2 minutes
3. ignore the connection attempt if there are already 9 established connections to the ORPort
4. accept the connection attempt to the ORPort

¹ for IPv4 the "source" is a regular ip, for IPv6 the corresponding /80 CIDR block

² the value is derived from ticket 40636

Basically just these rules were be implemented, for ipv4 here, the rest of the script deals with all the stuff around that.

Installation

If the parsing of the Tor and/or of the SSH config fails then:

1. define the local running relay/s explicitly at the command line after the keyword start, e.g.:

   sudo ./ipv4-rules.sh start 1.2.3.4:443 5.6.7.8:9001

2. -or- define them as environment variables, e.g.:

   sudo CONFIGURED_RELAYS="5.6.7.8:9001 1.2.3.4:443" ./ipv4-rules.sh start

   (CONFIGURED_RELAYS6 for IPv6).

A command line argument takes precedence over its environment variable. The same syntax is sued to allow inbound traffic to additional address:port destinations, e.g.:

export ADD_LOCAL_SERVICES="2.71.82.81:828 3.141.59.26:53"

(ADD_LOCAL_SERVICES6 appropriately) before running the script.

A slightly different syntax can be used for ADD_REMOTE_SERVICES and its IPv6 variant to allow inbound traffic, e.g.:

export ADD_LOCAL_SERVICES="4.3.2.1>4711"

allows traffic, i.e. from the remote address "4.3.2.1" to the local port "4711".

The script sets few sysctl values. If unwanted then please comment out the call of setSysctlValues(). If Hetzners system monitor isn't used, then comment out addHetzner(). To append rules onto existing iptables rule set (overwrite is the default) please comment out the call clearRules().

Metrics

The script metrics.sh exports DDoS metrics in a Prometheus-formatted file. The scrape of it is handled by node_exporter. More details plus few Grafana dashboards are here.

DDoS examples

Graphs¹ of rx/tx packets, traffic and socket counts from 5th, 6th and 7th of Nov show the results for few DDoS attacks over 3 days for 2 relays. A more heavier attack was observed at 12th of Nov. A periodic drop down of the socket count metric, vanishing over time, appeared at 5th of Dec. Current attacks e.g. at the 7th of March are still handled well. Few more helper scripts were developed to analyze the attack vector. Look here for details.

¹ using sysstat

More

I used this Ansible role to deploy and configure Tor relays (server, bridges, snowflake).

Query Tor via its API

Relay summary

info.py gives a summary of all connections, e.g.:

sudo ./info.py --address 127.0.0.1 --ctrlport 9051

 ORport 9051  0.4.8.0-alpha-dev   uptime: 02:58:04   flags: Fast, Guard, Running, Stable, V2Dir, Valid
+------------------------------+-------+-------+
| Type                         |  IPv4 |  IPv6 |
+------------------------------+-------+-------+
| Inbound to our OR from relay |  2304 |   885 |
| Inbound to our OR from other |  3188 |    68 |
| Inbound to our ControlPort   |       |     1 |
| Outbound to relay OR         |  2551 |   629 |
| Outbound to relay non-OR     |       |       |
| Outbound exit traffic        |       |       |
| Outbound unknown             |    16 |     4 |
+------------------------------+-------+-------+
| Total                        |  8059 |  1587 |
+------------------------------+-------+-------+
 relay OR connections  6369
 relay OR ips          5753
    3 inbound v4 with > 2 connections each

Watch Tor Exit connections

If your Tor relay is running as an Exit then ps.py gives live statistics:

sudo ./ps.py --address 127.0.0.1 --ctrlport 9051

Tor circuit closings

orstatus.py prints the closing reason to stdout, orstatus-stats.sh prints/plots statistics (see this example) from that.

orstatus.py --ctrlport 9051 --address ::1 >> /tmp/orstatus &
sleep 3600
orstatus-stats.sh /tmp/orstatus

Check expiration of Tor offline keys

key-expires.py helps to maintain Tor offline keys. It returns the expiration time in seconds of the mid-term signing key, a cronjob could be sth. like this:

seconds=$(sudo ./key-expires.py /var/lib/tor/keys/ed25519_signing_cert)
days=$((seconds / 86400))
[[ $days -lt 23 ]] && echo "Tor signing key expires in less than $days day(s)"

If Tor metrics are enabled then this 1-liner does the similar job (maybe replace 9052 with the metrics port):

date -d@$(curl -s localhost:9052/metrics | grep "^tor_relay_signing_cert_expiry_timestamp" | awk '{ print $2 }')

Prerequisites

An open Tor control port is needed to query the Tor process via API. Configure it in torrc, e.g.:

ControlPort 127.0.0.1:9051
ControlPort [::1]:9051

The python library Stem is needed. Install it either by your package manager -or- use the git sources, e.g.:

git clone https://github.com/torproject/stem.git
export PYTHONPATH=$PWD/stem

Search logs for pre-defined text patterns

The script watch.sh helps to constantly monitor the host and Tor log files. It sends findings via mailx.

log=/tmp/${0##*/}.log

# watch syslog
/opt/torutils/watch.sh /var/log/messages /opt/torutils/watch-messages.txt &>>$log &
# watch Tor
/opt/torutils/watch.sh /var/log/tor/notice.log /opt/torutils/watch-tor.txt -v &>>$log &