"Error: Too many requests" with trackers

[fflorent]

What happened?

I got a lot of connections this weekend for a video and this warning occurs a lot of time (172.18.0.5 being my Traefik RP):

peertube_1       | [skeptikon.fr:443] 2019-02-18 10:21:24.174 warn: Warning in tracker. {
peertube_1       |   "meta": {
peertube_1       |     "err": {
peertube_1       |       "stack": "Error: Too many requests (31 of ip 172.18.0.5 for torrent 1f8a46317005fb7abc562fd4e6197366a11111c1\n    at Server.filter [as _filter] (/app/dist/server/controllers/tracker.js:35:23)\n    at Server._onAnnounce (/app/node_modules/bittorrent-tracker/server.js:661:10)\n    at Server._onRequest (/app/node_modules/bittorrent-tracker/server.js:649:10)\n    at Server._onWebSocketRequest (/app/node_modules/bittorrent-tracker/server.js:500:8)\n    at WebSocket.socket.onMessageBound (/app/node_modules/bittorrent-tracker/server.js:467:10)\n    at emitOne (events.js:116:13)\n    at WebSocket.emit (events.js:211:7)\n    at Receiver.receiverOnMessage (/app/node_modules/ws/lib/websocket.js:741:20)\n    at emitOne (events.js:116:13)\n    at Receiver.emit (events.js:211:7)\n    at Receiver.dataMessage (/app/node_modules/ws/lib/receiver.js:417:14)\n    at Receiver.getData (/app/node_modules/ws/lib/receiver.js:347:17)\n    at Receiver.startLoop (/app/node_modules/ws/lib/receiver.js:133:22)\n    at Receiver._write (/app/node_modules/ws/lib/receiver.js:69:10)\n    at doWrite (_stream_writable.js:396:12)\n    at writeOrBuffer (_stream_writable.js:382:5)\n    at Receiver.Writable.write (_stream_writable.js:290:11)\n    at Socket.socketOnData (/app/node_modules/ws/lib/websocket.js:816:35)\n    at emitOne (events.js:116:13)\n    at Socket.emit (events.js:211:7)\n    at addChunk (_stream_readable.js:263:12)\n    at readableAddChunk (_stream_readable.js:250:11)\n    at Socket.Readable.push (_stream_readable.js:208:10)\n    at TCP.onread (net.js:601:20)",
peertube_1       |       "message": "Too many requests (31 of ip 172.18.0.5 for torrent 1f8a46317005fb7abc562fd4e6197366a11111c1"
peertube_1       |     }
peertube_1       |   },
peertube_1       |   "err": {
peertube_1       |     "stack": "Error: Too many requests (31 of ip 172.18.0.5 for torrent 1f8a46317005fb7abc562fd4e6197366a11111c1\n    at Server.filter [as _filter] (/app/dist/server/controllers/tracker.js:35:23)\n    at Server._onAnnounce (/app/node_modules/bittorrent-tracker/server.js:661:10)\n    at Server._onRequest (/app/node_modules/bittorrent-tracker/server.js:649:10)\n    at Server._onWebSocketRequest (/app/node_modules/bittorrent-tracker/server.js:500:8)\n    at WebSocket.socket.onMessageBound (/app/node_modules/bittorrent-tracker/server.js:467:10)\n    at emitOne (events.js:116:13)\n    at WebSocket.emit (events.js:211:7)\n    at Receiver.receiverOnMessage (/app/node_modules/ws/lib/websocket.js:741:20)\n    at emitOne (events.js:116:13)\n    at Receiver.emit (events.js:211:7)\n    at Receiver.dataMessage (/app/node_modules/ws/lib/receiver.js:417:14)\n    at Receiver.getData (/app/node_modules/ws/lib/receiver.js:347:17)\n    at Receiver.startLoop (/app/node_modules/ws/lib/receiver.js:133:22)\n    at Receiver._write (/app/node_modules/ws/lib/receiver.js:69:10)\n    at doWrite (_stream_writable.js:396:12)\n    at writeOrBuffer (_stream_writable.js:382:5)\n    at Receiver.Writable.write (_stream_writable.js:290:11)\n    at Socket.socketOnData (/app/node_modules/ws/lib/websocket.js:816:35)\n    at emitOne (events.js:116:13)\n    at Socket.emit (events.js:211:7)\n    at addChunk (_stream_readable.js:263:12)\n    at readableAddChunk (_stream_readable.js:250:11)\n    at Socket.Readable.push (_stream_readable.js:208:10)\n    at TCP.onread (net.js:601:20)",
peertube_1       |     "message": "Too many requests (31 of ip 172.18.0.5 for torrent 1f8a46317005fb7abc562fd4e6197366a11111c1"
peertube_1       |   }
peertube_1       | }

NB1: I have configured PEERTUBE_TRUST_PROXY and any other info logs begin with the real client IP.

NB2: Traefik seems to add the X-Forwarded-For and X-Real-IP headers by default:

$ cat docker-compose.yml 
version: '3'

services:
  reverse-proxy:
    image: traefik
    command: --api --docker
    ports:
      - "8880:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  whoami:
    image: containous/whoami
    labels:
      - "traefik.frontend.rule=Host:whoami.docker.localhost"
      - "traefik.frontend.passHostHeader=true"

$ curl -H Host:whoami.docker.localhost localhost:8880
Hostname: f79178a443ff
IP: 127.0.0.1
IP: 172.21.0.2
GET / HTTP/1.1
Host: whoami.docker.localhost
User-Agent: curl/7.63.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.21.0.1
X-Forwarded-Host: whoami.docker.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: e7b42ca8193b
X-Real-Ip: 172.21.0.1

What do you expect to happen instead?

The IP of the RP shouldn't be used for identifying the tracker request.

Additional information

• PeerTube version or URL: 1.2.1 / https://skeptikon.fr

• Browser name/version:

• NodeJS version:

• Using Docker: yes (as suggested with the docker-compose.yml)

• Link to browser console log if useful:

• Link to server log if useful (journalctl or /var/www/peertube/storage/logs/):

[fflorent]

Also I got this response from the community:

13:53:37 - tcit : fflorent: Is your reverse proxy configuration up to date with this file ?
13:53:38 - tcit : https://github.com/Chocobozzz/PeerTube/blob/develop/support/nginx/peertube#L150
13:53:38 - tcit : Especially the `Websocket tracker` part?
14:06:55 - lecygnenoir[m] : fflorent:  I am unsure of the traeffik equivalent to location, if it closes the connexion before the peer send news, it could be a intersting clue indeed 🤔
14:08:21 - fflorent : lecygnenoir[m]: your point is about this? proxy_read_timeout 1200s;
14:09:11 - lecygnenoir[m] : Yep, in the nginx configuration, it's configured only for the /tracker/socket path
14:09:48 - fflorent : I see. But… The error seems to indicate an issue with the detected IP
14:12:19 - lecygnenoir[m] : My guess is it could explain the problem you have with the p2p protocol (when multiple people are viewing the video)
14:12:20 - lecygnenoir[m] : But, it could be also related with the error itself
14:13:44 - lecygnenoir[m] : Perhaps something like the websocket closed by traeffik because of timeout, then it complains because traeffik reopen websocket between traeffik and peertube, and Peertube are waiting for news about the older connection (which Peertube does not close unlike traeffik)
14:16:17 - fflorent : I see
14:16:23 - lecygnenoir[m] : More precisely, in this case it's all about a TCP/IP connection between traeffik and Peertube, so peertube see the real IP, not any proxied one
14:16:50 - fflorent : I don't have time right now (sorry, I have other urgent priorities), but I guess I should take a look
14:17:04 - lecygnenoir[m] : If possible, try to increase the whole timeout in traeffik to check ^^

What do you think?

[Chocobozzz]

See traefik/traefik#4352

So try to add network_mode: "host" in the reverse-proxy section.

[Chocobozzz]

Fixed in f2e0cca & 31b6ddf