"Federation denied" when looking at #homeowners:matrix.org

[fridtjof]

Description

When opening the room #homeowners:matrix.org in Element, messages roughly before $v4CpP6vDtZyyLjl6ymWbpZ4AqQTKuTEkt2w_OIYmiiI will never appear (infinite loading spinner). An API error can be observed in the developer console:

Request:
GET https://das-labor.org/_matrix/client/r0/rooms/!iyIlInqJyxXrRmRHFx%3Amatrix.org/messages?from=t6161-2537059_84961551_18160_4388417_195188_2863_117383_5368163_104&limit=20&dir=b&filter=%7B%22lazy_load_members%22%3Atrue%7D

(takes a few seconds, so the homeserver is doing something - maybe actively fetching data from some other homeserver that denies federation?)

Response:
403 with the following body:
{"errcode":"M_FORBIDDEN","error":"Federation denied with das-labor.org."}

This response is not documented in the spec for that endpoint, so I assume it comes from something synapse is doing to fulfill that request?

@moe:moritzdietz.com also encountered the same problem on their homeserver: https://matrix.to/#/!mjbDjyNsRXndKLkHIe:matrix.org/$L-XfR1fgWCCqewMIVFYkWD49cOhc4UpFz2V12CoZbHI?via=matrix.org&via=privacytools.io&via=tchncs.de

Steps to reproduce

(I am not sure of the conditions needed to trigger this behavior)

• view the room in Element (should be the same for any client, I haven't tested that though)

Logs

Unfortunately, I could not find any relevant Synapse logs at loglevel INFO. DEBUG seemed to make Synapse unstable somehow.

Version information

• Homeserver:
  das-labor.org

• Version:
  v1.19.1

• Install method:
  Docker

• Platform:
  Docker on Ubuntu 20.04

[erikjohnston]

This sounds like you server is sending out a /backfill request to a server that has turned off federation and synapse isn't correctly handling that. I'd be interested to see the logs for /backfill requests to see what servers its trying to hit?

[erikjohnston]

There seems to be various reports of this (and other errors related to the room), so this is unlikely to be a problem with das-labor.org

[fridtjof]

I managed to collect some valuable looking logs:

synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913954} [matrix.org] Completed request: 200 OK in 0.21 secs - GET matrix://matrix.org/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from matrix.org because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913955} [maunium.net] Completed request: 200 OK in 0.15 secs - GET matrix://maunium.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from maunium.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913956} [archlinux.org] Completed request: 200 OK in 0.13 secs - GET matrix://archlinux.org/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from archlinux.org because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913958} [msrd0.de] Completed request: 200 OK in 0.95 secs - GET matrix://msrd0.de/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from msrd0.de because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913962} [logic.net] Completed request: 200 OK in 1.27 secs - GET matrix://logic.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from logic.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913963} [cl0secall.net] Completed request: 200 OK in 1.97 secs - GET matrix://cl0secall.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from cl0secall.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913964} [ssweeny.net] Completed request: 200 OK in 0.65 secs - GET matrix://ssweeny.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from ssweeny.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913965} [orgizm.net] Completed request: 200 OK in 0.18 secs - GET matrix://orgizm.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from orgizm.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913966} [faraone.cc] Completed request: 200 OK in 1.07 secs - GET matrix://faraone.cc/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from faraone.cc because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913967} [ks.matfyz.cz] Completed request: 200 OK in 0.24 secs - GET matrix://ks.matfyz.cz/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from ks.matfyz.cz because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913968} [schwifty.net] Completed request: 200 OK in 0.73 secs - GET matrix://schwifty.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from schwifty.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913969} [thebeckmeyers.xyz] Completed request: 200 OK in 0.48 secs - GET matrix://thebeckmeyers.xyz/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from thebeckmeyers.xyz because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619229- {GET-O-1913970} [chat.usr.nz] Completed request: 200 OK in 1.53 secs - GET matrix://chat.usr.nz/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619229- Failed to backfill from chat.usr.nz because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 578 - WARNING - GET-1619229- {GET-O-1913971} [cobryce.com] Request failed: GET matrix://cobryce.com/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100: HttpResponseException('403: Forbidden')
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913972} [matrix.org] Completed request: 200 OK in 0.15 secs - GET matrix://matrix.org/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from matrix.org because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913973} [maunium.net] Completed request: 200 OK in 0.14 secs - GET matrix://maunium.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from maunium.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913974} [archlinux.org] Completed request: 200 OK in 0.10 secs - GET matrix://archlinux.org/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from archlinux.org because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913975} [msrd0.de] Completed request: 200 OK in 0.14 secs - GET matrix://msrd0.de/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from msrd0.de because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913977} [logic.net] Completed request: 200 OK in 0.67 secs - GET matrix://logic.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from logic.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913978} [cl0secall.net] Completed request: 200 OK in 1.27 secs - GET matrix://cl0secall.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from cl0secall.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913979} [ssweeny.net] Completed request: 200 OK in 0.36 secs - GET matrix://ssweeny.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from ssweeny.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913980} [orgizm.net] Completed request: 200 OK in 0.13 secs - GET matrix://orgizm.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from orgizm.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913981} [faraone.cc] Completed request: 200 OK in 0.55 secs - GET matrix://faraone.cc/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from faraone.cc because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913982} [ks.matfyz.cz] Completed request: 200 OK in 0.15 secs - GET matrix://ks.matfyz.cz/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from ks.matfyz.cz because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913983} [schwifty.net] Completed request: 200 OK in 0.41 secs - GET matrix://schwifty.net/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from schwifty.net because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913984} [thebeckmeyers.xyz] Completed request: 200 OK in 0.46 secs - GET matrix://thebeckmeyers.xyz/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from thebeckmeyers.xyz because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 201 - INFO - GET-1619379- {GET-O-1913985} [chat.usr.nz] Completed request: 200 OK in 0.62 secs - GET matrix://chat.usr.nz/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100
synapse.handlers.federation - 1067 - INFO - GET-1619379- Failed to backfill from chat.usr.nz because 403: event id $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA: unable to verify signature for sender c.a2sc.eu: 401: Invalid signature for server c.a2sc.eu with key ed25519:a_wIti: Unable to verify signature for c.a2sc.eu: <class 'nacl.exceptions.BadSignatureError'> Signature was forged or corrupt
synapse.http.matrixfederationclient - 578 - WARNING - GET-1619379- {GET-O-1913986} [cobryce.com] Request failed: GET matrix://cobryce.com/_matrix/federation/v1/backfill/%21iyIlInqJyxXrRmRHFx%3Amatrix.org?v=%24rdUm5dKr61jm4VaNM6S-5UrH6fZez8sapMVy_319Ug8&v=%240FUzf16paSXnym3KCoq8X18yZtkCZelVU7d0dcFcRbk&v=%248_A7JzSus4oAReJtBcvD_Ga82wwK8RcOzbsvnH3MQ88&v=%249Sl9nqmjH8MTvJkhvzny63XpS9a2FhdSETFIdlV6SU4&v=%24ZY_UubwLYpCh-1Urx355qLJaL7sa9lP3GAaMjBn41RA&limit=100: HttpResponseException('403: Forbidden')

Note how the failing backfill requests all reference the same event $Ent5NC35ODmD2DJSZu80LZUozKo4p2PhzWR543ZxjqA.

[anoadragon453]

This should be fixed in Synapse v1.19.3 (partial fix, you can join a room but only backfill so far) and v1.20.0rc5 (you can join a room and backfill normally).

Let us know if it still occurs after updating to those versions! 🙂

[richvdh]

seems to be a dup of #3121