Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HttpLogger: Log redirects #11581

Merged
merged 1 commit into from
Apr 10, 2024
Merged

HttpLogger: Log redirects #11581

merged 1 commit into from
Apr 10, 2024

Conversation

TheOneRing
Copy link
Member

@TheOneRing TheOneRing commented Apr 8, 2024

During a debugging session, I discovered that the client can send the same request multiple times if the redirect is directly handled by Qt.
This wasn't discovered previously as we disallow redirects for most requests.

24-04-08 16:59:13:604 [ info sync.httplogger ]:	REQUEST 697f9664-e926-4569-be6c-5c2c228010cf {"request":{"body":{"length":0},"header":{"Accept":"*/*","Accept-Language":"en_DE","Authorization":"Basic [redacted]","Original-Request-ID":"697f9664-e926-4569-be6c-5c2c228010cf","User-Agent":"Mozilla/5.0 (Windows) mirall/6.0.0-git (ownCloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)","X-Request-ID":"697f9664-e926-4569-be6c-5c2c228010cf"},"info":{"cached":false,"id":"697f9664-e926-4569-be6c-5c2c228010cf","method":"GET","url":"https://download.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"}}}
24-04-08 16:59:13:746 [ info sync.httplogger ]:	REQUEST 697f9664-e926-4569-be6c-5c2c228010cf {"request":{"body":{"length":0},"header":{"Accept":"*/*","Accept-Language":"en_DE","Authorization":"Basic [redacted]","Original-Request-ID":"697f9664-e926-4569-be6c-5c2c228010cf","User-Agent":"Mozilla/5.0 (Windows) mirall/6.0.0-git (ownCloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)","X-Request-ID":"697f9664-e926-4569-be6c-5c2c228010cf"},"info":{"cached":false,"id":"697f9664-e926-4569-be6c-5c2c228010cf","method":"GET","redirects":["https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"],"url":"https://download.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"}}}
24-04-08 16:59:13:864 [ info sync.httplogger ]:	REQUEST 697f9664-e926-4569-be6c-5c2c228010cf {"request":{"body":{"length":0},"header":{"Accept":"*/*","Accept-Language":"en_DE","Authorization":"Basic [redacted]","Original-Request-ID":"697f9664-e926-4569-be6c-5c2c228010cf","User-Agent":"Mozilla/5.0 (Windows) mirall/6.0.0-git (ownCloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)","X-Request-ID":"697f9664-e926-4569-be6c-5c2c228010cf"},"info":{"cached":false,"id":"697f9664-e926-4569-be6c-5c2c228010cf","method":"GET","redirects":["https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi","https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi/"],"url":"https://download.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"}}}
24-04-08 16:59:13:897 [ info sync.httplogger ]:	RESPONSE 697f9664-e926-4569-be6c-5c2c228010cf {"response":{"body":{"length":0},"header":{"Authorization":"Basic [redacted]","Content-Type":"text/html; charset=utf-8","Date":"Mon, 08 Apr 2024 14:59:01 GMT","Referrer-Policy":"strict-origin-when-cross-origin","Server":"Caddy","Strict-Transport-Security":"max-age=315360000; preload","Transfer-Encoding":"chunked","X-Content-Type-Options":"nosniff","X-Frame-Options":"SAMEORIGIN","X-Xss-Protection":"0"},"info":{"id":"697f9664-e926-4569-be6c-5c2c228010cf","method":"GET","redirects":["https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi","https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi/"],"reply":{"cached":false,"duration":293,"durationString":"duration(0h, 0min, 0s, 293ms)","error":"Error transferring https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi/ - server replied: Not Found","status":404,"version":"HTTP 2"},"url":"https://download.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"}}}
{
  "request": {
    "body": {
      "length": 0
    },
    "header": {
      "Accept": "*/*",
      "Accept-Language": "en_DE",
      "Authorization": "Basic [redacted]",
      "Original-Request-ID": "697f9664-e926-4569-be6c-5c2c228010cf",
      "User-Agent": "Mozilla/5.0 (Windows) mirall/6.0.0-git (ownCloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)",
      "X-Request-ID": "697f9664-e926-4569-be6c-5c2c228010cf"
    },
    "info": {
      "cached": false,
      "id": "697f9664-e926-4569-be6c-5c2c228010cf",
      "method": "GET",
      "redirects": [
        "https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi",
        "https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi/"
      ],
      "url": "https://download.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"
    }
  }
},
{
  "response": {
    "body": {
      "length": 0
    },
    "header": {
      "Authorization": "Basic [redacted]",
      "Content-Type": "text/html; charset=utf-8",
      "Date": "Mon, 08 Apr 2024 14:59:01 GMT",
      "Referrer-Policy": "strict-origin-when-cross-origin",
      "Server": "Caddy",
      "Strict-Transport-Security": "max-age=315360000; preload",
      "Transfer-Encoding": "chunked",
      "X-Content-Type-Options": "nosniff",
      "X-Frame-Options": "SAMEORIGIN",
      "X-Xss-Protection": "0"
    },
    "info": {
      "id": "697f9664-e926-4569-be6c-5c2c228010cf",
      "method": "GET",
      "redirects": [
        "https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi",
        "https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi/"
      ],
      "reply": {
        "cached": false,
        "duration": 293,
        "durationString": "duration(0h, 0min, 0s, 293ms)",
        "error": "Error transferring https://attic.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi/ - server replied: Not Found",
        "status": 404,
        "version": "HTTP 2"
      },
      "url": "https://download.owncloud.com/desktop/ownCloud/daily/6.0/win/ownCloud-6.0.0.12944-daily20231221.x64.msi"
    }
  }
}

@TheOneRing
Copy link
Member Author

@jesmrec @felix-schwarz how are redirects logged on Android and ios?

@TheOneRing TheOneRing requested a review from erikjv April 8, 2024 15:02
@jesmrec
Copy link

jesmrec commented Apr 9, 2024

straight example of redirect: ocis.owncloud.works -> ocis.ocis-traefik.owncloud.works (301) logged on Android. Let us know whether this is the expected:

D: 2024-04-09 08:31:25:253(OwnCloudClient.java:129)Executing in request with id 357fb69c-25bb-4efc-8879-63268e8748fa
D: 2024-04-09 08:31:25:489(LogInterceptor.kt:67)REQUEST 357fb69c-25bb-4efc-8879-63268e8748fa  
{
  "request": {
    "headers": {
      "X-Request-ID": "357fb69c-25bb-4efc-8879-63268e8748fa",
      "User-Agent": "Mozilla/5.0 (Android) ownCloud-android/4.2.1",
      "Accept-Language": "en",
      "Accept-Encoding": "identity",
      "Host": "ocis.owncloud.works",
      "Connection": "Keep-Alive"
    },
    "info": {
      "id": "357fb69c-25bb-4efc-8879-63268e8748fa",
      "method": "GET",
      "url": "https://ocis.owncloud.works/.well-known/webfinger?rel=http%3A%2F%2Fopenid.net%2Fspecs%2Fconnect%2F1.0%2Fissuer&resource=https%3A%2F%2Focis.owncloud.works"
    }
  }
}
D: 2024-04-09 08:31:25:503(LogInterceptor.kt:133)RESPONSE 357fb69c-25bb-4efc-8879-63268e8748fa 
{
  "response": {
    "headers": {
      "Date": "Tue, 09 Apr 2024 06:31:25 GMT",
      "Transfer-Encoding": "chunked",
      "Connection": "keep-alive",
      "Cache-Control": "max-age=3600",
      "Expires": "Tue, 09 Apr 2024 07:31:25 GMT",
      "Location": "https://ocis.ocis-traefik.latest.owncloud.works/.well-known/webfinger?rel=http%3A%2F%2Fopenid.net%2Fspecs%2Fconnect%2F1.0%2Fissuer&resource=https%3A%2F%2Focis.owncloud.works",
      "Report-To": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=NzYQonBeZb7MtQ0WklXADwW%2Fo3jzRHYjXnQcHlbBagmk2%2FOb2KFNLRZa9uwH%2FhrawCz6C4vxRHkpmavjkjmzaeIW2J%2B0fgGT3Geeq%2FYXVV3D08m45LDiIxfrq75ezU511FjrYGu9\"}],\"group\":\"cf-nel\",\"max_age\":604800}",
      "NEL": "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}",
      "Vary": "Accept-Encoding",
      "Server": "cloudflare",
      "CF-RAY": "871874403ae22162-MAD",
      "alt-svc": "h3=\":443\"; ma=86400"
    },
    "info": {
      "id": "357fb69c-25bb-4efc-8879-63268e8748fa",
      "method": "GET",
      "reply": {
        "cached": false,
        "duration": 119,
        "durationString": "duration(0h, 0min, 0s, 119ms)",
        "status": 301,
        "version": "http/1.1"
      },
      "url": "https://ocis.owncloud.works/.well-known/webfinger?rel=http%3A%2F%2Fopenid.net%2Fspecs%2Fconnect%2F1.0%2Fissuer&resource=https%3A%2F%2Focis.owncloud.works"
    }

@felix-schwarz
Copy link

Same example for the iOS app:

  • HTTP request and response in JSON format:
2024-04-09 09:04:31.849000+0200 ownCloud[67026:12884642] ⚪️ | [HTTP, Request, …] REQUEST 4679621F-5409-4567-884F-167CC2EE7D95 {"request":{"body":{},"info":{"id":"4679621F-5409-4567-884F-167CC2EE7D95","method":"GET","url-session-task-id":5,"url":"https:\/\/ocis.owncloud.works\/.well-known\/webfinger?resource=https:\/\/ocis.owncloud.works\/"},"header":{"Original-Request-ID":"4679621F-5409-4567-884F-167CC2EE7D95","X-Request-ID":"4679621F-5409-4567-884F-167CC2EE7D95","User-Agent":"ownCloudApp\/12.1 (App\/288; iPadOS\/17.2; iPad)"}}} [… GET, RequestID:4679621F-5409-4567-884F-167CC2EE7D95, URLSessionTaskID:5] [OCHTTPPipeline.m:1238|FULL]
2024-04-09 09:04:32.212000+0200 ownCloud[67026:12884642] ⚪️ | [HTTP, Response, …] RESPONSE 4679621F-5409-4567-884F-167CC2EE7D95 {"response":{"body":{},"info":{"id":"4679621F-5409-4567-884F-167CC2EE7D95","method":"GET","reply":{"status":301,"metrics":"{ total: [2024-04-09 07:04:31 +0000 - 2024-04-09 07:04:32 +0000, 0.36 sec], startedAfter: 0.00, redirects: 0, transactions: [1: fetchStart: 0.00, DNS: 0.01..0.02 (0.00), connect: 0.02..0.25 (0.23), TLS: 0.13..0.25 (0.12), request: 0.24..0.24 (0.00), cloud: 0.24..0.35 (0.11), response: 0.35..0.36 (0.00)] }","status-name":"MOVED PERMANENTLY"},"url-session-task-id":5,"url":"https:\/\/ocis.owncloud.works\/.well-known\/webfinger?resource=https:\/\/ocis.owncloud.works\/"},"header":{"cf-ray":"8718a4c24e674372-EWR","Server":"cloudflare","report-to":"{\"endpoints\":[{\"url\":\"https:\\\/\\\/a.nel.cloudflare.com\\\/report\\\/v4?s=cjxgoCbDw7Mf1cDWYV3InQy%2B5%2FJ3BhKzKDbTyfLcduuApmJtB1DBm0WbsNgdRDhSriifEm0cMeQm69OmukmbeOjnmpAQruG7t2V3Pr2ut%2BYFjxp3mhQv3D7NQuyO%2BWKn7ZLlXCv0VySRBqRO9fm5dZu1\"}],\"group\":\"cf-nel\",\"max_age\":604800}","Cache-Control":"max-age=3600","nel":"{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}","Expires":"Tue, 09 Apr 2024 08:04:32 GMT","Date":"Tue, 09 Apr 2024 07:04:32 GMT","Location":"https:\/\/ocis.ocis-traefik.latest.owncloud.works\/.well-known\/webfinger?resource=https:\/\/ocis.owncloud.works\/","Alt-Svc":"h3=\":443\"; ma=86400","Vary":"Accept-Encoding"}}} [… GET, RequestID:4679621F-5409-4567-884F-167CC2EE7D95, URLSessionTaskID:5] [OCHTTPPipeline.m:1433|FULL]
  • HTTP request and response in plain text format, including an additional one-line summary (tagged HTSum)
2024-04-09 09:01:44.010000+0200 ownCloud[66744:12879199] ⚪️ | [HTTP, Request, …] Sending request:
# REQUEST ---------------------------------------------------------
URL:         https://ocis.owncloud.works/.well-known/webfinger?resource=https://ocis.owncloud.works/
Error:       -
Req Signals: (null)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
GET /.well-known/webfinger HTTP/1.1
Host: ocis.owncloud.works
[Redirect Policy: handle locally]
Original-Request-ID: EE35022C-F0E7-4F4B-BCDE-F0404450A3FB
X-Request-ID: EE35022C-F0E7-4F4B-BCDE-F0404450A3FB
User-Agent: ownCloudApp/12.1 (App/288; iPadOS/17.2; iPad)
----------------------------------------------------------------- [… GET, RequestID:EE35022C-F0E7-4F4B-BCDE-F0404450A3FB, URLSessionTaskID:5] [OCHTTPPipeline.m:1248|FULL]
2024-04-09 09:01:44.400000+0200 ownCloud[66744:12879199] ⚪️ | [HTTP, Local, …] <- 301 MOVED PERMANENTLY (GET https://ocis.owncloud.works/.well-known/webfinger?resource=https://ocis.owncloud.works/) -> https://ocis.ocis-traefik.latest.owncloud.works/.well-known/webfinger?resource=https://ocis.owncloud.works/  [… PipelineID:ephermal, Instance:0x148ad25a0, HTTP, Response, GET, RequestID:EE35022C-F0E7-4F4B-BCDE-F0404450A3FB, URLSessionTaskID:5, HTSum] [OCHTTPPipeline.m:1442|FULL]
2024-04-09 09:01:44.400000+0200 ownCloud[66744:12879199] ⚪️ | [HTTP, Response, …] Received response:
# RESPONSE --------------------------------------------------------
Method:      GET
URL:         https://ocis.owncloud.works/.well-known/webfinger?resource=https://ocis.owncloud.works/
Request-ID:  EE35022C-F0E7-4F4B-BCDE-F0404450A3FB
Error:       -
Req Signals: (null)
Metrics:     { total: [2024-04-09 07:01:44 +0000 - 2024-04-09 07:01:44 +0000, 0.39 sec], startedAfter: 0.00, redirects: 0, transactions: [1: fetchStart: 0.00, DNS: 0.02..0.02 (0.00), connect: 0.02..0.27 (0.25), TLS: 0.13..0.27 (0.14), request: 0.27..0.27 (0.00), cloud: 0.27..0.38 (0.11), response: 0.38..0.39 (0.01)] }
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
301 MOVED PERMANENTLY
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=oQWJYqzpqgU867Y3GIDTLtlVfGf%2BZ2dQV%2FRjqQCjK6KpZ6MBJ6UJgWCNaARRv%2FXoVXtW6Bdvxgw6B1UzHDjF8s6XRDzkzKKJp03ETldSAJu9Fpqbz%2BF3of1uwgTbzOw8Ng37cXwvo%2BFdBNQWSOy3xezD"}],"group":"cf-nel","max_age":604800}
Location: https://ocis.ocis-traefik.latest.owncloud.works/.well-known/webfinger?resource=https://ocis.owncloud.works/
Server: cloudflare
cf-ray: 8718a0a96b2941e3-EWR
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Expires: Tue, 09 Apr 2024 08:01:44 GMT
Date: Tue, 09 Apr 2024 07:01:44 GMT
Alt-Svc: h3=":443"; ma=86400
Cache-Control: max-age=3600
Vary: Accept-Encoding
----------------------------------------------------------------- [… GET, RequestID:EE35022C-F0E7-4F4B-BCDE-F0404450A3FB, URLSessionTaskID:5] [OCHTTPPipeline.m:1443|FULL]

The iOS app also alerts the user to the redirect and asks the user to confirm it:

Simulator Screenshot - iPad Air (5th generation) - 2024-04-09 at 09 06 01

@TheOneRing
Copy link
Member Author

Thx, so it's the location header (I don't get to see it if I tell Qt to follow redirect_no_less_secure).
The update download here is a real outlier.

@TheOneRing TheOneRing merged commit 15913ee into master Apr 10, 2024
6 of 7 checks passed
@delete-merged-branch delete-merged-branch bot deleted the work/log_redirect branch April 10, 2024 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants