External login source becomes disabled every now and then

[pixelspark]

• Gitea version (or commit ref): 1.14.2 (Docker image)
• Git version: (Whatever's included in the Docker image, does not seem to be relevant)
• Operating system: Linux (Docker)
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL)
  • No
• Log gist:

Did not find any mention in log, but this was running at production without debugging logs turned on. Can turn them on for a while if desired.

Description

For some reason, is_activated is sometimes reset to 0 for our primary login_source. Although I cannot rule out external causes completely (i.e. some weird issue with MySQL replication) my suspicion is that Gitea makes this change (e.g. when the login source is temporarily unavailable?).

Screenshots

This is what login_source looks like after the source was deactivated (the first two sources are deliberately disabled, the 'bouncer' login source is the one that becomes disabled every now and then):

[lunny]

Is there a SQL log to change the table?

[pixelspark]

Not in the current logs (I disabled detailed SQL logging a while ago). Will check logs when it happens again. Anyone an idea about places in Gitea code that might make this change? (I could then check these..)

[techknowlogick]

I suspect this is intentional on behalf of Gitea, in 1.14 there was a change that instead of failing via panic when an oidc auth source is not available on startup, Gitea would instead just disable the auth source (to at least allow the binary to start).

[pixelspark]

I suspect this is intentional on behalf of Gitea, in 1.14 there was a change that instead of failing via panic when an oidc auth source is not available on startup, Gitea would instead just disable the auth source (to at least allow the binary to start).

Hm, any idea how (or where in the code) it checks this? Why does it need OIDC auth on startup and could that check be disabled?

[vosdim-supportsquare]

I am experiencing the same issue, probably when a node restarts and gitea startup is faster than keycloak (no redundancy). So this would follow the scenario described by techknowlogick. I am also interested to know if this check can be disabled.

[zeripath]

@pixelspark @vosdim-supportsquare the disabling occurs as part of #14802.

Prior to this Gitea would simply Fatal and die.

Given that scenario disabling is clearly better.

---

I think a different approach is possible (re-attempt registration at time of login) but it means we will need to add locking to our oauth2 provider implementation otherwise there will be serious data races. (Some of which are already present but this means that they're actually more likely to cause trouble.)

I am currently in the middle of refactoring the login sources for 1.16 so having now seen these races I will be able to add changes in to ameliorate them - but backporting will be difficult.

[zeripath]

I would also note that no-one has told us the contents of the Critical log entry when Gitea explains that it is disabling the login source.

It would be helpful to see it.