AuthStore::Recover is not invalidating caches.

[ptabor]

What happened?

No repro in practice yet attempted. Adding FTR.

The method:

etcd/server/auth/store.go

Line 374 in a08d479

func (as *authStore) Recover(be AuthBackend) {

is used when a new snapshot is received over raft (e.g. server was lagging).

The method is not invalidating:

etcd/server/auth/store.go

Line 237 in a08d479

rangePermCache map[string]*unifiedRangePermissions // username -> unifiedRangePermissions

This creates risk that data server read out of the cache might be stale (e.g. permission got revoked in the 'meantime') but the cache is missing updates skipped in the WAL log application (because it loaded a fresher snapshot).

What did you expect to happen?

Reinitialize the cache.

How can we reproduce it (as minimally and precisely as possible)?

We need a test that will:

• run 3 etcd instances
• grant some permissions on ranges
• diconnect a single member
• revoke the permissions
• commit a lot of transactions (or force the disconnected member run out of log horizont)
• reconnect the member... the member is expected to download the snapshot...
• check whether the member revokes the access to the given key-range.

Anything else we need to know?

No response

Etcd version (please run commands below)

Spotted in the main tree. But the logic was rather not changed recently.

Etcd configuration (command line flags or environment variables)

No response

Etcd debug information (please run commands blow, feel free to obfuscate the IP address or FQDN in the output)

No response

Relevant log output

No response

[mitake]

@ptabor Thanks for figuring out this. I'll try to reproduce and analyze related code this weekend.

[mitake]

@ptabor I'm still trying to reproduce, but invalidating rangePermCache in authStore.Recover() is fine and good for defensive purpose. I'll open a PR for the change.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[serathius]

Fixed with #14358