Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secret rotation failing after LoginTokenExpiredEvent #867

Closed
benallaerts opened this issue Jun 6, 2024 · 2 comments
Closed

Secret rotation failing after LoginTokenExpiredEvent #867

benallaerts opened this issue Jun 6, 2024 · 2 comments
Assignees
Labels
type: regression A regression from a previous release
Milestone

Comments

@benallaerts
Copy link

benallaerts commented Jun 6, 2024

Hey,

Problem
I'm trying to use spring-vault in my application with dynamic secret rotation for the database.

All secret renewals and rotations are working fine until the login token expires. On token expiration all secrets are correctly rotated, but on the next renewal the following exception is thrown

org.springframework.vault.VaultException: Cannot renew lease
	at org.springframework.vault.core.lease.SecretLeaseContainer.doRenewLease(SecretLeaseContainer.java:766)
	at org.springframework.vault.core.lease.SecretLeaseContainer.renewAndSchedule(SecretLeaseContainer.java:644)
	at org.springframework.vault.core.lease.SecretLeaseContainer.lambda$scheduleLeaseRenewal$4(SecretLeaseContainer.java:636)
	at org.springframework.vault.core.lease.SecretLeaseContainer$LeaseRenewalScheduler$1.run(SecretLeaseContainer.java:937)
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
	at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:96)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException: Cannot invoke "org.springframework.vault.core.lease.SecretLeaseContainer$LeaseRenewalScheduler.isLeaseRenewable(org.springframework.vault.core.lease.domain.Lease, org.springframework.vault.core.lease.domain.RequestedSecret)" because "renewalScheduler" is null
	at org.springframework.vault.core.lease.SecretLeaseContainer.doStart(SecretLeaseContainer.java:408)
	at org.springframework.vault.core.lease.SecretLeaseContainer.onLeaseExpired(SecretLeaseContainer.java:810)
	at org.springframework.vault.core.lease.SecretLeaseContainer.doRenewLease(SecretLeaseContainer.java:742)
	... 11 more

The void restartSecrets() method in the SecretLeaseContainer seems to cancel and clear all stored renewals on login token expiration, but the newly created LeaseRenewalScheduler used to rotate the secret is never added to the stored renewals. This seem to cause the above exception when the next rotation is attempted.

Setup

framework version
java 21
spring 6.1.6
spring-boot 3.2.5
spring-vault 3.1.1
Hashicorp Vault 1.16.3

Authentication is done using APPROLE credentials

mp911de added a commit that referenced this issue Jun 6, 2024
Secret renewals are resumed after restarting secrets by re-registering LeaseRenewalScheduler.

Also, guard concurrent renewals that might have been in progress while the renewal scheduler has been in progress.

Closes gh-867
@mp911de mp911de closed this as completed in 14626b1 Jun 6, 2024
@mp911de mp911de added the type: regression A regression from a previous release label Jun 6, 2024
@mp911de mp911de added this to the 3.1.2 milestone Jun 6, 2024
@mp911de
Copy link
Member

mp911de commented Jun 6, 2024

That's fixed now.

@mp911de mp911de self-assigned this Jun 6, 2024
@sebastianreloaded
Copy link

sebastianreloaded commented Jul 31, 2024

Since this is a big Problem with long living services, is there an ETA for Release 3.1.2 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: regression A regression from a previous release
Projects
None yet
Development

No branches or pull requests

3 participants