Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In cluster, if connection was never established correctly it will never be established #2225

Closed
bironran opened this issue Aug 14, 2020 · 5 comments

Comments

@bironran
Copy link

Expected behavior

In redis cluster, if connection is defined with a set of of HostAndPort, even if it cannot be established during initialization, if it can be established later it will be

Actual behavior

Connection is never established.

JedisClusterConnectionHandler.renewSlotCache calls JedisClusterInfoCache.renewClusterSlots(null) which in turn makes JedisClusterInfoCache look at its internal Map<String, JedisPool> nodes field for information on how to renew the connection pool. However, if a connection was never successfully established before (e.g. if Redis was down during startup of the application using Jedis), this nodes field will remain empty. This causes the Jedis never recover from that type of error.

Steps to reproduce:

Start Jedis cluster connection pointing to a a redis host that is currently down.
Set a loop calling any Jedis method (e.g. get).
After a few seconds in the loop, start the redis server.

Connection is never actually established.

@bironran
Copy link
Author

Workaround: in your own code check if connectionHandler.getNodes().isEmpty(). If it is, call connectionHandler.renewSlotCache(connectionHandler.getConnectionFromNode(hostAndPort)); (where hostAndPort are the initial ones). Do this in a loop, e.g. in a Timer.scheduleAtFixedRate.

@sazzad16 sazzad16 added this to the 3.7.0 milestone Apr 7, 2021
@giladam
Copy link

giladam commented May 5, 2021

@bironran Thank you for providing the workaround. I'm trying to implement a similar workaround when using Jedis under Spring Data Redis. But I can't seem to figure out any way to get a handle on that connectionHandler instance.

I don't suppose anyone might know of a way to implement such a workaround when the connections are being handled via the Spring Data Redis library which doesn't seem to provide a way to access this connectionHandler?

@bironran
Copy link
Author

bironran commented May 5, 2021

@giladam sorry, not using Spring Data Redis. but I imagine you can sublcass JedisConnectionFactory to return a subclasses JedisCluster that has access to the protected field BinaryJedisCluster.connectionHandler.

@giladam
Copy link

giladam commented May 5, 2021

@bironran Thank you! Subclassing JedisCluster and getting the JedisClusterConnectionHandler from inside there definitely works!

@sazzad16 sazzad16 modified the milestones: 3.7.0, 3.8.0 Aug 7, 2021
@sazzad16 sazzad16 modified the milestones: 3.8.0, 4.1.0 Dec 3, 2021
@sazzad16 sazzad16 removed this from the 4.1.0 milestone Jan 14, 2022
@sazzad16
Copy link
Collaborator

Resolved by #2721 & #2643

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants