Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimizing cluster initialization changing the checks for cluster-enabled flag #3158

Merged
merged 17 commits into from
Mar 12, 2024

Conversation

willianmrs
Copy link
Contributor

@willianmrs willianmrs commented Feb 21, 2024

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

In this PR, I propose an optimized approach to RedisCluster initialization, focusing on improving the speed and efficiency for clusters. The key change involves replacing the initial INFO command check for cluster mode enablement with a direct attempt to execute "CLUSTER SLOTS". This method assumes that successful execution of "CLUSTER SLOTS" implies cluster mode is active, thus streamlining the initialization process. This optimization aims to reduce startup time without compromising the integrity of cluster checks, making it particularly beneficial for environments where rapid deployment and high performance are critical.

To compare an INFO command takes an average of 2 to 3 seconds to complete and REDIS SLOTS takes only a few milliseconds. This enables the timeout being used in the constructor, not being necessary to add a bigger timeout just to initialize the cluster.

Linked issue:
#3148

Willian Moreira and others added 3 commits February 21, 2024 14:08
…-checked

change if the cluster-mode is enabled by trying run CLUSTER SLOT inst…
@willianmrs willianmrs marked this pull request as ready for review February 21, 2024 18:23
@willianmrs willianmrs marked this pull request as draft February 21, 2024 18:50
@willianmrs willianmrs marked this pull request as ready for review February 22, 2024 21:05
@willianmrs
Copy link
Contributor Author

@dvora-h Almost all GitHub Actions have been cancelled. Is there something missing in MR to execute the actions?

Another question, I noticed this same behavior in async, but I haven't made the change yet because I thought it could result in strange behavior. I was unsure whether async should have the same behavior given that it is asynchronous.

@dvora-h
Copy link
Collaborator

dvora-h commented Mar 10, 2024

@willianmrs Thanks for this PR! I like this approach.
Why do you think in async it can result in strange behavior? In my opinion the same should work for async too...

@codecov-commenter
Copy link

codecov-commenter commented Mar 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.81%. Comparing base (2f88840) to head (0b35c47).
Report is 16 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3158      +/-   ##
==========================================
- Coverage   91.84%   91.81%   -0.04%     
==========================================
  Files         128      128              
  Lines       33232    33328      +96     
==========================================
+ Hits        30523    30600      +77     
- Misses       2709     2728      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dvora-h
Copy link
Collaborator

dvora-h commented Mar 10, 2024

@willianmrs also - notice the linters failure

@willianmrs
Copy link
Contributor Author

@dvora-h great. About async, I'm working on the async cluster changes. I guess in the first look at the Async connection I understand wrong. But it does the same as the sync cluster, so you are right, the same should work for it.

And thanks about the lint, I will submit the fix with the async changes.

Willian Moreira and others added 4 commits March 10, 2024 14:13
… connection, now only the CLUSTER_SLOT, so the total commands is minus 1
optimizing async cluster creation using CLUSTER SLOT command instead …
@willianmrs willianmrs requested a review from dvora-h March 10, 2024 17:20
@dvora-h dvora-h added the maintenance Maintenance (CI, Releases, etc) label Mar 12, 2024
@dvora-h dvora-h merged commit ddff7b5 into redis:master Mar 12, 2024
46 checks passed
gerzse pushed a commit to gerzse/redis-py that referenced this pull request Jul 11, 2024
…bled flag (redis#3158)

* change if the cluster-mode is enabled by trying run CLUSTER SLOT insted of INFO

* fix typo

* fixing cluster mode is not enabled on this node tests

* remove changes on asyncio

* rename mock flag to be more consistent

* optimizing async cluster creation using CLUSTER SLOT command instead of INFO command

* fixing test. Before INFO and CLUSTER_SLOT was used for performing the connection, now only the CLUSTER_SLOT, so the total commands is minus 1

* remove dot at the end of string

* remove unecessary print from test

* fix lint problems

---------

Co-authored-by: Willian Moreira <willian.moreira@ifood.com.br>
gerzse pushed a commit that referenced this pull request Jul 11, 2024
…bled flag (#3158)

* change if the cluster-mode is enabled by trying run CLUSTER SLOT insted of INFO

* fix typo

* fixing cluster mode is not enabled on this node tests

* remove changes on asyncio

* rename mock flag to be more consistent

* optimizing async cluster creation using CLUSTER SLOT command instead of INFO command

* fixing test. Before INFO and CLUSTER_SLOT was used for performing the connection, now only the CLUSTER_SLOT, so the total commands is minus 1

* remove dot at the end of string

* remove unecessary print from test

* fix lint problems

---------

Co-authored-by: Willian Moreira <willian.moreira@ifood.com.br>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Maintenance (CI, Releases, etc)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants