Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle RESP3 sets as Python lists #3324

Merged
merged 4 commits into from
Jul 22, 2024

Conversation

gerzse
Copy link
Contributor

@gerzse gerzse commented Jul 19, 2024

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

Although the RESP3 protocol defines the set data structure, sometimes the responses from the Redis server contain sets with nested maps, which cannot be represented in Python as sets with nested dicts, because dicts are not hashable.

Versions of HIREDIS before 3.0.0 would cause segmentation fault when parsing such responses. Starting with version 3.0.0 the problem was fixed, with the compromise that RESP3 sets are represented as Python lists.

The embedded RESP3 parser was so far trying to represent RESP3 sets as Python sets, if possible. Only when this was not possible it would switch to the list representation. Arguably this is not the best user experience, not knowing when you will get back a set or a list.

Upgrade the required hiredis-py version to be at least 3.0.0, and change the embedded parser to always represent RESP3 sets as lists. This way we get a consistent experience in all cases.

This is a breaking change.

Fixes #3074 and #3145

Although the RESP3 protocol defines the set data structure, sometimes
the responses from the Redis server contain sets with nested maps, which
cannot be represented in Python as sets with nested dicts, because dicts
are not hashable.

Versions of HIREDIS before 3.0.0 would cause segmentation fault when
parsing such responses. Starting with version 3.0.0 the problem was
fixed, with the compromise that RESP3 sets are represented as Python
lists.

The embedded RESP3 parser was so far trying to represent RESP3 sets as
Python sets, if possible. Only when this was not possible it would
switch to the list representation. Arguably this is not the best user
experience, not knowing when you will get back a set or a list.

Upgrade the required hiredis-py version to be at least 3.0.0, and change
the embedded parser to always represent RESP3 sets as lists. This way we
get a consistent experience in all cases.

This is a breaking change.
@gerzse gerzse added bug Bug breakingchange API or Breaking Change labels Jul 19, 2024
@gerzse gerzse requested a review from vladvildanov July 19, 2024 14:24
@vladvildanov vladvildanov merged commit fd0b0d3 into redis:master Jul 22, 2024
56 checks passed
@kosuke-zhang
Copy link
Contributor

Hi, when will this be released?

@ThiefMaster
Copy link

Can this please be fixed by making a release? Right now you get a (partially) broken combination of dependencies when you simply install redis[hiredis]. That should never happen. While people have to pin their transitive dependencies, the current state means that even in a new application where you add redis[hiredis] and then pin the dependencies, you get a broken combination, which can only fixed by adding an explicit pin for hiredis...

@vladvildanov
Copy link
Collaborator

@ThiefMaster Hi! New release planned for next week, we will include this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breakingchange API or Breaking Change bug Bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

redis.cluster.RedisCluster with RESP3 and hiredis causes segmentation fault
4 participants