Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pagination bug when only one field is selected and sorted at the same time #593

Merged
merged 7 commits into from
Nov 16, 2023

Conversation

adzialocha
Copy link
Member

@adzialocha adzialocha commented Nov 15, 2023

This PR fixes the issue we had when pagination suddenly broke for a query over one single sorted field.

We usually order by:

  1. The field which was selected by the user to-be-ordered
  2. document view id (except of when it was previously selected)
  3. Cursor

Two things are important in this context:

  • In SQL the ordering is only applied when the previous ordering stumbled upon duplicates
  • The pagination logic needs to follow the same ordering to correctly paginate over it

When multiple fields have been selected by the user and we have duplicate values:

  • We sort by documents (since we have 2 or more fields per document and we need to group them)
  • .. And use the cursor finally as a tie-breaker for the duplicates

When one field has been selected and we have duplicate values there's an issue:

  • We sort by documents, no duplicates are given here anymore because at this stage document view id's will be unique (when only one field is selected we always have a document per field)
  • .. still we go by cursor to paginate even though this one became insignificant for the order

To fix this we simply just don't order by document id when sorting over one field!

Closes #583

📋 Checklist

  • Add tests that cover your changes
  • Add this PR to the Unreleased section in CHANGELOG.md
  • Link this PR to any issues it closes
  • New files contain a SPDX license header

sandreae and others added 3 commits October 19, 2023 00:23
* main:
  Use `write_all`  when writing blob data to file system (#587)
  Use `p2panda-rs` `0.8.0` (#585)
Copy link

codecov bot commented Nov 15, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (1a7cefc) 92.70% compared to head (4fd90dc) 92.79%.

Files Patch % Lines
aquadoggo/src/db/stores/query.rs 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #593      +/-   ##
==========================================
+ Coverage   92.70%   92.79%   +0.09%     
==========================================
  Files         106      106              
  Lines       18405    18623     +218     
==========================================
+ Hits        17062    17281     +219     
+ Misses       1343     1342       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@adzialocha adzialocha changed the title Fix pagination query bug Fix pagination bug when only one field is selected and sorted at the same time Nov 15, 2023
@adzialocha adzialocha marked this pull request as ready for review November 15, 2023 23:51
@adzialocha adzialocha merged commit 86687b7 into main Nov 16, 2023
10 checks passed
@adzialocha adzialocha deleted the debug-pagination-query branch November 16, 2023 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect number of documents returned from ordered and filtered collection query
2 participants