Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[suggestion] Re-entrant lazily evaluated pagination #3468

Closed
7 tasks
appetrosyan opened this issue May 10, 2023 · 1 comment
Closed
7 tasks

[suggestion] Re-entrant lazily evaluated pagination #3468

appetrosyan opened this issue May 10, 2023 · 1 comment
Assignees
Labels
Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST

Comments

@appetrosyan
Copy link
Contributor

Feature request

To fix #3379 we should explore defaulting to reasonable pagination and optimising the behaviour for paginated query results to not cause a stoppage in Iroha.

Specifically pagination should be:

  • Re-entrant. There should be a mechanism to request a query result, paginate it, and request a different part of the query result at a later time.
  • Lazily-evaluated. We only want to compute the parts of the query which are relevant to generating the current page. If the query asks for all domains, but only the first 100 are present in the page, we should "pause" the evaluation until a different page is requested.
  • Optimal. If the 3-rd page is requested, we only resume evaluation to find the 3rd page of results and nothing more.
  • Idempotent. Requesting the same page twice is not an error. The result is identical.
  • Functional reactive. If the data changed during the evaluation of the query, the result consistent with the old data is returned.
  • Sessions expire. If a paginated result was requested and not resumed for a period of time, the re-entrant session expires and new data is returned.
  • Resettable. The session can be reset under certain conditions to be discussed with @gv-timur and @arndey.

Motivation

Needed for internal projects

Who can help?

@mversic

@appetrosyan appetrosyan added Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST labels May 10, 2023
@Mingela
Copy link
Contributor

Mingela commented May 22, 2023

I'd ask to consider a simpler approach for an ordinary node relaxing re-entrance/session oriented properties and leave those for a specific node type only (i.e. archival) not to potentially flood Iroha's resources (valuable for transaction throughput) allocation with this kind of page management tasks.
Pagination is generally used in combination with sorting, so the same value appearance in a next page may indicate the change of the data set.
For the hundred percent accuracy I'd support using an archival node capable of binding a request to a specific block height WSV.

Anyway, that's just my perception, an evaluation of potential pitfalls/tradeoffs preserving each of the qualities listed is up to @mversic I believe.

mversic added a commit to mversic/iroha that referenced this issue Jun 19, 2023
… result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 19, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 19, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 19, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 19, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 20, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 20, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 20, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 21, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 21, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 28, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jun 29, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 6, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 6, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 11, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 12, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 12, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 12, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 12, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 13, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 13, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 13, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 13, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 13, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 21, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 25, 2023
…result

Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 25, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 25, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit to mversic/iroha that referenced this issue Jul 25, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
appetrosyan pushed a commit that referenced this issue Jul 26, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
appetrosyan pushed a commit that referenced this issue Jul 26, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
appetrosyan pushed a commit that referenced this issue Jul 26, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
@mversic mversic closed this as completed Sep 13, 2023
mversic added a commit that referenced this issue Oct 17, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit that referenced this issue Oct 17, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
mversic added a commit that referenced this issue Oct 17, 2023
Signed-off-by: Marin Veršić <marin.versic101@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST
Projects
None yet
Development

No branches or pull requests

3 participants