Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace functional tests with Linearizability #15102

Closed
3 tasks done
serathius opened this issue Jan 13, 2023 · 1 comment · Fixed by #15284
Closed
3 tasks done

Replace functional tests with Linearizability #15102

serathius opened this issue Jan 13, 2023 · 1 comment · Fixed by #15284

Comments

@serathius
Copy link
Member

serathius commented Jan 13, 2023

What would you like to be added?

When linearizability tests reach feature parity we no longer will need functional tests. As was discussed in #14820, functional tests serve the same purpose however don't use e2e framework resulting in large maintanance code. This PR will track remaining features that we want to migrate to linearizability tests.

Property Functional tests Linearizability test
Cluster setup Single global config supported. Only 3 node cluster is tested Any number of configs. Any config supported by e2e tests
Failures Supports process termination, network termination, supports waiting for snapshot trigger Supports process termination, network termination, predefined failpoints. Doesn't support waiting for snapshot
Traffic Supports writes (small and large), reads on single key and range, transaction that writes or deletes Supports single key operations, transactions and leases
Checkers Checks consistency of hashes Validates linearizability which is much stricter than hashes
Max clients/QPS 100 clients generating 2000qps (not measured) 8 clients generating 200 qps

To implement in linearizability tests:

cc @ahrtr @ptabor

Why is this needed?

Reduce maintenance code, increase simplicity.

@serathius
Copy link
Member Author

We have implemented blackholing till snapshot, however we cannot enable it by default because we found an issue #15271.

With again linearizability tests finding issue that functional tests missed. Based on that I think we can proceed with removal of functional tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

1 participant