Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests/pbkdf2: de-flaky-fy test [backport 2022.10] #18941

Merged

Conversation

maribu
Copy link
Member

@maribu maribu commented Nov 21, 2022

Backport of #18932

Contribution description

Previously, the test vectors were encoded into the python test scripts, converted to base64, and send over to the device under test via stdio. The application sent back the output after converting it to base64 first, which was read back in by the test script and decoded. Finally, the test script compared the result with the expected result.

This made the test complex, slow and, flanky, as stdio on interfaces such as UART has a high bit error rate and some quirks (e.g. the EDBG UART bridge e.g. in the samr21-xpro dropping bytes when bursts of more than 64 bytes at a time are send).

This basically rewrites the test to embed the test vectors in the firmware and do the comparison on the devices. This fixes test failures on the samr21-xpro, the nRF52840-DK and likely many others. Also, it is now fast.

Beware: This sneaks in a non-API-change-API-change: The password and salt is now passed as const void * rather than as const uint8_t *, as having to cast c-strings to const uint8_t * is a needles pain in the ass.

Testing procedure

make BOARD=nrf52840dk -C tests/pbkdf2 flash test

should pass. Changing one bit in the expected digest in one of the test vectors should result in the test to no longer pass.

Issues/PRs references

None

Having to cast a password provided as `const char *` to
`const uint8_t *` is a needless pain in the ass when using the API.
Hence, fix it by accepting passwords and salts as `const void *`
instead.

(cherry picked from commit e92a716)
@maribu maribu added Area: sys Area: System Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR Process: release backport Integration Process: The PR is a release backport of a change previously provided to master labels Nov 21, 2022
@maribu maribu requested a review from benpicco November 21, 2022 11:21
@riot-ci
Copy link

riot-ci commented Nov 21, 2022

Murdock results

✔️ PASSED

2f6dd98 tests/pbkdf2: de-flanky-fy test

Success Failures Total Runtime
115836 0 115836 02h:26m:31s

Artifacts

This only reflects a subset of all builds from https://ci-prod.riot-os.org. Please refer to https://ci.riot-os.org for a complete build for now.

@kaspar030
Copy link
Contributor

seems to choke on the moved macro includes.

Previously, the test vectors were encoded into the python test scripts,
converted to base64, and send over to the device under test via stdio.
The application sent back the output after converting it to base64
first, which was read back in by the test script and decoded. Finally,
the test script compared the result with the expected result.

This made the test complex, slow and, flanky, as stdio on interfaces
such as UART has a high bit error rate and some quirks (e.g. the EDBG
UART bridge e.g. in the samr21-xpro dropping bytes when bursts of more
than 64 bytes at a time are send).

This basically rewrites the test to embed the test vectors in the
firmware and do the comparison on the devices. This fixes test failures
on the samr21-xpro, the nRF52840-DK and likely many others. Also, it
is now fast.
@maribu maribu force-pushed the backport/2022.10/sys/hashes/pbkdf2 branch from 5c580da to 2f6dd98 Compare November 21, 2022 15:41
@maribu maribu added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Nov 21, 2022
@maribu maribu changed the title tests/pbkdf2: de-flanky-fy test [backport 2022.10] tests/pbkdf2: de-flaky-fy test [backport 2022.10] Nov 22, 2022
@kaspar030 kaspar030 added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Nov 22, 2022
@maribu maribu enabled auto-merge November 22, 2022 16:34
@maribu maribu merged commit 70495b1 into RIOT-OS:2022.10-branch Nov 22, 2022
@maribu maribu deleted the backport/2022.10/sys/hashes/pbkdf2 branch November 22, 2022 17:05
@maribu
Copy link
Member Author

maribu commented Nov 22, 2022

Thx :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: sys Area: System Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Process: release backport Integration Process: The PR is a release backport of a change previously provided to master
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants