Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing storage format tests #9866

Open
yanesca opened this issue Dec 20, 2024 · 2 comments
Open

Missing storage format tests #9866

yanesca opened this issue Dec 20, 2024 · 2 comments
Labels

Comments

@yanesca
Copy link
Contributor

yanesca commented Dec 20, 2024

Summary

When I merged Mbed-TLS/mbedtls-framework#83 wrongly assumed that the interface stability failures on the CI can't be related to the framework changes. It turns out that thousands of interface stability tests have disappeared.

We need to confirm that they only have been renamed/moved if that is the case or add them back if not.

See:
https://mbedtls.trustedfirmware.org/blue/organizations/jenkins/mbed-tls-pr-merge/detail/PR-9841-merge/4/pipeline
https://mbedtls.trustedfirmware.org/blue/organizations/jenkins/mbed-tls-pr-merge/detail/PR-9842-merge/4/pipeline
https://mbedtls.trustedfirmware.org/blue/organizations/jenkins/mbed-tls-pr-merge/detail/PR-9835-merge/7/pipeline

@gilles-peskine-arm
Copy link
Contributor

One of the effects of Mbed-TLS/mbedtls-framework#83 was

Fix some test cases that were not properly detected as never-executed.

Sorry, I wasn't very explicit, but in more detail, the script that generates test cases tries to avoid generating positive test cases for mechanisms that are declared, but not implemented. This was buggy, and is more accurate after Mbed-TLS/mbedtls-framework#83. The PR description has an overview of the expected changes:

Reviewers of Mbed-TLS/mbedtls-framework#83 should have validated those claims. Therefore we expect the interface stability tests to report the changes in 119b3b9740b1a64d0ee8ca302a5dcd72b6d36d3d in consuming PR (#9841, #9796, Mbed-TLS/TF-PSA-Crypto#122).

Looking at the logs for 3.6 and for development, I believe that indeed, the logs are as expected: all the complaints are about disappearing positive key storage tests that use a mechanism that is not implemented. (There's a large number of them because this includes several hashes and several elliptic curves, and we test (almost) the whole 3D matrix of ECDSA-variant × curve × hash.)

I don't know why Mbed-TLS/TF-PSA-Crypto#122 has a passing interface stability test. I'm assuming that this is because the repo is still very new and there's no point of comparison yet? If that's not the problem then maybe the interface stability tests aren't set up properly on TF-PSA-Crypto.

@gilles-peskine-arm
Copy link
Contributor

(I'm not going to close this issue since it's really a review or gatekeeping consideration on Mbed-TLS/mbedtls-framework#83, and I have a conflict of interest as the author of Mbed-TLS/mbedtls-framework#83.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants