Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable concrete playback for contract and stubs #3389

Merged
merged 13 commits into from
Jul 31, 2024

Conversation

celinval
Copy link
Contributor

@celinval celinval commented Jul 26, 2024

This PR enables concrete playback for contract and stubs. Since these two cases may not actually behave as users expect (it can even have an internal failure), I am adding documentation to the generated test calling that out.

As I was testing this issue, I realized that concrete playback didn't quite work for arrays. So I introduced a new private function any_raw_array, which doesn't change the behavior during verification, but allow us to special case it in the concrete playback flow.

Resolves #3383

Call-out

I just realized I forgot to update kani-core. I'll leave it as ready for review, so I can already get some feedback.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

This allow us to get more information in the Kani driver about which
kind of proof it is. For proof_for_contract, we can now check which
function is the target.
@celinval celinval requested a review from a team as a code owner July 26, 2024 23:51
@github-actions github-actions bot added the Z-BenchCI Tag a PR to run benchmark CI label Jul 26, 2024
@celinval celinval force-pushed the issue-3383-contract-playback branch from 2cb1be8 to 5922fdb Compare July 29, 2024 20:55
Copy link
Contributor

@zhassan-aws zhassan-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Can you clarify how concrete playback for a contract can fail?

Can we also add a test for stubs?

kani_metadata/src/harness.rs Outdated Show resolved Hide resolved
library/kani_core/src/arbitrary.rs Outdated Show resolved Hide resolved
@celinval
Copy link
Contributor Author

celinval commented Jul 30, 2024

Nice!

Can you clarify how concrete playback for a contract can fail?

It can fail if your pre-conditions are too loose or if the post-conditions are too strict. I found it very hard to debug contract specification otherwise.

Can we also add a test for stubs?

Sure! I might just add a generic script to run multiple concrete playback tests. It's a bit annoying to write a new script for each case.

celinval and others added 2 commits July 29, 2024 20:37
Co-authored-by: Zyad Hassan <88045115+zhassan-aws@users.noreply.github.com>
Co-authored-by: Zyad Hassan <88045115+zhassan-aws@users.noreply.github.com>
@zhassan-aws
Copy link
Contributor

Nice!
Can you clarify how concrete playback for a contract can fail?

It can fail if your pre-conditions are too loose or if the post-conditions are too strict. I found it very hard to debug contract specification otherwise.

I meant when would concrete playback itself fail? I'm specifically referring to this part of the PR description:

This PR enables concrete playback for contract and stubs. Since these two cases may not actually behave as users expect (it can even have an internal failure), I am adding documentation to the generated test calling that out.

I can see why it would fail for stubs (hence the documentation that you added makes sense for stubs), but I'm not sure if the same applies to contracts? Or perhaps this is referring to the stubbing of a contract?

@celinval
Copy link
Contributor Author

The contract themselves may have calls to kani::any(), which could make the test misbehave. Additionally, on normal executions, contract clauses are not added to the execution flow, so a contract failure won't be triggered. If we implement #3326, then we wouldn't have this problem any more.

@celinval celinval enabled auto-merge (squash) July 31, 2024 05:00
tests/script-based-pre/playback_expected/playback.sh Outdated Show resolved Hide resolved
tests/script-based-pre/playback_expected/playback.sh Outdated Show resolved Hide resolved
tests/script-based-pre/playback_expected/playback.sh Outdated Show resolved Hide resolved
tests/script-based-pre/playback_expected/playback.sh Outdated Show resolved Hide resolved
tests/script-based-pre/playback_expected/playback.sh Outdated Show resolved Hide resolved
kani_metadata/src/harness.rs Outdated Show resolved Hide resolved
kani-driver/src/concrete_playback/test_generator.rs Outdated Show resolved Hide resolved
@celinval celinval merged commit 3bc4f0c into model-checking:main Jul 31, 2024
26 of 27 checks passed
github-merge-queue bot pushed a commit that referenced this pull request Aug 9, 2024
## 0.54.0

### Major Changes
* We added support for slices in the `#[kani::modifies(...)]` clauses
when using function contracts.
* We introduce an `#[safety_constraint(...)]` attribute helper for the
`Arbitrary` and `Invariant` macros.
* We enabled support for concrete playback for harness that contains
stubs or function contracts.
* We added support for log2*, log10*, powif*, fma*, and sqrt*
intrisincs.

### Breaking Changes
* The `-Z ptr-to-ref-cast-checks` option has been removed, and pointer
validity checks when casting raw pointers to references are now run by
default.

## What's Changed
* Make Kani reject mutable pointer casts if padding is incompatible and
memory initialization is checked by @artemagvanian in
#3332
* Fix visibility of some Kani intrinsics by @artemagvanian in
#3323
* Function Contracts: Modify Slices by @pi314mm in
#3295
* Support for disabling automatically generated pointer checks to avoid
reinstrumentation by @artemagvanian in
#3344
* Add support for global transformations by @artemagvanian in
#3348
* Enable an `#[safety_constraint(...)]` attribute helper for the
`Arbitrary` and `Invariant` macros by @adpaco-aws in
#3283
* Fix contract handling of promoted constants and constant static by
@celinval in #3305
* Bump CBMC Viewer to 3.9 by @tautschnig in
#3373
* Update to CBMC version 6.1.1 by @tautschnig in
#2995
* Define a struct-level `#[safety_constraint(...)]` attribute by
@adpaco-aws in #3270
* Enable concrete playback for contract and stubs by @celinval in
#3389
* Add code scanner tool by @celinval in
#3120
* Enable contracts in associated functions by @celinval in
#3363
* Enable log2*, log10* intrinsics by @tautschnig in
#3001
* Enable powif* intrinsics by @tautschnig in
#2999
* Enable fma* intrinsics by @tautschnig in
#3002
* Enable sqrt* intrinsics by @tautschnig in
#3000
* Remove assigns clause for ZST pointers by @carolynzech in
#3417
* Instrumentation for delayed UB stemming from uninitialized memory by
@artemagvanian in #3374
* Unify kani library and kani core logic by @jaisnan in
#3333
* Stabilize pointer-to-reference cast validity checks by @artemagvanian
in #3426
* Rust toolchain upgraded to `nightly-2024-08-07` by @jaisnan @qinheping
@tautschnig @feliperodri

## New Contributors
* @carolynzech made their first contribution in
#3387

**Full Changelog**:
kani-0.53.0...kani-0.54.0

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 and MIT licenses.

Signed-off-by: Felipe R. Monteiro <felisous@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Z-BenchCI Tag a PR to run benchmark CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable concrete playback for contracts
3 participants