Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adversarial Arbitrary values for fuzzing #1052

Closed
brson opened this issue Aug 10, 2023 · 4 comments
Closed

Adversarial Arbitrary values for fuzzing #1052

brson opened this issue Aug 10, 2023 · 4 comments

Comments

@brson
Copy link
Contributor

brson commented Aug 10, 2023

What problem does your feature solve?

In #957 I mentioned that the Arbitrary impls for contract types never generate "adversarial" values.

The Soroban type system allows for Vals to have various states that are unexpected but representable. The fuzzing infrastructure doesn't generate these.

What would you like to see?

Vals with bad tags

These are no longer allowed by the runtime.

If a fuzz tests asks for a raw Val type then we should sometimes give them one with a bogus tag.

Vecs and maps with heterogeneous element types

Done.

The Rust wrappers around vecs and maps are not fully recursively typechecked when created - individual element accesses may fail due to dynamic typing errors.
When arbitrary vecs and maps are requested we should occasionally return ones where the keys and/or values are not the expected type but a random type.

Object types with invalid references

These are no longer allowed by the runtime.

For any of the possible object types (the ones that aren't encoded directly in the 64-bit rawval), we should sometimes provide a broken value; a good tag, but a reference that doesn't exist. I believe these are representable within the typed Rust wrappers around the various object kinds, and will produce errors at access-time.

Object types that are shared references

These are not actually possible. Soroban values are immutable.

Any object type might be a shared and mutable reference in the Soroban type system. This is something that would likely be unexpected to Rust programmers.

It would be desirable that, for any object types, the Arbitrary implementation returns the exact same reference as in a previous invocation, creating shared object references within the contract.

This is a pretty tough thing to do because it requires remembering the state of previous Arbitrary invocations. We would need to do this inside our SorobanArbitrary implementations, probably by stashing away previous objects in some private location in Env, then pulling them back out and returning them.

Deeply nested vecs and maps

I don't think this is worth doing. It would be rare code that would be rare code that would actually fail to this type, and the inevitable impact would be stack overflow and transaction failure.

This probably only makes sense for arbitrary Vals - typed vectors and maps can't be arbitrarily-deeply recursive. So deeply-nested containers may not be so useful for finding real bugs - it will only matter if a contract is attempting to fully and recursively interpret Val types, which would be an ambitious thing for a contract to do.

But we could provide deeply nested vecs and maps when a Val is requested.

BytesN with more than N bytes.

I don't think this is worth doing. It's possible to construct unsafely, and probably possible to transmit between contracts, but the constructor from Val to BytesN will always fail and terminate the transaction.

These may be possible.

What alternatives are there?

Don't do any of these.

@brson
Copy link
Contributor Author

brson commented Aug 31, 2023

Another unexpected but probably possible case:

BytesN that actually has > N bytes.

@brson
Copy link
Contributor Author

brson commented Aug 31, 2023

soroban-env has checks for object validity now that should make it impossible to create invalid objects.

@brson
Copy link
Contributor Author

brson commented Sep 2, 2023

soroban-env now also rejects bad tags

github-merge-queue bot pushed a commit that referenced this issue Oct 26, 2023
### What

When fuzzing, sometimes generate vecs and maps with elements of the
wrong type.

These types are generated 1 in every 1000 iterations.

### Why

It's a weird case that is possible for contracts to hit in production,
so should be generated during fuzzing.

Part of #1052

### Known limitations

None.
@brson
Copy link
Contributor Author

brson commented Oct 31, 2023

Done.

@brson brson closed this as completed Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant