verification attestor

[mikhailswift]

Fixes in-toto/witness#14

The rough idea here is to have go-witness.Verify actually be a go-witness.Run with a verification attestor as it's Execute attestor. This allows a few different things:

1. We can create verification attestations. We're currently using the SLSA VSA format. There is some chatter about moving this under the in-toto attestation group: Attestation for policy verification attestation#277 . This would involve generalizing some of the SLSA specific data in the VSA, which would be ideal.
2. We can collect other attestations about the system that verified an artifact. Since the policy verification is happening within the verification attestor in an attestation context, we can also run our other suite of attestors, such as the GitHub or GitLab attestors, and collect more information about where an artifact was verified.
3. We can use verification attestations in a handful of ways, including "caching" decisions within an admission controller. A policy could potentially define a way to act as a quick pass if we see we've already verified an artifact against a specified policy. This could be time-gated, so we must re-evaluate if the verification attestation is past a certain age.
4. Act as a summary of a group of raw attestations. Some attestations may not be fit for public consumption. In these cases, organizations could publish the VSA instead of raw attestations to their end users.

There are a few philosophical questions and changes to the go-witness API happening here we should think a bit about:

In main today, the DSSE signing code will throw up if at least one signer isn't provided. This makes sense; why wrap something in a DSSE envelope if it won't be signed? However, we don't want every verifier to have keys distributed to them. Users should be able to run go-witness.Verify without providing a key. If one is provided, a signed VSA should be created. In this draft changeset, this error is removed. An unsigned envelope will be returned if no signers are provided to the DSSE code.

This has the side-effect of allowing users to run go-witness.Run without a signer provided. This may not be a bad thing, we would hand back an unsigned envelope of attestations. This could allow users to play with Witness more quickly and sort out key distribution once ready.

While we offer no guarantees of backward compatibility, this may involve some breaking changes to go-witness's surface APIs. We shouldn't let this stop us from making choices, but we should be cognizant of the pain this will cause and ensure we do it for the right reasons.

[naveensrinivasan]

A couple of suggestions.

[mikhailswift]

Additional changes pushed to this:

• Added the insecure flag to go-witness.Run to allow running without signers provided.
• If an ErrPolicyDenied error is received from policy.Verify, it is returned to the caller. This is the canonical way to communicate verification failures to callers currently, so this is an effort to maintain that consistency.
• Added signer options to VerifyOptions. This is a bit confusing and I don't love it... Because we also have a VerifyWithRunOptions that will forward Run options, the user could supply signers in various ways currently.

This still needs tests, but I am able to run our witness test scripts with these changes with this branch on witness: https://github.com/testifysec/witness/tree/feat/verification-attestation

[naveensrinivasan]

I think the code should check for policyErr instead of err here.

var accepted bool
	policyResult, policyErr := pol.Verify(ctx.Context(), policy.WithSubjectDigests(vo.subjectDigests), policy.WithVerifiedSource(verifiedSource))
	if _, ok := policyErr.(policy.ErrPolicyDenied); ok {
		accepted = false
	} else if policyErr != nil {
		return fmt.Errorf("failed to verify policy: %w", policyErr)
	} else {
		accepted = true
	}

[mikhailswift]

Yup, good catch

[naveensrinivasan]

A nit vo. Having consistent receiver name with vs helps :)

[mikhailswift]

Agreed, the receiver name is a byproduct of the code moving where the variable was originally of type verifyOptions, this should be renamed to be consistent with our new receiver.

[mikhailswift]

I added a second commit to this to support some work I'm doing in a separate project that's using go-witness. It's not directly related to VSAs, but I needed it for work in the archivista-data-provider.

When this PR gets moved out of draft and is ready for merge, the second commit will be broken out into it's own PR as it is a sizeable change in it's own right.

[naveensrinivasan]

I added a second commit to this to support some work I'm doing in a separate project that's using go-witness. It's not directly related to VSAs, but I needed it for work in the archivista-data-provider.

When this PR gets moved out of draft and is ready for merge, the second commit will be broken out into it's own PR as it is a sizeable change in it's own right.

Currently, the cryptoutil.DigestValue type is used extensively across the codebase. While this works well for our current needs, it might make the code more rigid and harder to test.

One way to improve this could be to define an interface that cryptoutil.DigestValue satisfies. This would allow us to create mock implementations for testing different scenarios and make the code more flexible. Here's a simple example of what this might look like:

type Digest interface {
    New() hash.Hash
    // other methods...
}

type DigestValue struct {
    // fields...
}

func (d DigestValue) New() hash.Hash {
    // implementation...
}

func SomeFunction(d Digest) {
    hash := d.New()
    // use hash...
}

In this example, SomeFunction can now accept any object that satisfies the Digest interface. This makes it easier to test SomeFunction with different implementations of Digest.

This is just a suggestion.

[ChaosInTheCRD]

Just being sure to ask what springs to mind, do we need all of these extra fields in the attestor? I can imagine that this information can be of use when performing validation on what was verified and how, but I supose this comes at the consequence of bloating the predicate output? Isn't one of the benefits of a VSA that it is lightweight and provides minimal predicate inspection by the final verifier? @mikhailswift @jkjell

[ChaosInTheCRD]

Oh ignore me here. I see that these are not fields to be added to the predicate but instead supporting fields for attestor functionality. The slsa.VerificationSummary is the only field that is used as the predicate.

One thing that springs to mind is that when we have the time, better moulding the structures so that predicates are defined very clearly could be a good thing 😄

[ChaosInTheCRD]

This is a nit so no action is required 👍

[ChaosInTheCRD]

just picking up on the TODO here. Given that we can't provide a public location of the policies, wouldn't we be better (for now) using https://witness.testifysec.com/policy/v0.1 for this URI field?

[ChaosInTheCRD]

This is addressed in a PR I created #230

[ChaosInTheCRD]

Noticed that this is a copy of https://github.com/in-toto/go-witness/blob/main/verify.go#L182. Would we not be better exporting the original function? alternatively we could stick it in an /internal location so that both verify.go and policyverify.go can make use of it without the copying? Unless there's something I am missing and this function is slightly different to the one in verify.go

[ChaosInTheCRD]

ahhhh I see what you've done here. You've moved a lot of the logic from verify.go into policyverify.go. I am not averse to this, however I do think that there may be a better shared place for this function to live, as I reckon this functionality might need to be leveraged elsewhere in the codebase in the future.

Having said all this, it's not necessary right now and I am okay leaving it as is and kicking the can down the road so to speak until we need it elsewhere.

[ChaosInTheCRD]

This is addressed in a PR I created #230

[ChaosInTheCRD]

I don't wish to make life more difficult in the process of getting this merged, but I think a new type should be declared here in to define its type.

I just ran the code locally, and in its current form, there's nothing from stopping me from trying to run this attestor during witness run. We should have an easy way to distinguish this attestor from the others that exist, and to me a verify type is the way we should to it.

Of course this is a small thing to implement (a variable declaration and a line somewhere in the Run logic to ensure that any verify attestors can't be executed, but I think it's crucial for user experience.

[ChaosInTheCRD]

This is addressed in a PR I created here

[ChaosInTheCRD]

I've created a PR here in the hope that it will speed up the process of getting this merged. I think there are still tests needed for the VSA logic, @mikhailswift let me know if you want me to take the reigns on writing these 😄