Notation sign/verify arbitrary file

[Two-Hearts]

UX proposal for sign in file system:

# Prerequisites:
# A default signing key is configured using CLI "notation key"

# Use boolean flag "--file" to enable signing a file
# The generated signature is saved at the same dir as the target file with name
# <target_file>.sig
notation sign --file <target_file_path>:<mediaType>

# Use boolean flag "--file" to enable signing a file
# Use flag "--signature" to specify path where the generated signature is stored
notation sign --file <target_file_path>:<mediaType> --signature <signature_path>

---

UX proposal for verify in file system:

# Prerequisites: Both target and signature files are stored in file system

# Use boolean flag "--file" to enable verifying a file
# Use flag "--signature" to specify path where the signature is stored
# The global trust policy is used by default
notation verify --file <target_file_path>:<mediaType> --signature <signature_path>

# Use boolean flag "--file" to enable verifying a file
# Use flag "--signature" to specify path where the signature is stored
# Trust policy with scope "example/myPolicy" is used, if it does not exist, the global trust policy is used
notation verify --file <target_file_path>:<mediaType> --signature <signature_path> --scope <example/myPolicy>

UX proposal for verify file stored in OCI-compliant registry:

# Prerequisites: Signatures are stored in the registry referencing the file artifact

# Use boolean flag "--file" to enable verifying a file
notation verify --file <full_URI_reference>

The above CLI design requires following changes in the specifications repo:

1. Signature payload:

   {
     "targetArtifact": {
       "mediaType": "sbom/example",             // can be defined by user, but we need a default value
       "digest": "sha256:abcdef",               // digest of the file content
       "size": 32654                            // size of the file content
     }
   }

2. Signature storage in file system
3. Trust policy: for files and signatures stored in file system, the registryScope term needs to be redefined.
4. Any sign and verify workflow changes involved in this new process
   @yizha1 @FeynmanZhou

Based on the above design, there is a major discussion started by @priteshbandi:
notation should sign/verify arbitrary data instead of just files. For example, data from stdin (we need a clearer scenario on this one).
Here's my understanding: the --file flag is introduced based on our discussion when we brought in the --oci-layout flag several months ago: #601 (comment). We are following the practice of one type per flag (this is also what ORAS is following). Thus, to sign/verify data from stdin, we'd have another flag such as --stdin. Another example is, I'd like to sign/verify a descriptor using notation. In this case, a --descriptor flag is necessary.
And of course, my understanding might not perfectly align with what our users are expecting. Therefore, we need more people's inputs to discuss on this part. @sajayantony @shizhMSFT @toddysm @yizha1 @FeynmanZhou

[shizhMSFT]

Is --file a flag of --file bool or --file string?

[Two-Hearts]

It's a --file bool. When --file flag is set, <reference> of the command becomes <target_file_path>. Updated the original post.

[yizha1]

@priteshbandi @Two-Hearts @shizhMSFT @FeynmanZhou I would suggest the following process for feature #741. What do you think? Of cause, we can continue discussing specifications and UX here before we finalize it in respective repo.

1. Finalize new specs under specifications repo, see two issues

• Add new specifications for non-OCI signatures specifications#275 for scenario 1: sign and verify arbitrary blobs
• Add new specifications for distributing non-OCI artifacts and signatures via OCI compliant registries specifications#276 for scenario 2: distribute arbitrary blobs and signatures via OCI compliant registries and verify signatures in registries.

2. Finalize CLI specs under notation repo for CLI UX
3. Implementation

Note: We may complete scenario 1 first then scenario 2.

[priteshbandi]

IMO we should start with requirements/usecases then work on user experience and once we have we UX finalized, we can make relevant changes to NotaryProject specifications. At high level I see following usecases

Requirements

1. User MUST be able to sign and verify arbitrary content
2. For verification, the user MUST be able to leverage existing fine-grain controls(expiry, revocation, authenticity/logging, enforce) offered by the trust policy.
   1. The user SHOULD be able to configure default values for the controls which will by default apply to all verify operations.
   2. User SHOULD be able to specify a set of predefined controls (different from the default) for a given verify operation
3. For signing, a user SHOULD be able to sign a precalculated digest. This is different because here notation won't recalculate digest instead sign whatever the user provided. (This will also help us confirm the extensibility for different kind of artifact)
4. For signing and verification, the user SHOULD be able to pass content as std-in or can be read from a file. std-in allows for better chaining/piping between tools

PS: we should go through comment on #765 to confirm that we didnt miss any requirement

[priteshbandi]

We dont really need to understand/decode the data that we are signing or verifying. Is CVE-2021-41190 applicable in that case?

[shizhMSFT]

In the above case, we are not decoding the data since the mediaType should be passed to notation verify by the user if the first verification method is used.

[gokarnm]

I assume this feature is for signing any kind of file on disk. Given a file, how will we deduce the media type during verification, to match with the media type in the signature. For e.g. given a image manifest, the verification workflow can interpret it as plain text, json or image manifest type. Am I missing something here?

[Two-Hearts]

@gokarnm Here's my understanding, instead of deducing to match the signature, a verifier should know what they are verifying (for example, this info can be provided by the signer). The UX for verify is:
notation verify --file <target_file_path>:<mediaType> --signature <signature_path>.
If the signature payload's media type does not match with the verifier's expectation, then the verification would fail.

[priteshbandi]

a verifier should know what they are verifying (for example, this info can be provided by the signer). The UX for verify is:
notation verify --file <target_file_path>: --signature <signature_path>.
If the signature payload's media type does not match with the verifier's expectation, then the verification would fail.

Why does a user need to know about mediaType to verify the integrity and authenticity of the data? For example, A user signed a file, Zduring the signing, they didn't provide media type (or if it's compulsory provided some value). Now during verification, if user forgot mediatype. Shouldn't they be able to verify the integrity and authenticity of the data?

[yizha1]

@priteshbandi as one action from community meeting on 8/28/2023, could you share your thoughts on using mediaType? If we are aligned on the solution, I will create a PR to add a new spec for signing non-OCI artifacts.

[yanivpaz]

@toddysm is this solution suppose to also sign AWS lambdas?