Ability to reproduce any observation

[brandtkeller]

Intent

For the Lula validation process - we have access to the validations (proofs) of what is measured and the policy it must adhere to - this can be linked and/or transient across OSCAL as the project defines.

The missing piece for historical assessment data is to have the collected data present in the Assessment-Results such that anyone performing an audit of what was assessed would have the ability to "replay" a given point-in-time observation.

I believe this creates a layer of trust where auditing can be a function of reviewing both the validation inputs, data applied against the inputs, and the policy decision that was made to influence the finding state.

Potential Considerations

• encode the data collected from domains that is processed by a provider and include it in the assessment-result in some way.
• Create a flag in the validate command that allows for the replay of a specific observation? Make the output more verbose by logging a pretty-json object and the results of this offline-replay.
• Other functionality?

Sub-issues

Since this is a decent chunk of work, the following issues have been created to close out this issue:

• Save resources from validation: Save resources from validation execution #620
• Dev commands to pull resources/validations from assessment results to enable dev replay: Dev commands to extract validation and resources by observation uuid #621
• Validate replay: Validate Component Definition Replay #623

[brandtkeller]

Is there other metadata that we need to include in the observation other information that is pertinent for reproducible result.

[brandtkeller]

Should we consider the use of a separate file for storing the data collected?

[brandtkeller]

This issue has been decomposed into the aforementioned issues. Closing this issue as complete.