feat(evaluate): add observation details

[meganwolf0]

Description

Adds some additional detail to the evaluate command when findings between New and Threshold assessments are different. Ideally this will give the person enough information to quickly know what changes changed the satisfaction of the findings.

Added a --summary flag to evaluate to print all observations.

Current output for changed assessments is in a table format for all observations so as to not duplicate single observations that fail on multiple controls.

Related Issue

Fixes #533

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Other (security config, docs update, etc)

Checklist before merging

• Test, docs, adr added or updated as needed
• Schema Updates applied
• Contributor Guide Steps followed

[meganwolf0]

Sample view for the user - shows both sets of results leading to failing eval + "summary" to get all observations. I think these can be rearranged slightly and can have more fidelity over the info returned to try and not overwhelm the user, but wanted to throw out for spears

[brandtkeller]

The information at play here makes the workflow very observation-centric. I don't see anything wrong with that and this is a huge improvement for exposing information.

I would be curious if you have thoughts for the output and the Observation to Finding relationship?

IE if a control that was previously passing now has multiple failing observations - this results in multiple line-items in the output - because multiple observations are failing. Ultimately these multiple failing observations roll-up to a single failing finding (for which has an actual field for capturing state).

Example (ignore the new/threshold remarks columns):

dev@dev:~/work/lula$ ./bin/lula evaluate -f assessment-results.yaml

 NOTE  Saving log file to /tmp/lula-2024-07-18-23-11-19-4230766809.log
  ⠋  Evaluating Assessment Results 5a0a9538-e734-48a5-a327-02e6aa6891b0 against b57d4be5-75a7-4b3b-be66-e3cfcb1cac15 (0s)
 WARNING  Evaluation Failed against the following:                                                                                                                                                                                                                                                                                                                             

Control ID(s) | Observation                                                | Satisfied | Change                     | New Remarks                                                         | Threshold Remarks                                                                                                                                                                  
sc-23         | 67456ae8-4505-4c93-b341-d977d90cb125 - istio-health-check  | false     | SATISFIED TO NOT SATISFIED | istiohealth.deployment_message: All deployment conditions are true. | istiohealth.deployment_message: All deployment conditions are true.
              |                                                            |           |                            | istiohealth.hpa_message: HPA has sufficient replicas.               | istiohealth.hpa_message: HPA has sufficient replicas.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sc-23         | ca49ac97-487a-446a-a0b7-92b20e2c83cb - enforce-mtls-strict | false     | SATISFIED TO NOT SATISFIED | validate.msg: All PeerAuthentications have mtls mode set to STRICT. | validate.msg: All PeerAuthentications have mtls mode set to STRICT.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

     ERROR:  failed to meet established threshold 

Current approach works well for the summary containing the state of observations and those mapped to many Findings (control-ids really) - but I wonder if we need to augment or add something in the future that is more Finding-centric?

[meganwolf0]

Current approach works well for the summary containing the state of observations and those mapped to many Findings (control-ids really) - but I wonder if we need to augment or add something in the future that is more Finding-centric?

Yeah I think this would be totally do-able. I do a kind of refactor of the paired results to get the observations by controls, so reporting out more info about the original object would probably be a simple thing to add. Probably would need to talk more about what would be best to try and tease out of that info for the user? This alternate table tried to just report each control along with the observations.. so you could kind of slice it that way too. I just didn't love that for the scenario where you had like a single observation that's attached to a bunch of different findings failing.

[brandtkeller]

Quick cleanup items and I believe this is ready. One of the first instances of improving UX and feedback will probably be the main driver for updates outside of general assumptions.