chore(oscal): begin integration of composed oscal with validations (#496

)

## Description

This change proposes the integration of
[Lula](https://github.com/defenseunicorns/lula) with uds-core for the
purposes of compliance assessment on Pull-Request. This Proposal
includes:
- all proposed future changes meet the minimum threshold for compliance
- framework established for iterating control body-of-evidence
- process for consuming uds-core compliance for downstream utilization

### How does it work?
Each component that comprises `uds-core` contains an
[OSCAL](https://pages.nist.gov/OSCAL/about/) file in the form of the
`component-definition` model. This captures 1 -> N controls for 1 -> N
standards (control implementations) that pertain to a specific
component.

When controls are identified as implemented by a component, we can
create or update a `component-definition` artifact through:
```bash
lula generate component -c <catalog-source> -r <controls-comma-delimited> --component "component name>
```

#### Validation
Lula operates on a model where controls are marked as satisfied when
presented with programmatic and repeatable processes to to produce
evidence that the control is satisfied.

These are often developed on a control-by-control basis and then
composed into a single OSCAL component-definition in a schema-compliant
addition in the back-matter. Reducing required network connectivity and
making the compliance information portable.

One control may require evidence from many `validations`. One
`validation` is the collection of data from some source and the
measurement of that data for required adherence.

The `lula validate` operation is an objective assessment - determining
control `satisfied/not-satisfied` state and writing the results to the
`assessment-results` artifact.

#### Evaluation

The last step for determining greater, equal, or lesser compliance than
the previous state is to execute a `lula evaluate`. This consumes the
latest `result` in the `assessment-results` artifact and compares it
against the `threshold` result.

The `threshold` result is identified as the `result` containing the
following prop:
```yaml
- name: threshold
   ns: https://docs.lula.dev/ns
   value: "true"
```

If the compliance of this latest result is worse than the threshold -
the execution fails with a non-zero exit code - allowing for pipeline
operations to fail.

If the compliance is equal - the command passes without any modification
required.

If the compliance is greater - Lula will automatically move the
threshold identifier to the applicable result item. (this would mean
that the updated `assessment-results` artifact should be downloaded and
submitted for update to the repository).

#### Naming

All files are titled with "oscal-" prepended to the OSCAL model type:
- oscal-component.yaml -> component definition
- oscal-assessment-results.yaml -> assessment results

This is primarily to allow for identification for linting workflows to
target oscal files for Lula to Lint.

### Metrics

The following are derived from the FedRAMP Moderate baseline:
```
24 controls of *329 total* mapped in OSCAL from Istio (7.29%)
24 controls of *161 total Technical* mapped in OSCAL from Istio (14.90%)
75 Unique controls captured in OSCAL of *329 total* (22.79%)
10 controls marked satisfied in OSCAL of *329 total* (3.03%)
75 Unique controls captured in OSCAL of *161 technical* (46.58%)
10 controls marked satisfied in OSCAL of *161 technical* (6.21%)
```

> [!NOTE]  
> DISA has recently published a new revision to the Cloud Computing
Security Requirements Guide which may result in future iterations to a
small subset of controls when reconciled.

[DISA link](https://public.cyber.mil/dccs/dccs-documents/)

### Review

The Lula team is actively working to make the review process better.
Composed OSCAL has it's benefits for portability but that comes with the
drawbacks of review.

To assist with the review process:
- Ensure the pipeline additions are sound
- Review the `src/istio/oscal-component.yaml` file. This will map
controls to the current state of validations.
- Understand the connection of `compliance/oscal-component.yaml` to the
individual components under `src/`

### Troubleshooting

If the pipeline should begin failing in the future - notify the
@defenseunicorns/lula-dev team.

It should be expected that any major revisions to applications could
break known assumptions about configuration required for compliance. The
Lula team can help navigate failures.

## Related Issue

Resolves #458 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Other (security config, docs update, etc)

## Checklist before merging

- [ ] Test, docs, adr added or updated as needed
- [ ] [Contributor Guide
Steps](https://github.com/defenseunicorns/uds-template-capability/blob/main/CONTRIBUTING.md)(https://github.com/defenseunicorns/uds-template-capability/blob/main/CONTRIBUTING.md#submitting-a-pull-request)
followed

---------

Co-authored-by: Andy Mills <61879371+CloudBeard@users.noreply.github.com>
Co-authored-by: Andy Mills <andy@defenseunicorns.com>
Co-authored-by: Micah Nagel <micah.nagel@defenseunicorns.com>

.github/actions/notify-lula/action.yaml

@@ -0,0 +1,51 @@
+name: Notify Lula
+description: "Comment on PR to notify Lula Team"
+
+
+inputs:
+  state:
+    description: 'state of the comment update'
+    required: true
+    default: ''
+  flavor:
+    description: 'flavor of the comment update'
+    required: true
+    default: ''
+  ghToken:
+    description: 'GITHUB_TOKEN'
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Find Comment
+      uses: peter-evans/find-comment@3eae4d37986fb5a8592848f6a574fdf654e61f9e # v3.1.0
+      id: fc
+      with:
+        issue-number: ${{ github.event.pull_request.number }}
+        comment-author: 'github-actions[bot]'
+        body-includes: Compliance ${{ inputs.flavor }} Evaluation
+        token: ${{ inputs.ghToken }}
+
+    - name: Create comment
+      if: ${{ steps.fc.outputs.comment-id == '' && inputs.state == 'failure'}}
+      uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
+      with:
+        issue-number: ${{ github.event.pull_request.number }}
+        token: ${{ inputs.ghToken }}
+        body: |
+          Compliance ${{ inputs.flavor }} Evaluation: ${{ inputs.state }}
+
+          CC: @defenseunicorns/lula-dev
+
+    - name: Update comment
+      if: ${{ steps.fc.outputs.comment-id != '' }}
+      uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
+      with:
+        comment-id: ${{ steps.fc.outputs.comment-id }}
+        token: ${{ inputs.ghToken }}
+        edit-mode: replace
+        body: |
+          Compliance ${{ inputs.flavor }} Evaluation: ${{ inputs.state }}
+
+          CC: @defenseunicorns/lula-dev

.github/actions/setup/action.yaml

@@ -35,6 +35,12 @@ runs:
       # renovate: datasource=github-tags depName=defenseunicorns/uds-cli versioning=semver
       run: brew install defenseunicorns/tap/uds@0.12.0
 
+    - name: Install Lula
+      uses: defenseunicorns/lula-action/setup@095636b7880051e11b05f10a582fdd911526161c
+      with:
+        # renovate: datasource=github-tags depName=defenseunicorns/lula versioning=semver-coerced
+        version: v0.4.1
+
     - name: Iron Bank Login
       if: ${{ inputs.registry1Username != '' }}
       env:

.github/workflows/compliance.yaml

@@ -0,0 +1,75 @@
+name: Compliance Evaluation
+
+on:
+  # Manual trigger
+  workflow_dispatch:
+    inputs:
+      flavor:
+        type: string
+        description: "Flavor of the source package to test"
+        required: true
+    # Triggered by pull-request-conditionals.yaml
+  workflow_call:
+    inputs:
+      flavor:
+        type: string
+        description: "Flavor of the source package to test"
+        required: true
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  evaluate:
+    runs-on: ubuntu-latest
+    name: Evaluate
+    continue-on-error: true
+    # env:
+    #   UDS_PKG: ${{ inputs.package }}
+    steps:
+      # Used to execute the uds run command
+      - name: Checkout repository
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
+
+      - name: Environment setup
+        uses: ./.github/actions/setup
+
+      - name: review compliance directory
+        run: ls -al ./compliance/
+        shell: bash
+
+      - name: remove overlapping file
+        run: rm ./compliance/oscal-assessment-results.yaml
+        shell: bash
+
+      - name: Download assessment
+        uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
+        with:
+          name: ${{ inputs.flavor }}-assessment-results
+          path: ./compliance
+
+      - name: review compliance directory again
+        run: ls -al ./compliance/
+        shell: bash
+
+      - name: Evaluate compliance
+        id: compliance-evaluation
+        run: uds run test-compliance-evaluate --no-progress
+
+      # steps in this action only run when there has been a previous failure - will indicate success thereafter
+      # need to think about how much noise this could create - noise currently = good
+      - name: Notify Lula Team of Compliance Assessment Results
+        if: ${{ always() }}
+        uses: ./.github/actions/notify-lula
+        with:
+          state: ${{ steps.compliance-evaluation.outcome }}
+          flavor: ${{ inputs.flavor }}
+          ghToken: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Upload Evaluated Assessment
+        uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
+        with:
+          name: ${{ inputs.flavor }}-assessment-results
+          path: ./compliance/oscal-assessment-results.yaml
+          overwrite: true

.github/workflows/lint-oscal.yaml

@@ -0,0 +1,56 @@
+name: Lint OSCAL Files
+
+on:
+  pull_request:
+    # milestoned is added here as a workaround for release-please not triggering PR workflows (PRs should be added to a milestone to trigger the workflow).
+    types: [milestoned, opened, reopened, synchronize]
+    paths:
+      - '**/*oscal*.yaml'
+
+permissions:
+  contents: read
+
+jobs:
+
+  check-oscal-paths:
+    runs-on: ubuntu-latest
+    name: OSCAL Change Detection
+    outputs:
+      oscal: ${{ steps.path-filter.outputs.oscal }}
+      oscal_files: ${{ steps.path-filter.outputs.oscal_files }}
+    steps:
+      - name: Checkout the code
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
+
+      # Uses a custom action to filter paths for source packages.
+      - name: Check src paths
+        id: path-filter
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
+        with:
+          filters: |
+            oscal:
+              - added|modified: "**/*oscal*.yaml"
+          list-files: shell
+
+  lint-oscal:
+    needs: check-oscal-paths
+    if: ${{ needs.check-oscal-paths.outputs.oscal == 'true' }}
+    runs-on: ubuntu-latest
+    steps:
+      # filter the files to remove not oscal files (such as those titles oscal-* under ./.github)
+      - name: Identify changed OSCAL files
+        id: find_changed_files
+        run: |
+          CHANGED_FILES=$(echo "${{ needs.check-oscal-paths.outputs.oscal_files }}" | tr ' ' '\n' | grep -v ".github*" | tr '\n' ',' | sed 's/.$//' || true)
+          echo "Changed OSCAL files: $CHANGED_FILES"
+          echo "oscal_files=$CHANGED_FILES" >> "$GITHUB_OUTPUT"
+        shell: bash
+      # checkout for access to the oscal files targeted for linting
+      - name: Checkout the code
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
+      - name: Environment setup
+        uses: ./.github/actions/setup
+      # lint the oscal files
+      - name: lint-oscal
+        run: uds run lint-oscal --set OSCALFILES=${{ steps.find_changed_files.outputs.oscal_files }} --no-progress
+        shell: bash

.github/workflows/pull-request-conditionals.yaml

@@ -10,7 +10,7 @@ on:
 permissions:
   id-token: write # Needed for OIDC-related operations.
   contents: read # Allows reading the content of the repository.
-  pull-requests: read # Allows reading pull request metadata.
+  pull-requests: write # Allows writing pull request metadata.
   packages: read # Allows reading the published GHCR packages
 
 # Default settings for all run commands in the workflow jobs.
@@ -86,3 +86,15 @@ jobs:
       flavor: ${{ matrix.flavor }}
       test_type: ${{ matrix.test_type }}
     secrets: inherit # Inherits all secrets from the parent workflow.
+
+  evaluate-package-compliance:
+    needs: run-package-test
+    name: Compliance Evaluation
+    strategy:
+      matrix:
+        flavor: [upstream, registry1, unicorn]
+      fail-fast: false
+    uses: ./.github/workflows/compliance.yaml
+    with:
+      flavor: ${{ matrix.flavor }}
+    secrets: inherit # Inherits all secrets from the parent workflow.

.github/workflows/test.yaml

@@ -65,6 +65,17 @@ jobs:
         if: ${{ inputs.package == 'all' && inputs.test_type == 'install' }}
         run: uds run test-uds-core --set FLAVOR=${{ inputs.flavor }} --no-progress
 
+      - name: Validate UDS Core Compliance
+        if: ${{ inputs.package == 'all' && inputs.test_type == 'install' }}
+        run: uds run test-compliance-validate --no-progress
+
+      - name: Upload Assessment
+        if: ${{ inputs.package == 'all' && inputs.test_type == 'install' }}
+        uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
+        with:
+          name: ${{ inputs.flavor }}-assessment-results
+          path: ./compliance/oscal-assessment-results.yaml
+
       - name: Test UDS Core Upgrade
         if: ${{ inputs.package == 'all' && inputs.test_type == 'upgrade' }}
         run: uds run test-uds-core-upgrade --set FLAVOR=${{ inputs.flavor }} --no-progress