Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add CLI option for consuming a validation statement directly #70

Closed
brandtkeller opened this issue Sep 26, 2023 · 1 comment · Fixed by #339
Closed

Add CLI option for consuming a validation statement directly #70

brandtkeller opened this issue Sep 26, 2023 · 1 comment · Fixed by #339
Assignees
@mike-winberry
Labels
enhancement New feature or request triage Awaiting triage from the team
Milestone
Quality of life

Comments

@brandtkeller
Copy link
Member

Developer workflow for creating validation payloads for inclusion in OSCAL documentation will take iteration. We can streamline the development process for these validating statements by making the testing of them consume the Lula payload directly instead of via the OSCAL.
@brandtkeller brandtkeller added the enhancement New feature or request label Sep 26, 2023
@meganwolf0 meganwolf0 assigned meganwolf0 and unassigned meganwolf0 Mar 26, 2024
@mike-winberry mike-winberry self-assigned this Mar 26, 2024
@brandtkeller brandtkeller added this to the Quality of life milestone Mar 26, 2024
mike-winberry added a commit that referenced this issue Mar 29, 2024
@mike-winberry
wip: #70. Add dev validate command tthat runs plula validations w/o o…
b8416c6 
…scal. Create initial e2e test.
@mike-winberry
Copy link
Collaborator

depends on the completion of #325

@mike-winberry mike-winberry mentioned this issue Mar 29, 2024
Merged
4 tasks
@mike-winberry mike-winberry linked a pull request Mar 29, 2024 that will close this issue
feat(dev): #70 add dev validate cmd #339
Merged
4 tasks
@brandtkeller brandtkeller added the triage Awaiting triage from the team label Mar 29, 2024
mike-winberry added a commit that referenced this issue Apr 1, 2024
@mike-winberry
feat(dev/validate): #70. Add logic for outputting lula Validation w/ …
fd55acf 
…result. Update e2e test
mike-winberry added a commit that referenced this issue Apr 1, 2024
@mike-winberry
fix: #70 fixed the expected output logic.
51f9e9c
mike-winberry added a commit that referenced this issue Apr 2, 2024
@mike-winberry
docs: #70 update dev validate cmd comments and cli output.
1c44236
mike-winberry added a commit that referenced this issue Apr 2, 2024
@mike-winberry
feat(dev/validate): #70. Create initial LintValidation functionality …
22aab2d 
…and added to dev validate cmd
mike-winberry added a commit that referenced this issue Apr 2, 2024
@mike-winberry
refactor: #70 extracted LintValidation to Lint method on the type Val…
28d8d2e 
…idation. Create types_test with validation.Lint tests
mike-winberry added a commit that referenced this issue Apr 2, 2024
@mike-winberry
refactor: #70 add comment to validation.Lint func
34a2a54
mike-winberry added a commit that referenced this issue Apr 3, 2024
@mike-winberry
feat(cmd): #70 Update the dev validate command to handle taking input…
44d161e 
… from stdin
mike-winberry added a commit that referenced this issue Apr 4, 2024
@mike-winberry
feat(dev-validate): #70 added a timeout cmd line arg to dev validate …
b616a3d 
…to kill stdin after the timeout

 (default: 1 (second), no timeout: -1)
mike-winberry added a commit that referenced this issue Apr 10, 2024
@mike-winberry
refactor: #70 extracted conversion and validation logic into struct m…
081bb5f 
…ethods for LulaValidation and Validation.

fix(dev/validate): updated to use new Validation and Lula validation types.
refactor(component): BackMatterToMap now uses validation.ToLulaValidation()
refactor(validate): ValidateOnCompDef now uses LulaValidation.Validate()
mike-winberry added a commit that referenced this issue Apr 10, 2024
@mike-winberry
Merge branch '70-add-cli-option-for-consuming-a-validation-statement-…
13bc72b 
…directly' into 298-validations-in-a-separate-manifest

fix: merge #70. Update tests and implementations for remote validation
to handle new LulaValidation obj
brandtkeller added a commit that referenced this issue Apr 10, 2024
@mike-winberry @meganwolf0
@renovate @brandtkeller
feat(dev): #70 add dev validate cmd (#339)
5c2eb65 
* added command to get resources from validation manifest

* added tests, updated get-resources functionality

* fixed copy pasta error

* wip: #70. Add dev validate command tthat runs plula validations w/o oscal. Create initial e2e test.

* test(dev_validation_test): created scenario, and fixed the test breaking due to pod being deleted

* feat(dev/validate): #70. Add logic for outputting lula Validation w/ result. Update e2e test

* feat(dev/validation): Added expected result boolean that will print a message if the expected result is not the result given

* fix: #70 fixed the expected output logic.

* docs: #70 update dev validate cmd comments and cli output.

* feat(dev/validate): #70. Create initial LintValidation functionality and added to dev validate cmd

* refactor: #70 extracted LintValidation to Lint  method on the type Validation. Create types_test with validation.Lint tests

* refactor: #70 add comment to validation.Lint func

* fix: dev validate kyverno not passing due to incorrect yaml unmarshal method.\nchore: add kyverno validation to dev_validation_test\nfix: title being required for validation

* feat(cmd): #70 Update the dev validate command to handle taking input from stdin

* feat(dev-validate): #70 added a timeout cmd line arg to dev validate to kill stdin after the timeout
 (default: 1 (second), no timeout: -1)

* deps Update dependency go to v1.22.2 (#355)

* docs: ADR for validation format (#354)

* initial stab at validation yaml

* added placeholder workflow example

* updated adr

* updated adr

* removed examples for domains/providers we've yet to implement

* updated adr

* Rename and review ADR with modifications

* minor adr updates

* updated adr

* docs: Move ADR to Accepted

---------

Co-authored-by: Brandt Keller <brandt.keller@defenseunicorns.com>

* deps Update module sigs.k8s.io/controller-runtime to v0.17.3 (#359)

* feat: refactor for domains and providers (#358)

* initial refactor for domains and providers

* fixed domain and provider instantiation

* updated tests

* fix: 332 component_test now passing successfully given a valid component definition

* fix: 332 actually fix the test.

* docs: update demo oscal to reflect changes

* added unit tests for GetProvider and GetDomain

* updated docs

* updated docs and tests

---------

Co-authored-by: Cole (Mike) Winberry <cole@defenseunicorns.com>
Co-authored-by: Brandt Keller <brandt.keller@defenseunicorns.com>
Co-authored-by: Brandt Keller <43887158+brandtkeller@users.noreply.github.com>

* refactor: #70 extracted conversion and validation logic into struct methods for LulaValidation and Validation.
fix(dev/validate): updated to use new Validation and Lula validation types.
refactor(component): BackMatterToMap now uses validation.ToLulaValidation()
refactor(validate): ValidateOnCompDef now uses LulaValidation.Validate()

* fix: remove types_test.go

* tests(dev/validate): update validation.kyverno.yaml with new schema

* fix(dev/validate): writeValidation -> writeValidationResult. Now writes a validation result instead of a common.Validation

* fix: typos and help menu not showing

---------

Co-authored-by: Megan Wolf <megan@defenseunicorns.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Megan Wolf <97549300+meganwolf0@users.noreply.github.com>
Co-authored-by: Brandt Keller <brandt.keller@defenseunicorns.com>
Co-authored-by: Brandt Keller <43887158+brandtkeller@users.noreply.github.com>
brandtkeller added a commit that referenced this issue Apr 16, 2024
@mike-winberry @meganwolf0
@brandtkeller @renovate
feat: Add the ability to pull validations from external sources (#357)
a868cdb 
* added command to get resources from validation manifest

* added tests, updated get-resources functionality

* fixed copy pasta error

* wip: #70. Add dev validate command tthat runs plula validations w/o oscal. Create initial e2e test.

* test(dev_validation_test): created scenario, and fixed the test breaking due to pod being deleted

* feat(dev/validate): #70. Add logic for outputting lula Validation w/ result. Update e2e test

* feat(dev/validation): Added expected result boolean that will print a message if the expected result is not the result given

* fix: #70 fixed the expected output logic.

* docs: #70 update dev validate cmd comments and cli output.

* feat(dev/validate): #70. Create initial LintValidation functionality and added to dev validate cmd

* refactor: #70 extracted LintValidation to Lint  method on the type Validation. Create types_test with validation.Lint tests

* refactor: #70 add comment to validation.Lint func

* fix: dev validate kyverno not passing due to incorrect yaml unmarshal method.\nchore: add kyverno validation to dev_validation_test\nfix: title being required for validation

* create initial set  up for remote_validation tests

* feat(cmd): #70 Update the dev validate command to handle taking input from stdin

* feat(dev-validate): #70 added a timeout cmd line arg to dev validate to kill stdin after the timeout
 (default: 1 (second), no timeout: -1)

* feat: Create network pkg methods to parse and fetch file from url.

* feat: added ability to link individual validations using href. TODO: add checksum validation

* feat(validate): added ability to fetch local validations given a path prefixed with file:.
feat(validate): added checksum validation to remote validation fetching

* feat: establish workflow for lula validation identifiers

* fix: remove back-matter identifier

* feat(validate): add functionality to switch cwd to input file path dir, and reset after validations for relative validation paths. Add ChangeCwd and ValidateChecksum methods to common. Remove ComponentDefinitionDir from config in favor or ChangeCwd method.

* deps Update dependency go to v1.22.2 (#355)

* docs: ADR for validation format (#354)

* initial stab at validation yaml

* added placeholder workflow example

* updated adr

* updated adr

* removed examples for domains/providers we've yet to implement

* updated adr

* Rename and review ADR with modifications

* minor adr updates

* updated adr

* docs: Move ADR to Accepted

---------

Co-authored-by: Brandt Keller <brandt.keller@defenseunicorns.com>

* deps Update module sigs.k8s.io/controller-runtime to v0.17.3 (#359)

* feat: refactor for domains and providers (#358)

* initial refactor for domains and providers

* fixed domain and provider instantiation

* updated tests

* fix: 332 component_test now passing successfully given a valid component definition

* fix: 332 actually fix the test.

* docs: update demo oscal to reflect changes

* added unit tests for GetProvider and GetDomain

* updated docs

* updated docs and tests

---------

Co-authored-by: Cole (Mike) Winberry <cole@defenseunicorns.com>
Co-authored-by: Brandt Keller <brandt.keller@defenseunicorns.com>
Co-authored-by: Brandt Keller <43887158+brandtkeller@users.noreply.github.com>

* refactor: #70 extracted conversion and validation logic into struct methods for LulaValidation and Validation.
fix(dev/validate): updated to use new Validation and Lula validation types.
refactor(component): BackMatterToMap now uses validation.ToLulaValidation()
refactor(validate): ValidateOnCompDef now uses LulaValidation.Validate()

* fix: remove types_test.go

* fix: add testing of error scenarios for ValidationFromString()

* tests(dev/validate): update validation.kyverno.yaml with new schema

* tests(network): fixed shasums.
tests(remote-validations): fix artifacts to work with new schemas

* fix(dev/validate): writeValidation -> writeValidationResult. Now writes a validation result instead of a common.Validation

* fix: dev validate now shows help if it fails to read from stdin

* fix: dev validate cmd typo's in validateHelp

* fix: typos and help menu not showing

* fix: migrate to using ToLulaValidation function

* Debugging validation results issue

* fix: val assignment bug

* fix: update tests to use rel: lula

* fix: #298 update to work with changes to identifier.
feat: #298. add logic to read in validation maps (map[string]Validation).

* chore(github): fix bug_report template

* refactor: move checksumm validation from common to network. Write tests for common and network.

* docs(oscal-validation-links): updated with information on remote validations

* refactor: clean up

* #298. Rewrite to handle "---" delimeted validations instead of validation map

* chore: clean up unusued validation map funcs

* docs: update oscal-validation-links with info on multiple validation files

* test: add info to scenarios/remote-validations/component-definition

* test: fix TestSwitchCwd so they will work on mac filesystem

* docs(oscal-validation-links): #298 updated docs with wildcard for all validations.
test(remote_validation_test): updated component-definition resource w/ wildcard (fails)

* fix: #298. validate now runs all validations only when the wildcard is present

* fix(validate): #298 now throws error if no validations are found.

* chore: #298. update the lula-version in the remote-validations scenarios. Fix network_test with new checksum after filechange

---------

Co-authored-by: Megan Wolf <megan@defenseunicorns.com>
Co-authored-by: Brandt Keller <brandt.keller@defenseunicorns.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Megan Wolf <97549300+meganwolf0@users.noreply.github.com>
Co-authored-by: Brandt Keller <43887158+brandtkeller@users.noreply.github.com>
@github-actions github-actions bot mentioned this issue May 24, 2024
Merged
This was referenced Jun 29, 2024
Merged
Merged
This was referenced Jul 12, 2024
Merged
Merged
@github-actions github-actions bot mentioned this issue Jul 26, 2024
Merged
@github-actions github-actions bot mentioned this issue Aug 5, 2024
Merged
This was referenced Aug 9, 2024
Merged
Merged
@github-actions github-actions bot mentioned this issue Aug 31, 2024
Merged
@github-actions github-actions bot mentioned this issue Sep 13, 2024
Merged
@github-actions github-actions bot mentioned this issue Sep 27, 2024
Merged
This was referenced Oct 11, 2024
Merged
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mike-winberry mike-winberry

Labels
enhancement New feature or request triage Awaiting triage from the team
Projects
Lula and Friends
Archived in project
Milestone
Quality of life
Development

Successfully merging a pull request may close this issue.

feat(dev): #70 add dev validate cmd defenseunicorns/lula
3 participants
@brandtkeller @mike-winberry @meganwolf0