Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "critical" validation phase to validation #2098

Merged
merged 1 commit into from
Jan 18, 2024

Conversation

mtdowling
Copy link
Member

@mtdowling mtdowling commented Jan 13, 2024

Runs "critical" validators before other validators so that they can be written without needing to account for things like whether traits are applied in valid places or not or if a trait's value matches its definition and constraints.

This can help to simplify validators as they can now call methods like .expectShape on a Model to throw if a shape is missing rather than having to defensively code around potentially incorrect models.

To ensure this works backward compatibly with existing errorfile based test runners, we will now detect whether to use a "legacy" validation mode that continues to validate after a critical validator emitted an event because an errorfiles mixes critical and non-critical events. This ensures backward compatibility and that any new addition of critical validators in the future will not break existing test cases.

@mtdowling mtdowling force-pushed the more-blocking-validation branch from 13be365 to 841f302 Compare January 13, 2024 22:52
@mtdowling mtdowling changed the title Add a "critical" validation phase to validation Add "critical" validation phase to validation Jan 14, 2024
@mtdowling mtdowling force-pushed the more-blocking-validation branch from 841f302 to 9dc4d30 Compare January 14, 2024 21:16
@mtdowling mtdowling force-pushed the more-blocking-validation branch from 9dc4d30 to d679d87 Compare January 18, 2024 20:52
@mtdowling mtdowling requested a review from kstich January 18, 2024 20:53
@mtdowling mtdowling marked this pull request as ready for review January 18, 2024 21:02
@mtdowling mtdowling requested a review from a team as a code owner January 18, 2024 21:02
@mtdowling mtdowling force-pushed the more-blocking-validation branch from d679d87 to 4640bcb Compare January 18, 2024 21:07
Runs "critical" validators before other validators so that they can be
written without needing to account for things like whether traits are
applied in valid places or not or if a trait's value matches its
definition and constraints.

This can help to simplify validators as they can now call methods like
.expectShape on a Model to throw if a shape is missing rather than
having to defensively code around potentially incorrect models.

To ensure this works backward compatibly with existing errorfile based
test runners, we will now detect whether to use a "legacy" validation
mode that continues to validate after a critical validator emitted an
event because an errorfiles mixes critical and non-critical events.
This ensures backward compatibility and that any new addition of
critical validators in the future will not break existing test cases.
@mtdowling mtdowling force-pushed the more-blocking-validation branch from 4640bcb to 2431108 Compare January 18, 2024 22:15
@mtdowling mtdowling merged commit 061a607 into main Jan 18, 2024
11 checks passed
@mtdowling mtdowling deleted the more-blocking-validation branch September 22, 2024 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants