Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): improve ruleset validation #2026

Merged
merged 2 commits into from
Aug 3, 2022
Merged

Conversation

P0lip
Copy link
Contributor

@P0lip P0lip commented Jan 18, 2022

Fixes #2186

This will let us remove https://github.com/stoplightio/platform-internal/tree/f7fddbb1bfe6a760819a2568a7eed50732774af7/services/projects/src/utils/assertValidRuleset

A non-exhaustive list of changes:

  • Added JSON Schema definitions that are suitable for JSON/YAML rulesets - going to be useful internally, particularly in the case of the ruleset-migrator package. Due to that, assertValidRuleset accepts a second parameter, namely format.
  • Fixed the validation of function options.
  • Instead of a single error with all messages concatenated, we now throw an AggregateError to retain each validation error, and ultimately make the errors easier to consume. This resulted in a lot of test changes, as well as the addition of that custom expect matcher.
  • Moved JSON Schema definitions describing rulesets from src/meta to src/ruleset/meta
  • Setup package exports so it's easier to import the validator
  • FormatsSet has been renamed to Formats
  • Improve toJSON signatures -> 2358b85
  • Moved some files under src/ruleset around to unify the directory structure

Do note that although some changes are marked as additions, they are not such - it's mostly me slicing certain chunks of code into separate files. In other words, a notable deletion of code is usually followed by an addition somewhere.

Checklist

  • Tests added / updated
  • Docs added / updated

Does this PR introduce a breaking change?

  • Yes
  • No

@P0lip P0lip added the enhancement New feature or request label Jan 18, 2022
@P0lip P0lip self-assigned this Jan 18, 2022
@P0lip P0lip force-pushed the feat/core/validation-rework branch 4 times, most recently from 8069c55 to 088d42b Compare January 19, 2022 02:23
@P0lip P0lip changed the title feat(core): improve validation feat(core): improve ruleset validation Jan 19, 2022
@P0lip P0lip force-pushed the feat/core/validation-rework branch 2 times, most recently from 704bc83 to c0d5c70 Compare January 31, 2022 13:14
@P0lip P0lip force-pushed the feat/core/validation-rework branch from c0d5c70 to 2358b85 Compare February 14, 2022 18:52
@P0lip P0lip marked this pull request as ready for review February 14, 2022 19:15
@P0lip P0lip force-pushed the feat/core/validation-rework branch from 2358b85 to 1020bfc Compare February 14, 2022 19:15
@P0lip P0lip requested a review from mmiask February 14, 2022 19:16
@Nezteb
Copy link

Nezteb commented Mar 2, 2022

@P0lip @mmiask This PR is pretty stale. Can it be closed? If not, what is its priority?

@mmiask
Copy link
Contributor

mmiask commented Mar 7, 2022

@P0lip @mmiask This PR is pretty stale. Can it be closed? If not, what is its priority?

@Nezteb IMHO this should be review ASAP, although I've been busy with sprint-related stuff. I'll take a look as soon as I can.

@P0lip
Copy link
Contributor Author

P0lip commented Mar 7, 2022

Let's review #2062 first. It's a subset of the following PR.

@P0lip P0lip marked this pull request as draft March 7, 2022 12:41
@pamgoodrich
Copy link
Contributor

@P0lip : Does this issue require Spectral doc updates and if so, would you like help?

@P0lip
Copy link
Contributor Author

P0lip commented Apr 14, 2022

Thanks @pamgoodrich, I think we're good. This mostly improves the existing functionality, so I expect no docs changes to be required, but I'll make sure to let you know in case something changes.

@P0lip P0lip force-pushed the feat/core/validation-rework branch from 1020bfc to 39708e9 Compare April 14, 2022 15:47
@P0lip P0lip mentioned this pull request Apr 26, 2022
4 tasks
@Nezteb Nezteb added team/dnc Team: Dazed and Confused and removed team/dnc Team: Dazed and Confused labels Apr 26, 2022
@P0lip P0lip force-pushed the feat/core/validation-rework branch from 39708e9 to e9fcea5 Compare April 27, 2022 12:44
@P0lip P0lip marked this pull request as ready for review April 27, 2022 12:51
@P0lip P0lip requested a review from a team as a code owner April 27, 2022 12:51
@P0lip P0lip requested a review from a team April 27, 2022 12:51
@P0lip P0lip force-pushed the feat/core/validation-rework branch from e9fcea5 to f53ee36 Compare April 27, 2022 12:53
@Nezteb Nezteb added team/dnc Team: Dazed and Confused and removed team/dnc Team: Dazed and Confused labels Apr 27, 2022
@Nezteb Nezteb added team/dnc Team: Dazed and Confused and removed team/dnc Team: Dazed and Confused labels May 12, 2022
@Nezteb Nezteb added the team/dnc Team: Dazed and Confused label May 24, 2022
philsturgeon
philsturgeon previously approved these changes Jun 20, 2022
Copy link
Contributor

@philsturgeon philsturgeon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We know im not the best at the sort of super advanced TypeScript here, but I believe the tests speak for themselves, and I can trust @P0lip with the rest.

mnaumanali94
mnaumanali94 previously approved these changes Aug 2, 2022
@P0lip P0lip dismissed stale reviews from mnaumanali94 and philsturgeon via 1329113 August 3, 2022 12:00
@P0lip P0lip merged commit 8315162 into develop Aug 3, 2022
@P0lip P0lip deleted the feat/core/validation-rework branch August 3, 2022 12:06
stoplight-bot pushed a commit that referenced this pull request Aug 3, 2022
@stoplight-bot
Copy link
Collaborator

🎉 This PR is included in version @stoplight/spectral-cli-v6.5.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

stoplight-bot pushed a commit that referenced this pull request Aug 3, 2022
@stoplight-bot
Copy link
Collaborator

🎉 This PR is included in version @stoplight/spectral-core-v1.13.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

stoplight-bot pushed a commit that referenced this pull request Aug 3, 2022
@stoplight-bot
Copy link
Collaborator

🎉 This PR is included in version @stoplight/spectral-functions-v1.7.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

stoplight-bot pushed a commit that referenced this pull request Nov 13, 2024
## [1.1.3](https://github.com/stoplightio/spectral/compare/@stoplight/spectral-runtime-v1.1.2...@stoplight/spectral-runtime-1.1.3) (2024-11-13)

### Bug Fixes

* **cli:** choose proxy agent based on requester protocol ([#2521](#2521)) ([056f2e1](056f2e1))
* **cli:** clarify usage of --format ([#2575](#2575)) ([96eee89](96eee89))
* **cli:** do not show 'or higher' if severity equals error ([#2172](#2172)) ([f31ec63](f31ec63))
* **cli:** missing line break ([#2251](#2251)) ([d16bf9a](d16bf9a))
* **cli:** output to stdout not working with multiple output formatters ([#2044](#2044)) ([77dfe3b](77dfe3b))
* **cli:** peer dependency incorrectly met ([#2268](#2268)) ([1b70398](1b70398))
* **cli:** Trigger cli release ([#2695](#2695)) ([c48a929](c48a929))
* **cli:** trigger docker release ([920f7b5](920f7b5))
* **cli:** update dependencies and trigger docker release ([c87eacf](c87eacf))
* **core:** async functions have undefined paths ([#2304](#2304)) ([df257b3](df257b3))
* **core:** bump @stoplight/better-ajv-errors from 1.0.1 to 1.0.3 ([7f9bcba](7f9bcba))
* **core:** bump nimma from 0.1.7 to 0.1.8 ([#2058](#2058)) ([fb756f2](fb756f2))
* **core:** bump nimma from 0.1.8 to 0.2.0 ([#2088](#2088)) ([36ec40e](36ec40e))
* **core:** bump nimma from 0.2.0 to 0.2.1 ([#2157](#2157)) ([4d5ebeb](4d5ebeb))
* **core:** bump nimma from 0.2.1 to 0.2.2 ([#2173](#2173)) ([65ba74f](65ba74f))
* **core:** consider `message` when de-duplicating results ([#2052](#2052)) ([b07cc7b](b07cc7b))
* **core:** dedupe paths containing special characters correctly ([758de21](758de21))
* **core:** fix 'resolved vs unresolved' json path mapping ([#2202](#2202)) ([157ec59](157ec59))
* **core:** fix for TypeError "this.formats.has is not a function" ([#2664](#2664)) ([75d642d](75d642d))
* **core:** improve deep ruleset inheritance ([#2326](#2326)) ([378b4b8](378b4b8))
* **core:** invalid then produced by Rule#toJSON ([#2496](#2496)) ([db91553](db91553))
* **core:** more accurate ruleset error paths ([66b3ca7](66b3ca7))
* **core:** pointer in overrides are applied too broadly ([#2511](#2511)) ([69403c1](69403c1))
* **core:** redeclared rules should always be re-enabled ([#2138](#2138)) ([6def6be](6def6be))
* **core:** require new formats version ([#2725](#2725)) ([8ec328f](8ec328f))
* **core:** reset path in fn context ([#2389](#2389)) ([3d47ec4](3d47ec4))
* **core:** support utf8 surrogates ([#2267](#2267)) ([a1bd6d2](a1bd6d2))
* **deps:** fix package.json ([0161072](0161072))
* **formats:** update spectral core version ([6b196da](6b196da))
* **formatters:** update spectral core to latest version ([8a382f4](8a382f4))
* **functions:** __importDefault undefined ([609ecb1](609ecb1))
* **functions:** bump stoplight/better-ajv-errors ([bd0c5fb](bd0c5fb))
* **functions:** handle percent encoded in unreferencedReusableObject([#2212](#2212)) ([d16b5a6](d16b5a6))
* **functions:** reset RegExp.lastIndex to zero when using cached RegExp objects ([#2079](#2079)) ([4839527](4839527))
* **functions:** update spectral core to latest ([ede60f3](ede60f3))
* **parsers:** update @stoplight/* dependencies ([a68c255](a68c255))
* **parsers:** update @stoplight/json from ~3.20.1 to ~3.21.0 ([e906d20](e906d20))
* **parsers:** update @stoplight/yaml from ~4.2.3 to ~4.3.0 ([91fdded](91fdded))
* **ref-resolver:** bump @stoplight/json-ref-resolver from ~3.1.4 to ~3.1.5 ([#3635](https://github.com/stoplightio/spectral/issues/3635)) ([215ae93](215ae93))
* **ref-resolver:** update @stoplight/json-ref-resolver from ~3.1.5 to ~3.1.6 ([6f73151](6f73151))
* **ref-resolver:** update @stoplight/json-ref-resolver from 3.1.3 to ~3.1.4 ([dc97f24](dc97f24))
* **repo:** remove discord link and fix typo in github bug template ([#2642](#2642)) ([048924d](048924d))
* **repo:** update yarn lock ([362cdb4](362cdb4))
* **ruleset-bundler:** __importDefault undefined ([874a80e](874a80e))
* **ruleset-bundler:** address Rollup.js warning ([1e36673](1e36673))
* **ruleset-bundler:** builtins plugin should create a new instance for each module ([b06903c](b06903c))
* **ruleset-bundler:** defaults should be last one ([#2403](#2403)) ([8780cfa](8780cfa))
* **ruleset-bundler:** never externalize builtins ([#2174](#2174)) ([fb1bbe6](fb1bbe6))
* **ruleset-bundler:** remove extraneous 'external dependency' warnings ([#2475](#2475)) ([e791534](e791534))
* **ruleset-bundler:** virtualFs plugin incompatible with commonjs plugin ([a48381b](a48381b))
* **ruleset-bundler:** virtualFs plugin not recognizing files ([#2271](#2271)) ([4bc38b3](4bc38b3))
* **ruleset-migrator:** avoid positive lookbehinds ([#2349](#2349)) ([455c324](455c324))
* **ruleset-migrator:** correct package.json's browser field ([#2497](#2497)) ([89a6a67](89a6a67))
* **ruleset-migrator:** http/https uris not followed correctly ([#2247](#2247)) ([573e112](573e112))
* **ruleset-migrator:** transform functions under overrides ([#2459](#2459)) ([45e817f](45e817f))
* **ruleset-migrator:** update @stoplight/json from ~3.20.1 to ~3.21.0 ([3f7eebc](3f7eebc))
* **ruleset-migrator:** use module for require.resolve ([#2405](#2405)) ([d7c0fa4](d7c0fa4))
* **ruleset-migrator:** validate aliases correctly ([#2085](#2085)) ([1f4ab20](1f4ab20))
* **rulesets:** __importDefault undefined ([fdd647b](fdd647b))
* **rulesets:** __importDefault undefined ([c123bdf](c123bdf))
* **rulesets:** __importDefault undefined ([#2243](#2243)) ([660f090](660f090))
* **rulesets:** always allow string examples in asyncapi schema ([#2625](#2625)) ([4e2f797](4e2f797))
* **rulesets:** avoid false errors from ajv ([#2408](#2408)) ([92dab78](92dab78))
* **rulesets:** bump @stoplight/better-ajv-errors from 1.0.1 to 1.0.3 ([4f55c4f](4f55c4f))
* **rulesets:** example validation for required readOnly and writeOnly properties ([#2573](#2573)) ([ae1fea5](ae1fea5))
* **rulesets:** fixed array-items type property selector ([#2638](#2638)) ([0845fb5](0845fb5))
* **rulesets:** handle empty payload and headers in AsyncAPI message's examples validation ([#2284](#2284)) ([4068221](4068221))
* **rulesets:** length.min said "must not be longer than" ([#2355](#2355)) ([df3b6f9](df3b6f9))
* **rulesets:** oasExample should clean id fields from non-schema objects ([#2561](#2561)) ([7f7583e](7f7583e))
* **rulesets:** operation-tags should fail on empty array ([#2050](#2050)) ([a4c421f](a4c421f))
* **rulesets:** remove step summary rule ([#2692](#2692)) ([d5a566f](d5a566f))
* **rulesets:** simplify schema used in duplicated-entry-in-enum ([#2055](#2055)) ([8451774](8451774))
* **rulesets:** tweak server variables function ([#2533](#2533)) ([244cbda](244cbda))
* **rulesets:** update spectral core to latest ([d74c2b0](d74c2b0))
* **rulesets:** use uri-reference for oauth security schemes ([#2652](#2652)) ([c411e63](c411e63))

### Features

* **cli:** add sarif formatter ([#2532](#2532)) ([959a86a](959a86a))
* **cli:** improve error logging ([#2071](#2071)) ([b194368](b194368))
* **cli:** require new stoplight dependencies ([#2726](#2726)) ([8a736b5](8a736b5))
* **cli:** require newer version of all Spectral dependencies ([10ddd97](10ddd97))
* **cli:** sort linting results alphabetically ([#2147](#2147)) ([84d48cf](84d48cf))
* **cli:** trigger release ([87a90b3](87a90b3))
* **cli:** use Content-Type header to detect ruleset format ([#2272](#2272)) ([b4c3c11](b4c3c11))
* **cli:** use hpagent ([#2513](#2513)) ([9b2d347](9b2d347))
* **core:** improve alias validation ([#2164](#2164)) ([a15150a](a15150a))
* **core:** improve validation ([#2026](#2026)) ([8315162](8315162))
* **core:** include error codes in RulesetValidationError ([c01c6b5](c01c6b5))
* **core:** relax formats validation ([#2151](#2151)) ([de16b4c](de16b4c))
* **core:** support end-user extensions in the rule definitions ([#2345](#2345)) ([365fced](365fced))
* **core:** support JSON ruleset validation ([#2062](#2062)) ([aeb7d5b](aeb7d5b))
* **core:** support x- extensions in the ruleset ([#2440](#2440)) ([964151e](964151e))
* **core:** trigger release ([b73d5e8](b73d5e8))
* **formats:** add arazzo format ([#2663](#2663)) ([dc1a8ef](dc1a8ef))
* **formats:** add support for 2.5.0 AsyncAPI ([#2292](#2292)) ([a7f9fa7](a7f9fa7))
* **formats:** jsonSchemaLoose format should search for enum keyword ([#2551](#2551)) ([0835545](0835545))
* **formats:** support 2.1.0, 2.2.0, 2.3.0 AsyncAPI versions ([#2067](#2067)) ([b0b008d](b0b008d))
* **formats:** support AsyncAPI 2.4 ([#2146](#2146)) ([8b5d6b3](8b5d6b3))
* **formats:** support AsyncAPI 2.6.0 ([#2391](#2391)) ([b8e51b4](b8e51b4))
* **formatters:** add code climate (GitLab) formatter ([#2648](#2648)) ([41eca61](41eca61))
* **formatters:** add export entrypoint for utils ([#2482](#2482)) ([d4b883c](d4b883c))
* **formatters:** add GitHub Actions formatter ([#2508](#2508)) ([6904927](6904927))
* **formatters:** add markdown formatter ([#2662](#2662)) ([b5edf5e](b5edf5e))
* **formatters:** add sarif formatter ([#2532](#2532)) ([908c308](908c308))
* **formatters:** move formatters to a separate package ([#2468](#2468)) ([664e259](664e259))
* **ruleset-bundler:** add fullOutput option to bundleRuleset ([#2194](#2194)) ([a31d34c](a31d34c))
* **ruleset-bundler:** expose commonjs plugin ([91a4b80](91a4b80))
* **ruleset-bundler:** plugins should be easy to override ([0263bf0](0263bf0))
* **ruleset-bundler:** skypack plugin accepts ignore list ([#2318](#2318)) ([6e6d0de](6e6d0de))
* **ruleset-migrator:** relax validation ([#2307](#2307)) ([d5ce09e](d5ce09e))
* **ruleset-migrator:** use Content-Type header to detect ruleset format ([#2317](#2317)) ([7abbe95](7abbe95))
* **rulesets:** add AsyncAPI v3 support ([#2697](#2697)) ([3d69be7](3d69be7))
* **rulesets:** add multiple xor ([#2614](#2614)) ([af9c742](af9c742))
* **rulesets:** add new rule that requires sibling items field for type array ([#2632](#2632)) ([24198bc](24198bc))
* **rulesets:** add oas3_1-servers-in-webhook and oas3_1-callbacks-in… ([#2581](#2581)) ([7a8cc0e](7a8cc0e))
* **rulesets:** add oas3-server-variables rule ([#2526](#2526)) ([4c4de85](4c4de85))
* **rulesets:** add rule to check if the AsyncAPI document is using the latest version ([#2282](#2282)) ([366779f](366779f))
* **rulesets:** add rule to validate AsyncAPI message's examples ([#2126](#2126)) ([87ef046](87ef046))
* **rulesets:** add rules for validation of server variables and channel parameters ([#2101](#2101)) ([9acc633](9acc633))
* **rulesets:** add rules for validation uniqueness of tag names ([#2104](#2104)) ([4447d81](4447d81))
* **rulesets:** add scope validation to oas{2,3}-operation-security-defined rules ([#2538](#2538)) ([68aacd6](68aacd6))
* **rulesets:** add support for 2.5.0 AsyncAPI ([#2292](#2292)) ([0cb2e85](0cb2e85))
* **rulesets:** add traits array path to headers rule ([#2460](#2460)) ([9ceabca](9ceabca))
* **rulesets:** add unused components server rule ([#2097](#2097)) ([71b312e](71b312e))
* **rulesets:** check uniqueness of AsyncAPI messages ([#2224](#2224)) ([297531b](297531b))
* **rulesets:** check uniqueness of AsyncAPI operations ([#2121](#2121)) ([8b3cce4](8b3cce4))
* **rulesets:** improve {oas2,oas3}-valid-schema rule ([#2574](#2574)) ([8df2c36](8df2c36))
* **rulesets:** initial rulesets for the Arazzo Specification ([#2672](#2672)) ([8443232](8443232))
* **rulesets:** support 2.1.0, 2.2.0, 2.3.0 AsyncAPI versions ([#2067](#2067)) ([2f1d7bf](2f1d7bf))
* **rulesets:** support AsyncAPI 2.4 ([#2146](#2146)) ([7364b2d](7364b2d))
* **rulesets:** support AsyncAPI 2.6.0 ([#2391](#2391)) ([94a7801](94a7801))
* **rulesets:** validate API security in oas-operation-security-defined ([#2046](#2046)) ([5540250](5540250))
* **rulesets:** validate channel servers, server securities and operation securities ([#2122](#2122)) ([9accd31](9accd31))

### Performance Improvements

* **core:** bump jsonpath-plus to 7.1.0 ([#2259](#2259)) ([aacdcd7](aacdcd7))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request released team/dnc Team: Dazed and Confused
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid ruleset gives Error [object Object]
7 participants