Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add linter for protobuf schema files #384

Closed
jkowalleck opened this issue Feb 22, 2024 · 3 comments · Fixed by #385
Closed

chore: add linter for protobuf schema files #384

jkowalleck opened this issue Feb 22, 2024 · 3 comments · Fixed by #385
Assignees
Labels
chore: QA A chore related to Quality Assurance format: ProtoBuf

Comments

@jkowalleck
Copy link
Member

jkowalleck commented Feb 22, 2024

we currntly do have linters for the XML schema.
we currntly do have linters for the JSON schema.
we currntly do NOT have linters for the ProtocolBuffer schema, don't we?

goal: introduce an automated tool chain, that can

  • lint protocol buffer schema
    • find malformed schema files
    • find conflicts in field enumeration (duplicate use of a field number, etc ...)
    • optional but highly appreciated: detect breaking changes

this tooling should monitor

  • all current (and future?) protobuff schema files in this repo's schema folder
    • glob schema/bom-*.proto

this tooling should run automatically on:

  • each pullrequest
  • each change in "master" and other relevant branches

the (github) workflow should be encapsulated standalone, not interfering any other/existing workflows

this tooling should be able to run locally

@jkowalleck jkowalleck self-assigned this Feb 22, 2024
@jkowalleck
Copy link
Member Author

jkowalleck commented Feb 22, 2024

a quick internet search found the following candidates for the job

@jkowalleck
Copy link
Member Author

jkowalleck commented Feb 22, 2024

while on it, it might help to add schema annotation to all the test resources(*.textproto)

spec: https://protobuf.dev/reference/protobuf/textformat-spec/#header

example:

# proto-file: schema/bom-1.6.proto
# proto-message: Bom

@jkowalleck jkowalleck changed the title chore: add linter for protobuff schema files chore: add linter for protobuf schema files Feb 22, 2024
@jkowalleck jkowalleck mentioned this issue Feb 22, 2024
@jkowalleck jkowalleck linked a pull request Feb 22, 2024 that will close this issue
8 tasks
@jkowalleck jkowalleck added chore: QA A chore related to Quality Assurance format: ProtoBuf labels Feb 28, 2024
jkowalleck added a commit that referenced this issue Mar 2, 2024
current protobuf schema files are not perfect.

this PR aims to prevent mistakes in the future, while acknowledging
issues from the past.

fixes #384

----

## status
- [x] introduce protobuf QA tools and configure them to our needs
- [x] baseline existing protobuf QA violations - as acknowledgement
- [x] introduce the protobuf QA tools in automated pipeline
- [x] introduce tools that detect and prevent breaking changes (BCD) in
protobuf
- [x] introduce the protobuf BCD tools in automated pipeline
- [x] have our own protobuf test files checked against the schemas


## followup
- [ ] add the appropriate header to `*.textproto`  
see
#384 (comment)
- [ ] create ticket for BC: fix the proto3 schema enum value `0` -- they
are intended to be fallbacks, not actual values.

---------

Signed-off-by: Jan Kowalleck <jan.kowalleck@gmail.com>
@jkowalleck
Copy link
Member Author

all done via #385 and ff

stevespringett added a commit that referenced this issue Apr 9, 2024
## Added

* Core enhancement: Attestation
([#192](#192) via
[#348](#348))
* Core enhancement: Cryptography Bill of Materials — CBOM
([#171](#171),
[#291](#291) via
[#347](#347))
* Feature to express the URL to source distribution
([#98](#98) via
[#269](#269))
* Feature to express the URL to RFC 9116 compliant documents
([#380](#380) via
[#381](#381))
* Feature to express tags/keywords for services and components (via
[#383](#383))
* Feature to express details for component authors
([#335](#335) via
[#379](#379))
* Feature to express details for component and BOM manufacturer
([#346](#346) via
[#379](#379))
* Feature to express communicate concluded values from observed
evidences ([#411](#411)
via [#412](#412))
* Features to express license acknowledgement
([#407](#407) via
[#408](#408))
* Feature to express environmental consideration information for model
cards ([#396](#396) via
[#395](#395))
* Feature to express the address of organizational entities (via
[#395](#395))
* Feature to express additional component identifiers: Universal Bill Of
Receipts Identifier and Software Heritage persistent IDs
([#413](#413) via
[#414](#414))

## Fixed

* Allow multiple evidence identities by XML/JSON schema
([#272](#272) via
[#359](#359))
  This was already correct via ProtoBuff schema.
* Prevent empty `license` entities by XML schema
([#288](#288) via
[#292](#292))
  This was already correct in JSON/ProtoBuff schema.
* Prevent empty or malformed `property` entities by JSON schema
([#371](#371) via
[#375](#375))
  This was already correct in XML/ProtoBuff schema.
* Allow multiple `licenses` in `Metadata` by ProtoBuff schema
([#264](#264) via
[#401](#401))
  This was already correct in XML/JSON schema.

## Changed

* Allow arbitrary `$schema` values by JSON schema
([#402](#402) via
[#403](#403))
* Increased max length of `versionRange` (via
[`3e01ce6`](3e01ce6))
* Harmonized length of `version` (via
[#417](#417))

## Deprecated

* Data model "Component"'s field `author` was deprecated. (via
[#379](#379))
  Use field `authors` or field `manufacturer` instead.
* Data model "Metadata"'s field `manufacture` was deprecated.
([#346](#346) via
[#379](#379))
  Use "Metadata"'s field `component`'s field `manufacturer` instead. 
  - for XML: `/bom/metadata/component/manufacturer`
  - for JSON: `$.metadata.component.manufacturer`
  - for ProtoBuf: `Bom:metadata.component.manufacturer`

## Documentation

* Centralize version and version-range (via
[#322](#322))
* Streamlined SPDX expression related descriptions (via
[#327](#327))
* Enhanced descriptions of `bom-ref`/`refType`
([#336](#336) via
[#344](#344))
* Enhanced readability of enum documentation in JSON schema
([#361](#361) via
[#362](#362))
* Fixed typo "compliment" -> "complement" (via
[#369](#369))
* Added documentation for enum "ComponentScope"'s values in JSON schema
([#293](#293) via
[`d92e58e`](d92e58e))
  Texts were a taken from the existing ones in XML/ProtoBuff schema.
* Added documentation for enum "TaskType"'s values
([#245](#245) via
[#377](#377))
* Improve documentation for data model "Metadata"'s field `licenses`
([#273](#273) via
[#378](#378))
* Added documentation for enum "MachineLearningApproachType"'s values
([#351](#351) via
[#416](#416))
* Rephrased some texts here and there.

## Test data

* Added test data for newly added use cases
* Added quality assurance for our ProtoBuf schemas
([#384](#384) via
[#385](#385))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore: QA A chore related to Quality Assurance format: ProtoBuf
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant