Validation of DCAT-US record

[jbrown-xentity]

User Story

In order to validate a DCAT-US record, data.gov admins (and community) wants a well defined json-schema file.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

• GIVEN a valid DCAT-US record exists and the validation file
  WHEN the record is processed via the validation file
  THEN the record passes
  AND a test replicates this process

• GIVEN an invalid DCAT-US record exists and the validation file
  WHEN the invalid record is processed via the validation file
  THEN the record fails with meaningful response
  AND a test replicates this process

Background

We currently have a custom configuration file that plugs into some custom logic. We theoretically do this now, except we use some strange function called Draft4Validator instead of the standard implementation.

Security Considerations (required)

None

Sketch

This might be as simple as copying https://github.com/GSA/ckanext-datajson/tree/main/ckanext/datajson/pod_schema/federal-v1.1 to repo created from #4255, and installing https://github.com/python-jsonschema/jsonschema#jsonschema and validating working as expected. We would like to test against our current test files here. There might be new features/broken items lingering, so consider implementing some of the type of tests here (though not the actual code, as much of that isn't what we want or expect the output to look like).

[nickumia-reisys]

There is an code architecture question of how to best abstract this code and organize it so that it makes sense. What other functions will need to call the validation function? Are we going for a functional vs. object-oriented design? What are the high level abstractions other than "E", "T", "L" that we want to have (if any)?

[jbrown-xentity]

As part of cleanup of the repo, also look to tackle:

• Implement poetry setup as an action: https://github.com/marketplace/actions/python-poetry-action
• Pair with GSA gov team members to publish on pypi like we do for other python repo's, requires secrets setup (or if not possible, create ticket for this item).

[rshewitt]

Ruff doesn't like unused imports. The issue derives from an imported fixture function being used as a string within @pytest.mark.parametrize( "args...", [ ( "fixture_function", file_to_validate.json, is_valid ) ] ). This has been resolved. I learned about the conftest.py file for storing fixtures and making them accessible to other files.

[rshewitt]

jsonschema validation errors

[rshewitt]

pytest-depends plugin

[jbrown-xentity]

Discovered that json-schema has releases to the spec. Since our spec uses release 4, we need to follow the upgrade path from the official documentation to get on the most supported/most recent json-schema spec.
This will also allow us to get response for all validation errors (not just the first one found), and will allow us to give complete feedback/changes needed to data provider.

[rshewitt]

json schema release transformer