Skip to content
This repository has been archived by the owner on Jun 13, 2021. It is now read-only.

Compose validation #760

Merged
merged 9 commits into from
Nov 28, 2019
Merged

Compose validation #760

merged 9 commits into from
Nov 28, 2019

Conversation

rumpl
Copy link
Member

@rumpl rumpl commented Nov 21, 2019

- What I did

Created a "framework" to help write validation rules for a compose file, the Validator will do the heavy lifting of traversing all the properties in the compose file.

The validator works in two phases:

  • the frist is the collection phase where the validator traverses the whole compose file and passes the values to the rules. Rules can collect things they would need later for validation. For example, the relativepathrule needs the top level volumes to check if the volume exists.
  • the second phase is the validation phase

A rule has 3 parts:

The Collect method is called in the first phase, rules collect dependent data if needed here.
The Accept method is used to filter out only thing that a rule needs to validate
The Validate method does the actual validation and returns a list of errors for a given value.

The PR introduces two rules:

  • externalsecretrule to force all secrets to be external
  • relativepathrule to ban relative paths in volumes

In a followup we should:

  • create rules for the remaining check that we have in the build command
  • add the validation to the validate command

I won't do them here since the changeset is starting to get big.

- How to verify it
Unit and e2e tests were added for the two rules

- A picture of a cute animal (not mandatory)
image

@codecov
Copy link

codecov bot commented Nov 21, 2019

Codecov Report

Merging #760 into master will increase coverage by 1.2%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #760     +/-   ##
=========================================
+ Coverage   70.07%   71.27%   +1.2%     
=========================================
  Files          64       67      +3     
  Lines        3519     3837    +318     
=========================================
+ Hits         2466     2735    +269     
- Misses        727      768     +41     
- Partials      326      334      +8
Impacted Files Coverage Δ
internal/commands/inspect.go 18.03% <0%> (-32.79%) ⬇️
internal/validator/rules/relativepath.go 100% <0%> (ø)
internal/validator/rules/externalsecrets.go 100% <0%> (ø)
internal/validator/validator.go 70.49% <0%> (ø)
internal/store/bundle.go 82.39% <0%> (+5.98%) ⬆️
internal/commands/build/compose.go 86.66% <0%> (+8.09%) ⬆️
internal/packager/extract.go 32.2% <0%> (+8.39%) ⬆️
internal/packager/init.go 73.24% <0%> (+8.6%) ⬆️
internal/commands/image/rm.go 69.76% <0%> (+11.14%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b621499...582f377. Read the comment docs.

for _, p := range m {
str, ok := p.(string)
if !ok {
errs = append(errs, fmt.Errorf("invalid volume in service %q", s.service))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if those checks could be removed, if we validate the compose file first using the json schema.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once we have schema validation then yeah, sure

Copy link
Contributor

@silvin-lubecki silvin-lubecki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with small nits

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Remove the no longer used check for the relative path

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
This check is done by the packager now

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
The map implementation of go iterates a map random;y so we can't know
which error has what message.

It's ok to remove this here because we have an e2e test (the final
errors are sorted alphabetically).

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
A secret *must* be external, not *should*

Signed-off-by: Djordje Lukic <lukic.djordje@gmail.com>
internal/packager/extract.go Outdated Show resolved Hide resolved
internal/validator/rules/externalsecrets.go Outdated Show resolved Hide resolved
internal/validator/rules/relativepath.go Outdated Show resolved Hide resolved
It can have more than one `:` in its definition.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Copy link
Member

@eunomie eunomie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eunomie eunomie merged commit e18898a into docker:master Nov 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants