Why checksums does not have the same behaviors as package and allow 'import' keyword to split

[jmauro]

Question

I was wondering why there is a way to define different path to add package and there is not the mechanism for the checksums? I mean, if you split your packages' configuration, you want to do the same for the checksums, I don't see the point of split the packages files and having a unique big checksums file with all the packages.

Background of the question

No response

Example Code

$ 

Configuration

Note

No response

[suzuki-shunsuke]

Thank you for your feedback.

I was wondering why there is a way to define different path to add package and there is not the mechanism for the checksums?

This is because we have the motivation to split aqua.yaml but don't have the motivation to split aqua-checksums.json.

About the motivation to split aqua.yaml, we mentioned in the document too.

https://aquaproj.github.io/docs/guides/split-config

This is useful for CI. You can execute test and lint only when the specific package is updated.

On the other hand, we have no motivation to split aqua-checksums.json.
We don't want to add the unnecessary complexity, so we don't support splitting aqua-checksums.json at the moment.

If you have any motivations to split aqua-checksums.json, please describe the detail.
Why do you want to split aqua-checksums.json?

[suzuki-shunsuke]

The feature to enforce the checksum verification is interesting and valuable in terms of security.
The feature would be useful for Monorepo.
I created the issue.

• Support enforcing the checksum verification #2702

[jmauro]

Ok this could work but in that case the aqua.yml has to be a link to a common aqua.yaml (like that if the file has to be updated for all team it is done once). That's why having some kind of inheritance would be easier:

/etc/aqua.d/
  aqua.yaml
  aqua-checksums.json
  teamA.d/
    aqua.yaml
    aqua-checksums.json
  teamB.d/
    aqua.yaml
    aqua-checksums.json

aqua.yaml

checksum:
  enabled: true
  require_checksum: true
  supported_envs:
    - linux/amd64
registries:
  - type: standard
    ref: v4.138.0 # renovate: depName=aquaproj/aqua-registry
packages:
  - import: "*.d/*.yml"

teamA/aqua.yml

---
packages:
  - name: helm/helm@v3.14.0
    description: The Kubernetes Package Manager
    link: https://github.com/helm/helm

teamA/aqua-checksums.json

{
  "checksums": [
    {
      "id": "http/get.helm.sh/helm-v3.14.0-linux-amd64.tar.gz",
      "checksum": "F43E1C3387DE24547506AB05D24E5309C0CE0B228C23BD8AA64E9EC4B8206651",
      "algorithm": "sha256"
    }
  ]
}

And just doing a deep merge would help.
But I can understand it would bring complexity.

On a larger point of view, having inheritance for every top level (registry, packages, checksums) could easy this kind of setup

[suzuki-shunsuke]

the aqua.yml has to be a link to a common aqua.yaml (like that if the file has to be updated for all team it is done once).

Sorry, what do you mean?
As I mentioned you can specify multiple global configuration files, so you just update common aqua.yaml then the update would be reflected to all teams. https://github.com/orgs/aquaproj/discussions/2699#discussioncomment-8550478

Generally speaking, the inheritance has both pros and cons.
The cons is that it is difficult to understand the merged settings and the implementation gets complicated.
To understand the merged settings, we need to check all files (I talk about not only aqua but also more general stuffs).
So we avoid the inheritance unless the inheritance is really necessary when we develop tools.

[jmauro]

Does not seem to work the way I understood it:

$ export AQUA_GLOBAL_CONFIG=/etc/aqua/team.d/aqua.yml:/etc/aqua/aqua.yml
$ cat /etc/aqua/team.d/aqua.yml
---
packages:
  - name: aws/aws-cli@2.15.21
    description: The AWS Command Line Interface (AWS CLI) is a unified tool that provides a consistent interface for interacting with all parts of Amazon Web Services
    link: https://github.com/aws/aws-cli
$ cat /etc/aqua/aqua.yml
---
# aqua - Declarative CLI Version Manager
# https://aquaproj.github.io/
checksum:
  enabled: true
  require_checksum: true
  supported_envs:
    - linux/amd64
registries:
  - type: standard
    ref: v4.137.0 # renovate: depName=aquaproj/aqua-registry
  - name: test
    type: local
    path: /opt/registry/registry.yaml
packages:
  - import: "*.d/*.yml"
  - import: "*.d/*.yaml"
 $ cd /etc/aqua/team.d/
$ aqua install -a
aqua install -a
ERRO[0000] install the package                           aqua_version=2.23.1 env=linux/amd64 error="registry isn't found" package_name=aws/aws-cli package_version=2.15.21 program=aqua registry=standard
FATA[0000] aqua failed                                   aqua_version=2.23.1 env=linux/amd64 error="it failed to install some packages" program=aqua

Like I said, I thought there was some kind of inheritance, but it is not the case.

Thanks for taking the time to help me.

[suzuki-shunsuke]

$ cat /etc/aqua/team.d/aqua.yml
---
packages:
  - name: aws/aws-cli@2.15.21
    description: The AWS Command Line Interface (AWS CLI) is a unified tool that provides a consistent interface for interacting with all parts of Amazon Web Services
    link: https://github.com/aws/aws-cli

ERRO[0000] install the package                           aqua_version=2.23.1 env=linux/amd64 error="registry isn't found" package_name=aws/aws-cli package_version=2.15.21 program=aqua registry=standard

You specified /etc/aqua/team.d/aqua.yml in AQUA_GLOBAL_CONFIG, so registries is required.
We can omit registries if the file is imported.

[suzuki-shunsuke]

BTW, I have a trivial suggestion.
In my understanding, you manage tools with aqua on a specific server, right?
If so, you can specify not only os but also arch in checksum.supported_envs.

checksum:
  enabled: true
  require_checksum: true
  supported_envs:
    - linux/amd64 # or linux/arm64

Then aqua gets checksums of only the given os and arch.
This makes aqua-checksums.json slim and prevents aqua from getting unnecessary checksums.

[jmauro]

Thanks for the tips (it helps)