'charm-strict-dependencies' requires at least one requirements file. in field 'parts.charm.charm-strict-dependencies'

[carlcsaposs-canonical]

Bug Description

I'm guessing charm-strict-dependencies defaults to requirements.txt during charmcraft pack? Looks like the same default doesn't apply to charmcraft clean

To Reproduce

charmcraft installed from edge (rev 1824 2.5.0.post16+git23bf390)

1. clone https://github.com/canonical/mysql-router-k8s-operator
2. charmcraft pack
3. charmcraft clean

Bad charmcraft.yaml content:
- 'charm-strict-dependencies' requires at least one requirements file. in field 'parts.charm.charm-strict-dependencies'                                                                                                                              
Full execution log: '/home/ubuntu/.local/state/charmcraft/log/charmcraft-20231124-090230.749172.log'    

Environment

Ubuntu 22.04

charmcraft.yaml

# Copyright 2022 Canonical Ltd.
# See LICENSE file for licensing details.

type: charm
bases:
  - build-on:
      - name: "ubuntu"
        channel: "22.04"
    run-on:
      - name: "ubuntu"
        channel: "22.04"
parts:
  charm:
    override-pull: |
      craftctl default
      if [[ ! -f requirements.txt ]]
      then
          echo 'ERROR: Use "tox run -e build" instead of calling "charmcraft pack" directly' >&2
          exit 1
      fi
    charm-entrypoint: src/kubernetes_charm.py
    prime:
      - charm_version
      - workload_version
      - scripts
    charm-strict-dependencies: true
    build-packages:
      - libffi-dev
      - libssl-dev
      - pkg-config
      - rustc
      - cargo

Relevant log output

charmcraft-20231124-090230.749172.log
charmcraft-20231124-084822.286563.log

[carlcsaposs-canonical]

Issue also on charmcraft latest/candidate (rev 1790 2.5.0.post1+gitf5558b8)

[lengau]

Thanks for the report! This bug is actually in the pack command, as when charm-strict-dependencies: true is set it shouldn't be magicking up requirements.txt for you (instead requiring an explicit list of requirements files).

[lengau]

The only way I've been able to reproduce this is by deleting requirements.txt, but then both pack and clean complain about the lack of a requirements file. Could you check the state of the requirements.txt file when this happens?

Since this behaviour already infers a requirements.txt file (if it exists), I'm going to leave it that way.

[carlcsaposs-canonical]

The only way I've been able to reproduce this is by deleting requirements.txt, but then both pack and clean complain about the lack of a requirements file. Could you check the state of the requirements.txt file when this happens?

Since this behaviour already infers a requirements.txt file (if it exists), I'm going to leave it that way.

oh sorry, I think that might be because of this

    override-pull: |
      craftctl default
      if [[ ! -f requirements.txt ]]
      then
          echo 'ERROR: Use "tox run -e build" instead of calling "charmcraft pack" directly' >&2
          exit 1
      fi

on mysql-router-k8s, we're using a wrapper around charmcraft to generate requirements.txt from poetry.lock

you can run tox run -e build instead of charmcraft pack
or run tox run -e pack-wrapper and mv requirements-last-build.txt requirements.txt before charmcraft pack

[lengau]

Ah, looking at tox.ini I've figured it out. The commands_post includes moving requirements.txt. I've made a PR to create a clean environment to work around these conflicting requirements for now.

We might be able to make the clean command less strict about the existence of requirements.txt later, but it's sadly not feasible right now.

[carlcsaposs-canonical]

Tried reproducing this & wasn't able to (even with same snap revision)

It appears that charmcraft clean does not require requirements.txt

[carlcsaposs-canonical]

Encountering this issue again on 2.5.4 with charmcraft version and charmcraft clean if requirements.txt does not exist

https://github.com/canonical/mysql-router-k8s-operator/ branch wrapper

[carlcsaposs-canonical]

@lengau This is affecting https://github.com/canonical/charming-actions

e.g. failing release CI https://github.com/canonical/mysql-router-k8s-operator/actions/runs/7447379642/job/20259574786

[taurus-forever]

Dear @lengau

Can you please revise your decision to close this without fixing. It affects Data SQL charms... as even charmcraft login or charmcraft upload are causing issues for us, see Carl comments above.

Thank you in advance!

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-2509.

This message was autogenerated

[lengau]

OK @taurus-forever , @carlcsaposs-canonical , I'm convinced :-)

The fix probably won't make it into 3.0.0, but I'm going to consider options for how to better resolve this in Charmcraft.

[lucasgameiroborges]

Hello! Just checking in this issue, I faced the same issue during my onboarding tasks.

[taurus-forever]

@lengau can you please share the plans for the fix here?

We would like to avoid workarounds on our side, ETA will help us a lot. Thank you!

[lengau]

This is fixed in #1808. However, the original error message from the sample file will persist, as Charmcraft doesn't assume a requirements file if none exists. The workaround is to add:

charm-requirements:
  - requirements.txt

to your charm part. Files explicitly listed in charm-requirements won't get an existence check until the relevant part of the build is actually running, so they can be created during build. I've put together a demo PR for mysql-router-k8s-operator here.

[carlcsaposs-canonical]

@lengau I don't think this is fixed

Files explicitly listed in charm-requirements won't get an existence check until the relevant part of the build is actually running

As of charmcraft 2.7.3, adding a file to charm-requirements doesn't solve this issue

$ charmcraft version --format json
Bad charmcraft.yaml content:
- requirements file '/home/ubuntu/mysql-router-k8s-operator/requirements.txt' not found in field 'parts.charm.charm-requirements'
- 'charm-strict-dependencies' requires at least one requirements file. in field 'parts.charm.charm-strict-dependencies'                                                                                                                                                                                                                                                                               
Full execution log: '/home/ubuntu/.local/state/charmcraft/log/charmcraft-20240924-075645.585831.log' 

[carlcsaposs-canonical]

same issue when running charmcraft pack instead of charmcraft version