Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Luet fail to create new repository with metdata #1189

Closed
mudler opened this issue Mar 21, 2022 · 7 comments · Fixed by #1197
Closed

Luet fail to create new repository with metdata #1189

mudler opened this issue Mar 21, 2022 · 7 comments · Fixed by #1197
Assignees
@mudler
Labels
kind/bug Something isn't working prio/critical

Comments

@mudler
Copy link
Contributor

mudler commented Mar 21, 2022

See #1188 (comment)

CI run https://github.com/rancher-sandbox/cOS-toolkit/runs/5624238784?check_suite_focus=true
@mudler mudler added the kind/bug Something isn't working label Mar 21, 2022
@mudler mudler self-assigned this Mar 21, 2022
@mudler mudler removed their assignment Mar 21, 2022
@mudler mudler changed the title Luet fail to create new repository with mtree metdata Luet fail to create new repository with metdata Mar 21, 2022
@mudler
Copy link
Contributor Author

mudler commented Mar 21, 2022

Doesn't seem related to mtree. so I'm wondering how it slipped throughout tests

@mudler
Copy link
Contributor Author

mudler commented Mar 21, 2022

I can't reproduce it locally by using integration tests from luet with go1.18

@mudler
Copy link
Contributor Author

mudler commented Mar 21, 2022

Locally seems to pass:

❯ sudo -E make LUET=$PWD/luet-0.30.0-linux-amd64 PACKAGES="utils/helm" create-repo
/home/ettore/_git/cOS/luet-0.30.0-linux-amd64 create-repo --debug --tree "/home/ettore/_git/cOS/packages" \
    --output /home/ettore/_git/cOS/build \
    --from-metadata \
    --packages /home/ettore/_git/cOS/build \
    --name "cOS" \
    --descr "cOS green" \
    --urls "" \
    --tree-compression zstd \
    --tree-filename tree.tar \
    --meta-compression zstd \
    --type http
  DEBUG   (config.go:#204:github.com/mudler/luet/cmd/util.initContext)  System rootfs: /
  DEBUG   (config.go:#205:github.com/mudler/luet/cmd/util.initContext)  Colors true
  DEBUG   (config.go:#206:github.com/mudler/luet/cmd/util.initContext)  Logging level info
  DEBUG   (config.go:#207:github.com/mudler/luet/cmd/util.initContext)  Debug mode true
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding efi/arm64-0.1 from metadata file /home/ettore/_git/cOS/build/arm64-efi-0.1.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding system/cos-0.6.5+6 from metadata file /home/ettore/_git/cOS/build/cos-system-0.6.5+6.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding toolchain/dasel-1.22.1 from metadata file /home/ettore/_git/cOS/build/dasel-toolchain-1.22.1.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding utils/helm-3.7.2 from metadata file /home/ettore/_git/cOS/build/helm-utils-3.7.2.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding system/immutable-rootfs-0.1.1+10 from metadata file /home/ettore/_git/cOS/build/immutable-rootfs-system-0.1.1+10.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding toolchain-fips/luet-makeiso-0.3.8+8 from metadata file /home/ettore/_git/cOS/build/luet-makeiso-toolchain-fips-0.3.8+8.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding kernel/odroid-c2-3.14 from metadata file /home/ettore/_git/cOS/build/odroid-c2-kernel-3.14.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding kernel/odroid-c2-3.16 from metadata file /home/ettore/_git/cOS/build/odroid-c2-kernel-3.16.metadata.yaml
  DEBUG   (repository.go:#329:github.com/mudler/luet/pkg/installer.GenerateRepository.func1)  Adding utils/rancherd-0.0.1-alpha07+1 from metadata file /home/ettore/_git/cOS/build/rancherd-utils-0.0.1-alpha07+1.metadata.yaml
 INFO   Repository cOS: creating revision 3 and last update 1647857846...

@mudler mudler self-assigned this Mar 21, 2022
@mudler
Copy link
Contributor Author

mudler commented Mar 21, 2022

I've managed to retrigger it. the issue is when creating the packages metadata files. The move from helm to sprig slightly changed how things are rendered:

     BuildValues: ([]map[string]interface {}) (len=1 cap=1) {                                                                                                                                          [83/1882]      (map[string]interface {}) (len=14) {                                                                                                                                                                      
       (string) (len=4) "name": (string) (len=4) "helm",                                                                                                                                                        
       (string) (len=8) "packages": (string) (len=542) "systemd-sysvinit grub2-i386-pc grub2-x86_64-efi iproute2 squashfs conntrack-tools findutils haveged lsscsi lvm2 mdadm multipath-tools nfs-utils open-isc
si open-vm-tools python-azure-agent qemu-guest-agent rng-tools systemd vim parted dracut e2fsprogs dosfstools kernel-firmware-bnx2 kernel-firmware-i915 kernel-firmware-intel kernel-firmware-iwlwifi kernel-fir
mware-mellanox kernel-firmware-network kernel-firmware-platform kernel-firmware-realtek coreutils less device-mapper grub2 which curl nano gawk haveged tar rsync timezone jq gptfdisk",                        
       (string) (len=7) "version": (string) (len=7) "3.8.1-1",                                                                                                                                                  
       (string) (len=4) "arch": (string) (len=6) "x86_64",                                                                                                                                                      
       (string) (len=8) "codename": (string) (len=5) "green",                                                                                                                                                   
       (string) (len=11) "description": (string) (len=30) "The Kubernetes Package Manager",                                                                                                                     
       (string) (len=14) "kernel_package": (string) (len=14) "kernel-default",                                                                                                                                  
       (string) (len=6) "labels": (map[interface {}]interface {}) (len=3) {                                                                                                                                     
        (string) (len=25) "autobump.skip_if_contains": (string) (len=6) "[\"rc\"]",                                                                                                                             
        (string) (len=12) "github.owner": (string) (len=4) "helm",                                                                                                                                              
        (string) (len=11) "github.repo": (string) (len=4) "helm"                                                                                                                                                
       },                                                                                                                                                                                                       
       (string) (len=7) "license": (string) (len=18) "Apache-2.0 License",                                                                                                                                      
--                                                                                                                                                                                                              
      MaxAttempts: (int) 0,

(string) (len=6) "labels": (map[interface {}]interface {}) seems to be the culprit here

@mudler
Copy link
Contributor Author

mudler commented Mar 21, 2022

Confirming we have been hit by go-yaml/yaml#139 . Updating the yaml dependency to v3 solved the issue locally.

mudler added a commit to mudler/luet that referenced this issue Mar 21, 2022
@mudler
🐛 Update yaml to v3
114d489 
The recent switch to mergo throws in a yaml version which is affected
by go-yaml/yaml#139, and caused
rancher/elemental-toolkit#1189.

Updating to yaml.v3 where it was affected.

Note mergo still uses v2, so this can be problematic on other areas
darccio/mergo#206.
@mudler
Copy link
Contributor Author

mudler commented Mar 23, 2022

x86_64 is fixed, fixing now arm64 builds

@mudler mudler linked a pull request Mar 23, 2022 that will close this issue
ci: Drop override #1197
Merged
@mudler
Copy link
Contributor Author

mudler commented Mar 23, 2022

arm64 builds should be good by now. Dropping version override in #1197

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mudler mudler

Labels
kind/bug Something isn't working prio/critical
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ci: Drop override rancher/elemental-toolkit
1 participant
@mudler