Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for unique-per-image tag (was DIGEST_HEX) #2162

Closed
elisiano opened this issue May 21, 2019 · 23 comments
Closed

Add support for unique-per-image tag (was DIGEST_HEX) #2162

elisiano opened this issue May 21, 2019 · 23 comments
Assignees
Labels
area/tag bugbash/q4-2022 Issues related to Skaffold Bugbash Q4 2022 help wanted We would love to have this done, but don't have the bandwidth, need help from contributors kind/feature-request priority/p3 agreed that this would be good to have, but no one is available at the moment.

Comments

@elisiano
Copy link

In #1482 the tagging mechanism changed.

Expected behavior

with a tagging policy like this:

[...]
  tagPolicy:
    envTemplate:
      template: '{{.IMAGE_NAME}}:something-'
[...]

I was expecting to see the digest appended automatically to the name.

Actual behavior

The digest is not appended

Information

  • Skaffold version: v0.29
  • Operating system: macOS/Linux
  • Contents of skaffold.yaml:
apiVersion: skaffold/v1beta10
kind: Config
build:
  artifacts:
  - image: quay.io/somecompany/someimage
    context: ./docker/path
    docker:
      dockerfile: Dockerfile
  tagPolicy:
    envTemplate:
      template: '{{.IMAGE_NAME}}:something-'
  local: {}
deploy:
  kustomize:
    path: somepath

Steps to reproduce the behavior

  1. start minikube
  2. make sure it's the default context
  3. eval $(minikube docker-env)
  4. skaffold build
  5. docker images
@briandealwis
Copy link
Member

For example, I applied the following change to examples/getting-started to use {{.DIGEST_HEX}}:

--- a/examples/getting-started/skaffold.yaml
+++ b/examples/getting-started/skaffold.yaml
@@ -1,6 +1,9 @@
 apiVersion: skaffold/v1beta10
 kind: Config
 build:
+  tagPolicy:
+    envTemplate: 
+      template: "{{.IMAGE_NAME}}:{{.USER}}{{.DIGEST_HEX}}"
   artifacts:
   - image: gcr.io/k8s-skaffold/skaffold-example
 deploy:

skaffold dev then produces a warning

Generating tags...
 - gcr.io/k8s-skaffold/skaffold-example -> WARN[0000] {{.DIGEST}}, {{.DIGEST_ALGO}} and {{.DIGEST_HEX}} are deprecated, image digest will now automatically be appended to image tags 
gcr.io/k8s-skaffold/skaffold-example:bsd_DEPRECATED_DIGEST_HEX_

But the images are tagged as my username and do not have a digest appended.

@dgageot dgageot added area/tag kind/bug Something isn't working labels May 27, 2019
@nkubala
Copy link
Contributor

nkubala commented Jun 19, 2019

this is the intended behavior of the tagger. the envTemplate tagger will generate a tag based exactly on what is provided in the skaffold.yaml: it won't append anything else on there without being explicitly specified by the user.

we compute the tags for image BEFORE we build them, so there's no image ID for us to use when generating the tag. we can't use the image digest for local images anyway, because we don't actually have the digest (this would require pushing the image, then retrieving the image manifest from the remote registry where we pushed it).

@dgageot
Copy link
Contributor

dgageot commented Jun 20, 2019

Skaffold builds the image then tags it with {{.IMAGE_NAME}}:something- with, in your case resolves to quay.io/somecompany/someimage:something-.

If you run skaffold dev or skaffold run, it won't put this value in the yaml files though because that would be the same value no matter what the content of the image is -> Kubernetes would never update the running container.

So here's what Skaffold does. It depends wether images are pushed or not:

  • If images are pushed (working with a remote k8s), then skaffold will push the quay.io/somecompany/someimage:something- tag and will use the fully qualified images name in the k8s manifests. Something like quay.io/somecompany/someimage:something-@sha2546:4ac032ecafcf2a6c3ed9f40cdd997028f42d1afb9e3eeda9c7345504308a7e1a
  • If images are not pushed, there's no image digest so skaffold fallsback to tagging the image with it's local imageID and use that tag in the k8s manifests. So the same image locally has two tags. One that's mutable: quay.io/somecompany/someimage:something- and one that's immutable: quay.io/somecompany/someimage:IMAGE_ID

I wish it wouldn't be two different behaviours but since this is due to Docker's behaviour, there's little Skaffold can do.

@elisiano
Copy link
Author

what I'm looking for is an alternative to the old {{DIGEST_HEX}} (available before pushing).
It would need to be something unique per image (dependent on the content). Doesn't matter if it's a real digest or not.

@dgageot how would I go about using the immutable tag in skaffold config? I tried {{.IMAGE_ID}} but it doesn't contain any value.

Error parsing reference: "quay.io/somecompany/someimage:something-<no value>" is not a valid repository/tag: invalid reference format

if interested, our use case is to have a common repository used as a playground for users to test their images before finalizing the projects and creating the repos.
As such we need a way to differentiate images with a custom prefix in the tag.
example:

  • quay.io/somecompany/experimental:project1-$DIGEST
  • quay.io/somecompany/experimental:project2-$DIGEST
  • and so on

The old DIGEST vars would allow us to achieve the above. Now them being removed leaves us in a place where we cannot update skaffold.

Is there any alternative mechanism with which we can achieve a similar result as the one discussed above?

@dgageot dgageot self-assigned this Jun 26, 2019
@balopat balopat added help wanted We would love to have this done, but don't have the bandwidth, need help from contributors and removed help wanted We would love to have this done, but don't have the bandwidth, need help from contributors labels Jul 2, 2019
@balopat balopat added the priority/p1 High impact feature/bug. label Jul 16, 2019
@dgageot
Copy link
Contributor

dgageot commented Aug 30, 2019

Hi @elisiano, I wonder if there's any way you can test #2772 and see if solves your issue.

@elisiano
Copy link
Author

elisiano commented Sep 2, 2019

@dgageot if I read that correctly, I should use the {{.DIGEST}} var in the template?

@dgageot
Copy link
Contributor

dgageot commented Sep 2, 2019

@elisiano yes. That's more a Fully Qualified Name than a digest. We kept the name for compatibility reasons but we might want to revisit that.

@morozov
Copy link
Contributor

morozov commented Sep 2, 2019

This is what I tried:

$ git diff
diff --git a/examples/bazel/skaffold.yaml b/examples/bazel/skaffold.yaml
index ef6b3b5b..3151111e 100644
--- a/examples/bazel/skaffold.yaml
+++ b/examples/bazel/skaffold.yaml
@@ -5,3 +5,6 @@ build:
   - image: gcr.io/k8s-skaffold/skaffold-bazel
     bazel:
       target: //:skaffold_example.tar
+  tagPolicy:
+    envTemplate:
+      template: '{{.IMAGE_NAME}}:{{.DIGEST}}'

$ ../../out/skaffold version
v0.37.0-82-g4899ae57

$ ../../out/skaffold build
Generating tags...
 - gcr.io/k8s-skaffold/skaffold-bazel -> WARN[0000] {{.DIGEST}}, {{.DIGEST_ALGO}} and {{.DIGEST_HEX}} are deprecated, image digest will now automatically be appended to image tags
gcr.io/k8s-skaffold/skaffold-bazel:_DEPRECATED_DIGEST_
Tags generated in 133.374µs
Checking cache...
 - gcr.io/k8s-skaffold/skaffold-bazel: Not found. Building
Cache check complete in 134.010476ms
Starting build...
Building [gcr.io/k8s-skaffold/skaffold-bazel]...
INFO: Analyzed target //:skaffold_example.tar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:skaffold_example.tar up-to-date:
  bazel-bin/skaffold_example.tar
INFO: Elapsed time: 0.085s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
FATA[0000] build failed: build failed: building [gcr.io/k8s-skaffold/skaffold-bazel]: build artifact: writing image "gcr.io/k8s-skaffold/skaffold-bazel:_DEPRECATED_DIGEST_": UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication

It seems like the digest is not replaced in the image name.

@dgageot
Copy link
Contributor

dgageot commented Sep 2, 2019

Yeah, this whole envTemplate/DIGEST/DIGEST_ALGO/DIGEST_HEX thing is a mess.

envTemplate used to make sense and be useful. I'm not sure it's very useful now.

  • If you remove the tagPolicy config, everything should work out of the box.
  • One thing that envTemplate can do for you is to add a prefix/suffix to images. Fixed ones or from env variables. Something like:
tagPolicy:
    envTemplate:
        template: 'prefix-{{.IMAGE_NAME}}-{{.SUFFIX}}:{{.TAG}}'

The digest will then be automatically be appended without you having to use {{.DIGEST}}

@morozov
Copy link
Contributor

morozov commented Sep 2, 2019

If you remove the tagPolicy config, everything should work out of the box.

If I remove it entirely and use the same example as above, I get:

$ ../../out/skaffold build
Generating tags...
 - gcr.io/k8s-skaffold/skaffold-bazel -> gcr.io/k8s-skaffold/skaffold-bazel:v0.37.0-82-g4899ae57
Tags generated in 20.19983ms
Checking cache...
 - gcr.io/k8s-skaffold/skaffold-bazel: Not found. Building
Cache check complete in 125.299096ms
Starting build...
Building [gcr.io/k8s-skaffold/skaffold-bazel]...
INFO: Analyzed target //:skaffold_example.tar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:skaffold_example.tar up-to-date:
  bazel-bin/skaffold_example.tar
INFO: Elapsed time: 0.143s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
FATA[0001] build failed: build failed: building [gcr.io/k8s-skaffold/skaffold-bazel]: build artifact: writing image "gcr.io/k8s-skaffold/skaffold-bazel:v0.37.0-82-g4899ae57": UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication

The image gets tagged with git-sourced attributes which is not what I want. In the case when I build multiple images from the same repo, I want images re-tagged only if their content has been changed by a given git commit (hence, I'm using the sha256 tagger with skaffold v0.21).

One thing that envTemplate can do for you is to add a prefix/suffix to images. Fixed ones or from env variables. Something like:

tagPolicy:
    envTemplate:
        template: 'prefix-{{.IMAGE_NAME}}-{{.SUFFIX}}:{{.TAG}}'

Patching the same example with the above yields:

../../out/skaffold build
Generating tags...
 - gcr.io/k8s-skaffold/skaffold-bazel -> prefix-gcr.io/k8s-skaffold/skaffold-bazel-<no value>:<no value>
Tags generated in 76.244µs
Checking cache...
 - gcr.io/k8s-skaffold/skaffold-bazel: Not found. Building
Cache check complete in 62.95956ms
Starting build...
Building [gcr.io/k8s-skaffold/skaffold-bazel]...
INFO: Analyzed target //:skaffold_example.tar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:skaffold_example.tar up-to-date:
  bazel-bin/skaffold_example.tar
INFO: Elapsed time: 0.150s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
FATA[0000] build failed: build failed: building [gcr.io/k8s-skaffold/skaffold-bazel]: build artifact: parsing tag "prefix-gcr.io/k8s-skaffold/skaffold-bazel-<no value>:<no value>": tag can only contain the runes `abcdefghijklmnopqrstuvwxyz0123456789_-.ABCDEFGHIJKLMNOPQRSTUVWXYZ`: <no value> 

Specifically, the {{.TAG}} attribute doesn't contain a value.

@dgageot could you try using examples/bazel/skaffold.yaml as the baseline and make a working example of what could be used as an alternative to the sha256 tagger?

@dgageot
Copy link
Contributor

dgageot commented Sep 3, 2019

@morozov Sorry, I should have said that in my envTemplate usage example, SUFFIX and TAG are two env variables.

These 3 examples should work:

apiVersion: skaffold/v1beta13
kind: Config
build:
  tagPolicy:
    envTemplate:
      template: "{{.IMAGE}}"
  artifacts:
    - image: gcr.io/k8s-skaffold/skaffold-bazel
      bazel:
        target: //:skaffold_example.tar
apiVersion: skaffold/v1beta13
kind: Config
build:
  tagPolicy:
    gitCommit: {}
  artifacts:
    - image: gcr.io/k8s-skaffold/skaffold-bazel
      bazel:
        target: //:skaffold_example.tar

apiVersion: skaffold/v1beta13
kind: Config
build:
  tagPolicy:
    sha256: {}
  artifacts:
    - image: gcr.io/k8s-skaffold/skaffold-bazel
      bazel:
        target: //:skaffold_example.tar

@morozov
Copy link
Contributor

morozov commented Sep 3, 2019

Thank you @dgageot. Unfortunately, neither of the examples above works as I'd expect:

  1. apiVersion: skaffold/v1beta13
    kind: Config
    build:
      tagPolicy:
        envTemplate:
          template: "{{.IMAGE}}"
      artifacts:
        - image: gcr.io/k8s-skaffold/skaffold-bazel
          bazel:
            target: //:skaffold_example.tar
    $ ../../out/skaffold build
    Generating tags...
     - gcr.io/k8s-skaffold/skaffold-bazel -> <no value>
    Tags generated in 489.458µs
    Checking cache...
     - gcr.io/k8s-skaffold/skaffold-bazel: Not found. Building
    Cache check complete in 508.219369ms
    Starting build...
    Building [gcr.io/k8s-skaffold/skaffold-bazel]...
    INFO: Analyzed target //:skaffold_example.tar (1 packages loaded, 2 targets configured).
    INFO: Found 1 target...
    Target //:skaffold_example.tar up-to-date:
      bazel-bin/skaffold_example.tar
    INFO: Elapsed time: 1.238s, Critical Path: 0.66s
    INFO: 12 processes: 12 linux-sandbox.
    INFO: Build completed successfully, 19 total actions
    FATA[0002] build failed: build failed: building [gcr.io/k8s-skaffold/skaffold-bazel]: build artifact: parsing tag "<no value>": repository can only contain the runes `abcdefghijklmnopqrstuvwxyz0123456789_-./`: <no value> 
    
    The tag is <no value>. Probably some values/configuration are still missing.
  2. apiVersion: skaffold/v1beta13
    kind: Config
    build:
      tagPolicy:
        gitCommit: {}
      artifacts:
        - image: gcr.io/k8s-skaffold/skaffold-bazel
          bazel:
            target: //:skaffold_example.tar
    $ ../../out/skaffold build
    Generating tags...
     - gcr.io/k8s-skaffold/skaffold-bazel -> gcr.io/k8s-skaffold/skaffold-bazel:v0.37.0-82-g4899ae57-dirty
    Tags generated in 27.503497ms
    Checking cache...
     - gcr.io/k8s-skaffold/skaffold-bazel: Not found. Building
    Cache check complete in 159.692667ms
    Starting build...
    Building [gcr.io/k8s-skaffold/skaffold-bazel]...
    INFO: Analyzed target //:skaffold_example.tar (0 packages loaded, 0 targets configured).
    INFO: Found 1 target...
    Target //:skaffold_example.tar up-to-date:
      bazel-bin/skaffold_example.tar
    INFO: Elapsed time: 0.106s, Critical Path: 0.00s
    INFO: 0 processes.
    INFO: Build completed successfully, 1 total action
    FATA[0004] build failed: build failed: building [gcr.io/k8s-skaffold/skaffold-bazel]: build artifact: writing image "gcr.io/k8s-skaffold/skaffold-bazel:v0.37.0-82-g4899ae57-dirty": UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication 
    
    The tag is derived from git (v0.37.0-82-g4899ae57-dirty), not from the file contents.
  3. apiVersion: skaffold/v1beta13
    kind: Config
    build:
      tagPolicy:
        sha256: {}
      artifacts:
        - image: gcr.io/k8s-skaffold/skaffold-bazel
          bazel:
            target: //:skaffold_example.tar
    $ ../../out/skaffold build
    Generating tags...
     - gcr.io/k8s-skaffold/skaffold-bazel -> gcr.io/k8s-skaffold/skaffold-bazel:latest
    Tags generated in 55.731µs
    Checking cache...
     - gcr.io/k8s-skaffold/skaffold-bazel: Not found. Building
    Cache check complete in 61.56631ms
    Starting build...
    Building [gcr.io/k8s-skaffold/skaffold-bazel]...
    INFO: Analyzed target //:skaffold_example.tar (0 packages loaded, 0 targets configured).
    INFO: Found 1 target...
    Target //:skaffold_example.tar up-to-date:
      bazel-bin/skaffold_example.tar
    INFO: Elapsed time: 0.079s, Critical Path: 0.00s
    INFO: 0 processes.
    INFO: Build completed successfully, 1 total action
    FATA[0000] build failed: build failed: building [gcr.io/k8s-skaffold/skaffold-bazel]: build artifact: writing image "gcr.io/k8s-skaffold/skaffold-bazel:latest": UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication 
    
    The tag is latest.

@dgageot
Copy link
Contributor

dgageot commented Sep 4, 2019

Sorry @morozov, I made a type. Instead of {{.IMAGE}}, it should have been {{.IMAGE_NAME}}

Now, the feature you are asking is not currently possible. There's no tagger that gives you the digests of the inputs. Since it was asked many times, I'll write a proposal.

@dgageot
Copy link
Contributor

dgageot commented Sep 4, 2019

See #2804

@elisiano
Copy link
Author

elisiano commented Sep 5, 2019

@dgageot I had a look at the #2804 proposal and I think it's still missing what we're looking for.
The functionality you're describing for sha256 should also be somehow available for the envTemplate tagger for the reason specified in #2162 (comment).

@dgageot
Copy link
Contributor

dgageot commented Sep 5, 2019

@elisiano Thanks a lot for the feedback! I'll update the proposal.

@tejal29
Copy link
Member

tejal29 commented Sep 24, 2019

Waiting for the proposal to get in.

@balopat balopat added priority/p2 May take a couple of releases and removed priority/p1 High impact feature/bug. labels Oct 14, 2019
@douglascamata
Copy link

I was also surprisingly affected by this issue. Reading the official documentation about the tagging strategies gives the impression that sha256 will put a digest in the image tag and it doesn't. Can the official docs be update to avoid this confusion until the situation is fixed?

@dgageot dgageot removed their assignment Feb 28, 2020
@Norbo11
Copy link

Norbo11 commented Mar 12, 2020

For those that are trying to consume the tag that skaffold generates on skaffold build, I am running it with skaffold build --file-output /dev/stdout which prints out the full tag name (including image digest) regardless of your tagPolicy.

@morozov
Copy link
Contributor

morozov commented Mar 12, 2020

@Norbo11 so what does the process look like? You build the image once to get the digest and then rebuild it with the tag?

@Norbo11
Copy link

Norbo11 commented Mar 17, 2020

@Norbo11 so what does the process look like? You build the image once to get the digest and then rebuild it with the tag?

Not sure what your specific use-case is, but for me, I just wanted to use Skaffold to build me an image and tag it with something that included the digest of the contents. I needed to retrieve that tag somehow to pass it to our deployment mechanism (we are not using Skaffold to deploy). Skaffold refused to output anything that included the digest.

If you pass --file-output some_file.json and then read the contents of that file after running the skaffold build command, it will contain the tag you need. I am not sure what running skaffold run or skaffold deploy would do after that, but it appears to me that Skaffold seems to build an image that is tagged with the contents digest by default without needing to specify the tagPolicy field.

@tstromberg tstromberg added the triage/discuss Items for discussion label Apr 24, 2020
@tstromberg
Copy link
Contributor

Will bring this up for discussion again now that there is a proposal: https://github.com/GoogleContainerTools/skaffold/blob/master/docs/design_proposals/digest-tagger.md

@tstromberg tstromberg added priority/p3 agreed that this would be good to have, but no one is available at the moment. and removed priority/p2 May take a couple of releases labels Apr 24, 2020
@tstromberg tstromberg changed the title Digest not appended to image tag Add support for a tag that is unique per image (was DIGEST_HEX) Apr 27, 2020
@tstromberg tstromberg changed the title Add support for a tag that is unique per image (was DIGEST_HEX) Add support for unique-per-image tag (was DIGEST_HEX) Apr 27, 2020
@balopat balopat removed the triage/discuss Items for discussion label Apr 27, 2020
@dgageot dgageot added kind/feature-request and removed kind/bug Something isn't working labels May 5, 2020
@nkubala nkubala added this to the Icebox [P2+] milestone Sep 1, 2020
@nkubala nkubala removed this from the Icebox [P2+] milestone May 11, 2021
@aaron-prindle
Copy link
Contributor

I believe this was fixed with:
#5192

which added the inputDigest tagger, more info in the docs here:
https://skaffold.dev/docs/pipeline-stages/taggers/

Closing as fixed, feel free to comment here and I will re-open if there is some unaddressed issue(s) here

@aaron-prindle aaron-prindle self-assigned this Dec 8, 2022
@aaron-prindle aaron-prindle added the bugbash/q4-2022 Issues related to Skaffold Bugbash Q4 2022 label Dec 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tag bugbash/q4-2022 Issues related to Skaffold Bugbash Q4 2022 help wanted We would love to have this done, but don't have the bandwidth, need help from contributors kind/feature-request priority/p3 agreed that this would be good to have, but no one is available at the moment.
Projects
None yet
Development

No branches or pull requests