Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: kubectl for amd64 was not found #88

Open
1 task done
lucaslosi opened this issue Oct 18, 2023 · 22 comments
Open
1 task done

Bug: kubectl for amd64 was not found #88

lucaslosi opened this issue Oct 18, 2023 · 22 comments
Labels
bug Something isn't working idle Inactive for 14 days

Comments

@lucaslosi
Copy link

lucaslosi commented Oct 18, 2023

What happened?

I'm running the following step on my github workflow and got an error telling that the version '1.28.2' for 'amd64' was not found.

            - name: install kubectl
              uses: azure/setup-kubectl@v3

i've tried to downgrade the version adding the following, but got the same error:

              with:
                  version: '1.28.2'`

The error is strange because reaching the download link locally on my machine works correctly.

Version

  • I am using the latest version

Runner

ubuntu-latest

Relevant log output

##[debug]Evaluating condition for step: 'install kubectl'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: install kubectl
##[debug]Loading inputs
##[debug]Loading env
Run azure/setup-kubectl@v3
##[debug]Downloading https://storage.googleapis.com/kubernetes-release/release/stable.txt
##[debug]Destination /home/runner/work/_temp/8ed25fc3-ddb3-4b8b-847a-244571e4a0e9
##[debug]download complete
##[debug]isExplicit: 1.28.3
##[debug]explicit? true
##[debug]checking cache: /opt/hostedtoolcache/kubectl/1.28.3/x64
##[debug]not found
##[debug]Downloading https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl
##[debug]Destination /home/runner/work/_temp/50d81c8f-95cc-4889-8d7a-ab8d1a2da771
##[debug]Failed to download from "https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl". Code(404) Message(Not Found)
Error: Error: Kubectl 'v1.28.3' for 'amd64' arch not found.
##[debug]Node Action run completed with exit code 1
##[debug]Finishing: install kubectl

image

@lucaslosi lucaslosi added the bug Something isn't working label Oct 18, 2023
@lucaslosi lucaslosi changed the title Bug: Bug: kubectl for amd64 was not found Oct 18, 2023
@thecodeassassin
Copy link

Please check all (including closed) issues before you post.

#87

@nathanjulsrud
Copy link

nathanjulsrud commented Oct 18, 2023

Please check all (including closed) issues before you post.

kubernetes/kubectl#87

This isn't helping. We've tried 1.28.3 which is https://storage.googleapis.com/kubernetes-release/release/stable.txt

Today We've tried running 1.28.2 and 1.27.6 of kubectl with the same errors.

I am not in the same organization as him, additionally the issue is sporadic which suggests configuration drift on the runners.

use with:
version: "v1.28.2"

works consistently, like they said use a "v" doh!

@ash-shepard
Copy link

FWIW: I was able to get it to work by adding a v to my version, like so 'vX.XX.XX' in my workflow:

It failed here:

      - uses: azure/setup-kubectl@v3
        id: install
        with: 
          version: '1.28.2'

And was successful here:

      - uses: azure/setup-kubectl@v3
        id: install
        with: 
          version: 'v1.28.2'

@thecodeassassin
Copy link

Please check all (including closed) issues before you post.
kubernetes/kubectl#87

This isn't helping. We've tried 1.28.3 which is https://storage.googleapis.com/kubernetes-release/release/stable.txt

Today We've tried running 1.28.2 and 1.27.6 of kubectl with the same errors.

I am not in the same organization as him, additionally the issue is sporadic which suggests configuration drift on the runners.

use with: version: "v1.28.2"

works consistently, like they said use a "v" doh!

How is my reply not helping.

  1. you are opening the same issue that was closed today
  2. My example literally mentions a "v" in the version number

@lucaslosi
Copy link
Author

lucaslosi commented Oct 18, 2023

It's very inconsistent today. had 5 workflows working and like 20 with errors.

I ran a curl command on the same url inside the workflow and it downloaded fine, so I think the issue can be related to the github toolcache:

kubectlDownloadPath = await toolCache.downloadTool(

setting the version with the 'v' worked for me, but i think the action should work with the default configs

@MikeSpreitzer
Copy link

I have a workflow that does not specify a version. It has been working fine for months. Now it fails with the error message shown in the original post here.

@clubanderson
Copy link

clubanderson commented Oct 19, 2023

got the same issue - let us know when there is a fix. not specifying a version. default was working fine up until a few hours ago. would be great if you continue to support default (no version specified). I ran with "v1.28.3" and got it to work, but no version specified was working until earlier today.

@thecodeassassin
Copy link

thecodeassassin commented Oct 19, 2023

Just as an FYI, I opened an issue on the kubectl issue tracker (kubernetes/kubernetes#121400). Maybe some release update failed and now the stable.txt contains a version that doesn't correspond to a binary.

That's why default doesn't work, although it should probably use the fallback version specified in the script, and it just fails rather than using the fallback which would alleviate some pain points since all of my projects don't specify a pinned version.

I could make the PR that fixes the fallback version, but not sure if the authors see that as an issue.

@OrBin
Copy link

OrBin commented Oct 19, 2023

We have pinned the version to v1.28.3, but it still doesn't work for us:

Full step screenshot

image

Looking at the final lines:

##[debug]Downloading https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl
##[debug]Destination /home/runner/work/_temp/81868074-d646-4f0e-932e-7814c84d2d82
##[debug]Failed to download from "https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl". Code(404) Message(Not Found)
Error: Error: Kubectl 'v1.28.3' for 'amd64' arch not found.

I click on the URL and succeed in downloading the binary. Using curl -Li, I see that I get HTTP/2 302 and then HTTP/2 200. No 404 here.

At this point, it really feels like it's a GitHub Actions cache thing, even though I can't find where such thing is configured (I'm not very familiar with the actions/toolkit repo).

Anyone still having this issue after pinning the version and has additional thoughts or suggestions?

@martimors
Copy link

martimors commented Oct 19, 2023

Facing this issue.

Action called without arguments:

jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - uses: azure/setup-kubectl@v3

Log shows:

Run azure/setup-kubectl@v3
  with:
    version: latest

Error: Error: Kubectl 'v1.28.3' for 'amd64' arch not found.

Edit; resolved for us after specifying a pin that isn't v1.28.3, as suggested/alluded to earlier in this thread.

@OrBin
Copy link

OrBin commented Oct 19, 2023

Facing this issue.

Action called without arguments:

jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - uses: azure/setup-kubectl@v3

Log shows:

Run azure/setup-kubectl@v3
  with:
    version: latest

Error: Error: Kubectl 'v1.28.3' for 'amd64' arch not found.

It makes sense, latest is the default, and v1.28.3 is indeed the latest version. Problem is with downloading the binary.

@OrBin
Copy link

OrBin commented Oct 19, 2023

Anyone still having this issue after pinning the version and has additional thoughts or suggestions?

We pinned to v1.28.2 instead, and everything seems to work. This evidence supports my assumption that it's an issue with cache while downloading the binary.

@thecodeassassin
Copy link

Anyone still having this issue after pinning the version and has additional thoughts or suggestions?

We pinned to v1.28.2 instead, and everything seems to work. This evidence supports my assumption that it's an issue with cache while downloading the binary.

Yeah, so what most likely happened here is that they released v1.28.3, updated stable.txt but the binary failed to build. So when they rectified the situation they uploaded the binary to GCS but some third parties (actions for example) still cache the 404.

@clubanderson
Copy link

last night the pinned approach stopped working in our CI. Not sure why yet.

@alikhantara
Copy link

i faced with this issue today.
i rerun job while read this topic.
first and second is stopped with subj error, but third is successfully finished.

i don't have any idea =\

@fdstevex
Copy link

This is still occasionally failing - some builds succeed, some fail. Bad cache somewhere?

@clubanderson
Copy link

I have a slew of failed jobs today.

@kenjiejima
Copy link

This part constructs the URL for kubectl: https://github.com/Azure/setup-kubectl/blob/main/src/helpers.ts#L15

As of 2023-10-19, the default version translates to https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl. On one computer, I get a 302 followed by a 200. On another computer, I get a 404. Therefore, I think it's likely to be a CDN cache issue on dl.k8s.io, and not a problem of this setup-kubectl module.

@lucaslosi
Copy link
Author

This part constructs the URL for kubectl: https://github.com/Azure/setup-kubectl/blob/main/src/helpers.ts#L15

As of 2023-10-19, the default version translates to https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl. On one computer, I get a 302 followed by a 200. On another computer, I get a 404. Therefore, I think it's likely to be a CDN cache issue on dl.k8s.io, and not a problem of this setup-kubectl module.

You can reach the same generated URL running curl inside the same job that fails the setup-kubectl, so it's probably related to a manual file upload as @thecodeassassin said or the actions/toolkit cache

@clubanderson
Copy link

Makes sense. It should start to clear up soon

Copy link

github-actions bot commented Nov 3, 2023

This issue is idle because it has been open for 14 days with no activity.

@github-actions github-actions bot added the idle Inactive for 14 days label Nov 3, 2023
realMartinez pushed a commit to realMartinez/camel-k that referenced this issue Nov 13, 2023
@kpetrovic
Copy link

Same problem occurs when you use the newest v4.0.0.
Downgrade to version 3 worked for me:

- name: install kubectl
  uses: azure/setup-kubectl@v3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working idle Inactive for 14 days
Projects
None yet
Development

No branches or pull requests