`argo auth token` fails with `http: nil Request.URL` error #7175

dchancogne · 2021-11-05T17:10:02Z

Summary

What happened/what you expected to happen?
Using version 3.1.5 I was able to run argo auth token and get an authentication token.
After upgrading to 3.2.3 I get an error when running that command

> argo auth token
FATA[2021-11-05T13:06:07.303Z] http: nil Request.URL

What version is it broken in?
3.2.3

What version was it working in?
3.1.5

Diagnostics

My kube config is otherwise valid and I can do all other kubectl operation on the cluster

With debug, I get:

> argo auth token -v
DEBU[2021-11-05T13:02:24.571Z] CLI version                                   version="{v3.2.3 2021-10-27T02:10:33Z e5dc961b7846efe0fe36ab3a0964180eaedd2672 v3.2.3 clean go1.16.9 gc darwin/amd64}"
I1105 13:02:24.573193   29379 loader.go:372] Config loaded from file:  /Users/david/.kube/config
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x12103aa]

goroutine 1 [running]:
net/url.(*URL).String(0x0, 0xc0000a8528, 0x0)
	/opt/hostedtoolcache/go/1.16.9/x64/src/net/url/url.go:813 +0x4a
k8s.io/client-go/transport.newRequestInfo(...)
	/home/runner/go/pkg/mod/k8s.io/client-go@v0.21.5/transport/round_trippers.go:323
k8s.io/client-go/transport.(*debuggingRoundTripper).RoundTrip(0xc000447e18, 0xc00071a600, 0xa, 0x3623033, 0x5)
	/home/runner/go/pkg/mod/k8s.io/client-go@v0.21.5/transport/round_trippers.go:429 +0x47
k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip(0xc0008e2620, 0xc00071a600, 0x362c148, 0xa, 0xc000493528)
	/home/runner/go/pkg/mod/k8s.io/client-go@v0.21.5/transport/round_trippers.go:156 +0x286
github.com/argoproj/argo-workflows/v3/util/kubeconfig.userAgentRoundTripper.RoundTrip(0x3623033, 0x5, 0x44054a0, 0xc0008e2620, 0xc00071a600, 0x0, 0x36289ee, 0x100ddbb)
	/home/runner/work/argo-workflows/argo-workflows/util/kubeconfig/roundtripper.go:12 +0x109
github.com/argoproj/argo-workflows/v3/util/kubeconfig.GetBearerToken(0xc000236900, 0x0, 0x0, 0x1054b67, 0x0, 0xc000a3fbc8, 0x3)
	/home/runner/work/argo-workflows/argo-workflows/util/kubeconfig/kubeconfig.go:175 +0x42f
github.com/argoproj/argo-workflows/v3/util/kubeconfig.GetAuthString(0xc000236900, 0x0, 0x0, 0x43d0310, 0x1, 0xc000114340, 0x1e)
	/home/runner/work/argo-workflows/argo-workflows/util/kubeconfig/kubeconfig.go:124 +0xde
github.com/argoproj/argo-workflows/v3/cmd/argo/commands/client.GetAuthString(0x43cfc24, 0x6)
	/home/runner/work/argo-workflows/argo-workflows/cmd/argo/commands/client/conn.go:102 +0x253
github.com/argoproj/argo-workflows/v3/cmd/argo/commands/auth.NewTokenCommand.func1(0xc0003edb80, 0xc0009c6ff0, 0x0, 0x1)
	/home/runner/work/argo-workflows/argo-workflows/cmd/argo/commands/auth/token.go:21 +0x30
github.com/spf13/cobra.(*Command).execute(0xc0003edb80, 0xc0009c6fe0, 0x1, 0x1, 0xc0003edb80, 0xc0009c6fe0)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:860 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc0009f0280, 0xc000084778, 0xc000a3ff78, 0x1006045)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:974 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.2.1/command.go:902
main.main()
	/home/runner/work/argo-workflows/argo-workflows/cmd/argo/main.go:14 +0x2b

This issue might be related but different? #6344

What Kubernetes provider are you using?
EKS

What executor are you running? Docker/K8SAPI/Kubelet/PNS/Emissary

N/A

Message from the maintainers:

Impacted by this regression? Give it a 👍. We prioritise the issues with the most 👍.

The text was updated successfully, but these errors were encountered:

alexec · 2021-11-05T21:08:23Z

Can you please share redacted KUBECONFIG?

dchancogne · 2021-11-08T02:22:55Z

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: [REDACTED]
    server: https://[REDACTED].eks.amazonaws.com
  name: dev-infra-eks
- cluster:
    certificate-authority-data: [REDACTED]
    server: https://[REDACTED].eks.amazonaws.com
  name: prd-infra-eks
contexts:
- context:
    cluster: dev-infra-eks
    namespace: argo
    user: dchancogne-dev
  name: dev-infra-eks
- context:
    cluster: prd-infra-eks
    namespace: argo
    user: dchancogne-prd
  name: prd-infra-eks
current-context: dev-infra-eks
kind: Config
preferences: {}
users:
- name: dchancogne-dev
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - --region
      - [REDACTED]
      - eks
      - get-token
      - --cluster-name
      - dev-[REDACTED]-infra-ek
      command: aws
      env: null
      provideClusterInfo: false
- name: dchancogne-prd
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - --region
      - [REDACTED]
      - eks
      - get-token
      - --cluster-name
      - prd-[REDACTED]-infra-eks
      command: aws
      env: null
      provideClusterInfo: false

alexec · 2021-11-09T01:18:29Z

The panic is a red herring. This is caused be enabled debugging!

alexec · 2021-11-09T01:20:53Z

I don't think this can be caused by the linked issue, as that was released in v3.1.2. I think this is something else, but similar.

Signed-off-by: Alex Collins <alex_collins@intuit.com>

alexec · 2021-11-09T15:42:12Z

@dchancogne I've not been able to reproduce this issue, but I've create a fix for what I believe is the problem.

If I give you a new argo binary can you test it please? I'll need to know what system arch it needs to build for.

dchancogne · 2021-11-09T18:03:58Z

@alexec Sure, happy to give it a try.
Macbook Pro Intel, MacOS 11.5.2

alexec · 2021-11-09T19:03:59Z

Here you go:

https://github.com/argoproj/argo-workflows/releases/tag/v0.0.0-dev-kc-0

dchancogne · 2021-11-09T19:16:03Z

Better, no error, but still no-go:

> ./argo-darwin-amd64 auth token
Bearer

I.e. No token (empty string) returned.

alexec · 2021-11-10T21:47:03Z

Please can you test again with more diagnostics:

https://github.com/argoproj/argo-workflows/releases/tag/v0.0.0-dev-kc-1

dchancogne · 2021-11-10T22:24:04Z

Here we go. Let me know if it's the output you were expecting or if I need to pass some additional argument:

> ./argo-darwin-amd64 auth token
ALEX err= (0x3f2adc0,0xc0006423b0)
ALEX req= 0xc00065aa00
ALEX resp= 0xc0000d9a70
Bearer

>

dchancogne · 2021-11-10T22:26:40Z

With verbose on:

> ./argo-darwin-amd64 auth token -v
DEBU[2021-11-10T17:24:34.019Z] CLI version                                   version="{v0.0.0-dev-kc-1 2021-11-09T21:20:53Z b815a4ca3dd412b5a6a356c4da989c01f054f6be v0.0.0-dev-kc-1 clean go1.17.2 gc darwin/amd64}"
I1110 17:24:34.021626   31809 loader.go:372] Config loaded from file:  /Users/david/.kube/config
ALEX err= (0x3f2adc0,0xc0000c7900)
I1110 17:24:35.854353   31809 round_trippers.go:454] GET https://[REDACTED].eks.amazonaws.com 200 OK in 1832 milliseconds
ALEX req= 0xc0000ab300
ALEX resp= 0xc000158ab0
Bearer

>

alexec · 2021-11-10T23:02:59Z

Ah. Maybe I should format the output rather than print out the pointer.

alexec · 2021-11-15T20:20:46Z

Please try v0.0.0-dev-kc-2

dchancogne · 2021-11-15T20:25:20Z

@alexec Do you have a darwin-amd64 compiled version?

alexec · 2021-11-16T17:07:26Z

Should be attached to release.

dchancogne · 2021-11-16T18:25:08Z

Thanks @alexec. I tried v2 and v3:

> ./argo-darwin-amd64 version
argo: v0.0.0-dev-kc-2
  BuildDate: 2021-11-15T21:37:20Z
  GitCommit: 996ca136834d1a7feb5f0d663ddcddb0886dc811
  GitTreeState: clean
  GitTag: v0.0.0-dev-kc-2
  GoVersion: go1.17.3
  Compiler: gc
  Platform: darwin/amd64

> ./argo-darwin-amd64 auth token
ALEX err= (0x3f2aec0,0xc00068cb60)
ALEX resp.statusCode= 200
ALEX resp.headers= {"Audit-Id":["ff8bd978-8922-4f71-b93d-e54d2b7204e9"],"Cache-Control":["no-cache, private"],"Content-Type":["application/json"],"Date":["Tue, 16 Nov 2021 18:21:50 GMT"]}
Bearer

>

>  ./argo-darwin-amd64 version
argo: v0.0.0-dev-kc-3
  BuildDate: 2021-11-15T21:49:08Z
  GitCommit: 996ca136834d1a7feb5f0d663ddcddb0886dc811
  GitTreeState: clean
  GitTag: v0.0.0-dev-kc-3
  GoVersion: go1.17.3
  Compiler: gc
  Platform: darwin/amd64

> ./argo-darwin-amd64 auth token
ALEX err= (0x3f2aec0,0xc000172cd0)
ALEX resp.statusCode= 200
ALEX resp.headers= {"Audit-Id":["de8a2acd-a425-4828-8083-7975000998c0"],"Cache-Control":["no-cache, private"],"Content-Type":["application/json"],"Date":["Tue, 16 Nov 2021 18:23:59 GMT"]}
Bearer

>

alexec · 2021-11-16T18:38:32Z

Please try with patch 5, just pushed so will take 20m to build.

dchancogne · 2021-11-17T03:13:54Z

@alexec Here we go:

> ./argo-darwin-amd64 version
argo: v0.0.0-dev-kc-5
  BuildDate: 2021-11-16T19:45:38Z
  GitCommit: 7f47e6d2155da988341c7f3b5a707be8e4ac2878
  GitTreeState: clean
  GitTag: v0.0.0-dev-kc-5
  GoVersion: go1.17.3
  Compiler: gc
  Platform: darwin/amd64

 > ./argo-darwin-amd64 auth token
ALEX err= (0x3f2af60,0xc000090c00)
ALEX req.headers= {}
ALEX resp.statusCode= 200
ALEX resp.headers= {"Audit-Id":["6c74c970-001e-462c-be8e-455d5340ea29"],"Cache-Control":["no-cache, private"],"Content-Type":["application/json"],"Date":["Wed, 17 Nov 2021 03:12:37 GMT"]}
Bearer

>

alexec · 2021-11-17T23:03:40Z

Please try with patch 6?

dchancogne · 2021-11-18T19:57:07Z

@alexec Getting closer

> ./argo-darwin-amd64 version
argo: v0.0.0-dev-kc-6
  BuildDate: 2021-11-18T00:06:15Z
  GitCommit: c85dc9a99e509bd9138b37d3b2b862f4d65d0cd3
  GitTreeState: clean
  GitTag: v0.0.0-dev-kc-6
  GoVersion: go1.17.3
  Compiler: gc
  Platform: darwin/amd64

> ./argo-darwin-amd64 auth token
ALEX err= can't add TLS certificate callback: transport.Config.TLS.GetCert already set
ALEX req.headers= {}
ALEX resp.statusCode= 200
ALEX resp.headers= {"Audit-Id":["c49a10b2-5c42-4f20-8b45-80f897c5fd64"],"Cache-Control":["no-cache, private"],"Content-Type":["application/json"],"Date":["Thu, 18 Nov 2021 19:55:48 GMT"]}
Bearer

>

kevin-hanselman · 2021-11-18T21:05:28Z

FWIW I'm also seeing this in both 3.2.3 and 3.2.4, and I'm running EKS as well. I also confirmed auth token works on 3.1.5.

alexec · 2021-11-18T23:23:03Z

I'm pretty sure this is caused by Kubernetes library upgrade.

alexec · 2021-11-18T23:58:49Z

Please try patch 7.

dchancogne · 2021-11-19T02:06:37Z

@alexec Success!!
Thanks for figuring this out.

./argo-darwin-amd64 version
argo: v0.0.0-dev-kc-7
  BuildDate: 2021-11-19T01:29:47Z
  GitCommit: 47a5208e9c940abb014c08847179865a5b227f49
  GitTreeState: clean
  GitTag: v0.0.0-dev-kc-7
  GoVersion: go1.17.3
  Compiler: gc
  Platform: darwin/amd64

> ./argo-darwin-amd64 auth token
Bearer k8s-aws-v1.aH0cHM6Ly9zdH[REDACTED]

>

Signed-off-by: Alex Collins <alex_collins@intuit.com>

Enables the Argo-workflow server UI in the cluster. This will allow users to view and potentially troubleshoot workflows. Note that this work is about making the UI work but not make the Single Sign-On (SSO) login feature works, which is detailed in another [card](https://trello.com/c/hF9uauRU/587-google-sso-for-argo) The URL for the service is: https://argo-workflow.eks.<environment>.govuk.digital/ To get the authentication token: 1. install the `argo` cli using brew: `brew install argo` Must be version 3.2.5 or greater due to [bug](argoproj/argo-workflows#7175) 2. get token: `gds aws govuk-<environment>-admin -- argo auth token` 3. copy whole token, starting with `Bearer ` and paste it in website. Ref: 1. [trello card](https://trello.com/c/GHtiRMqe/781-argo-workflows-ui)

dchancogne added type/bug type/regression Regression from previous behavior (a specific type of bug) triage labels Nov 5, 2021

alexec changed the title ~~arto auth token fails~~ argo auth token fails with http: nil Request.URL error Nov 5, 2021

alexec added the more-information-needed label Nov 5, 2021

no-response bot removed the more-information-needed label Nov 8, 2021

alexec added a commit that referenced this issue Nov 9, 2021

fix: Prevent nil panic. Fixes #7175

494f811

Signed-off-by: Alex Collins <alex_collins@intuit.com>

alexec mentioned this issue Nov 9, 2021

fix: Fix argo auth token. Fixes #7175 #7186

Merged

1 task

alexec self-assigned this Nov 9, 2021

alexec removed the triage label Nov 9, 2021

totycro mentioned this issue Nov 15, 2021

argo auth token --verbose segfaults #7216

Closed

alexec closed this as completed in #7186 Nov 22, 2021

alexec added a commit that referenced this issue Nov 22, 2021

fix: Fix argo auth token. Fixes #7175 (#7186)

e045577

Signed-off-by: Alex Collins <alex_collins@intuit.com>

alexec added a commit to alexec/argo-workflows that referenced this issue Nov 24, 2021

fix: Fix argo auth token. Fixes argoproj#7175 (argoproj#7186)

f05523e

Signed-off-by: Alex Collins <alex_collins@intuit.com>

sarabala1979 mentioned this issue Dec 15, 2021

v3.2.5 Release #7414

Closed

73 tasks

sarabala1979 pushed a commit that referenced this issue Dec 15, 2021

fix: Fix argo auth token. Fixes #7175 (#7186)

2f96c46

Signed-off-by: Alex Collins <alex_collins@intuit.com>

fredericfran-gds mentioned this issue Jan 25, 2022

Enables Argo-workflow Server UI alphagov/govuk-infrastructure#521

Merged

sarabala1979 mentioned this issue Mar 1, 2022

v3.2.9 Cherry-pick #8043

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`argo auth token` fails with `http: nil Request.URL` error #7175

`argo auth token` fails with `http: nil Request.URL` error #7175

dchancogne commented Nov 5, 2021 •

edited

Loading

alexec commented Nov 5, 2021

dchancogne commented Nov 8, 2021 •

edited

Loading

alexec commented Nov 9, 2021

alexec commented Nov 9, 2021

alexec commented Nov 9, 2021

dchancogne commented Nov 9, 2021

alexec commented Nov 9, 2021

dchancogne commented Nov 9, 2021 •

edited

Loading

alexec commented Nov 10, 2021

dchancogne commented Nov 10, 2021

dchancogne commented Nov 10, 2021

alexec commented Nov 10, 2021

alexec commented Nov 15, 2021

dchancogne commented Nov 15, 2021

alexec commented Nov 16, 2021

dchancogne commented Nov 16, 2021

alexec commented Nov 16, 2021

dchancogne commented Nov 17, 2021

alexec commented Nov 17, 2021

dchancogne commented Nov 18, 2021

kevin-hanselman commented Nov 18, 2021 •

edited

Loading

alexec commented Nov 18, 2021

alexec commented Nov 18, 2021

dchancogne commented Nov 19, 2021

argo auth token fails with http: nil Request.URL error #7175

argo auth token fails with http: nil Request.URL error #7175

Comments

dchancogne commented Nov 5, 2021 • edited Loading

Summary

Diagnostics

alexec commented Nov 5, 2021

dchancogne commented Nov 8, 2021 • edited Loading

alexec commented Nov 9, 2021

alexec commented Nov 9, 2021

alexec commented Nov 9, 2021

dchancogne commented Nov 9, 2021

alexec commented Nov 9, 2021

dchancogne commented Nov 9, 2021 • edited Loading

alexec commented Nov 10, 2021

dchancogne commented Nov 10, 2021

dchancogne commented Nov 10, 2021

alexec commented Nov 10, 2021

alexec commented Nov 15, 2021

dchancogne commented Nov 15, 2021

alexec commented Nov 16, 2021

dchancogne commented Nov 16, 2021

alexec commented Nov 16, 2021

dchancogne commented Nov 17, 2021

alexec commented Nov 17, 2021

dchancogne commented Nov 18, 2021

kevin-hanselman commented Nov 18, 2021 • edited Loading

alexec commented Nov 18, 2021

alexec commented Nov 18, 2021

dchancogne commented Nov 19, 2021

`argo auth token` fails with `http: nil Request.URL` error #7175

`argo auth token` fails with `http: nil Request.URL` error #7175

dchancogne commented Nov 5, 2021 •

edited

Loading

dchancogne commented Nov 8, 2021 •

edited

Loading

dchancogne commented Nov 9, 2021 •

edited

Loading

kevin-hanselman commented Nov 18, 2021 •

edited

Loading