"File exists" errors when un-taring golangci-lint

[andig]

Follow-up to #135, #156 and #221.

Using v2, I'm still seeing these errors:

- name: Lint
  uses: golangci/golangci-lint-action@v2
  with:
    version: v1.37

Example is in https://github.com/andig/evcc/pull/1075/checks?check_run_id=2655591730

Apart from fixing using tar- why not just use go install instead?

[andig]

Looking at https://pipelines.actions.githubusercontent.com/D1tYAQGUpNaqlpNKaplcT4IVn3rJSlK5kNFys0tSrNSiqbBdeP/_apis/pipelines/1/runs/3119/signedlogcontent/4?urlExpires=2021-05-24T11%3A43%3A31.7174061Z&urlSigningMethod=HMACV1&urlSignature=fl95dXdfFb1YndQYp1OdBV8sg4IEP%2Fdu9Jw4VQz1%2B8c%3D I'm a bit confused what is really going on here:

2021-05-24T11:30:59.9592849Z Requested golangci-lint 'v1.37', using 'v1.37.1', calculation took 101ms
2021-05-24T11:30:59.9593619Z Installing golangci-lint v1.37.1...
2021-05-24T11:30:59.9594890Z Downloading https://github.com/golangci/golangci-lint/releases/download/v1.37.1/golangci-lint-1.37.1-linux-amd64.tar.gz ...
2021-05-24T11:30:59.9604130Z [command]/usr/bin/tar xz --overwrite --warning=no-unknown-keyword -C /home/runner -f /home/runner/work/_temp/699e2cb2-9f46-4ed3-ba70-55cf1b855115
2021-05-24T11:30:59.9607159Z Installed golangci-lint into /home/runner/golangci-lint-1.37.1-linux-amd64/golangci-lint in 879ms
2021-05-24T11:30:59.9607999Z Received 50331648 of 370718094 (13.6%), 47.8 MBs/sec
2021-05-24T11:31:00.0583863Z Received 180355072 of 370718094 (48.7%), 85.8 MBs/sec
2021-05-24T11:31:01.0577699Z Received 301989888 of 370718094 (81.5%), 95.9 MBs/sec
2021-05-24T11:31:01.8965434Z Received 370718094 of 370718094 (100.0%), 92.0 MBs/sec
2021-05-24T11:31:01.8967722Z Cache Size: ~354 MB (370718094 B)
2021-05-24T11:31:01.9067971Z [command]/usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/ce838b4c-e0e7-4371-b6a5-e0bbc5c6235e/cache.tzst -P -C /home/runner/work/evcc/evcc
2021-05-24T11:31:02.8551876Z /usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/.gitignore: Cannot open: File exists
2021-05-24T11:31:02.8562145Z ##[error]/usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/conn.go: Cannot open: File exists
2021-05-24T11:31:02.8575744Z ##[error]/usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/examples/echo/server.go: Cannot open: File exists
2021-05-24T11:31:02.8578177Z ##[error]/usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/examples/echo/main.go: Cannot open: File exists
2021-05-24T11:31:02.8580108Z /usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/examples/echo/README.md: Cannot open: File exists
2021-05-24T11:31:02.8581576Z ##[error]/usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/examples/echo/server_test.go: Cannot open: File exists
2021-05-24T11:31:02.8583504Z /usr/bin/tar: ../../../go/pkg/mod/nhooyr.io/websocket@v1.8.7/examples/README.md: Cannot open: File exists

It seems that the install actually went fine but there's a follow-on step that breaks?

[seh]

I see similar errors why I try to use this action. Here's a sample of the first two complaints:

  /usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/154ee373-4600-4395-b547-3643dff1d28a/cache.tzst -P -C /home/runner/work/my-repo/my-repo
  Error: /usr/bin/tar: ../../../go/pkg/mod/google.golang.org/grpc@v1.37.0/server.go: Cannot open: File exists
  Error: /usr/bin/tar: ../../../go/pkg/mod/google.golang.org/grpc@v1.37.0/attributes/attributes_test.go: Cannot open: File exists

[seh]

Related issues: #23, #135

[andig]

maintainers: any chance to get this fixed? Has a number of past and current related issues.

updated sorry for the bad @

[seh]

I see previous discussion about not running certain Go toolchain commands ahead of golangci-lint in the same GitHub Actions job. In my case, the only thing I run before this action is go list -find -json ./... in order to ensure that there are actually some Go files available on the current branch. If I try to run golangci-lint when there are no Go files available, it fails, instead of succeeding trivially. When go list either produces no output or its "GoFiles" object field has no entries, I skip running golangci-lint.

[seh]

Oh, I just noticed that this action runs actions/setup-go itself. I was taking care of that myself before running this action, because I need to run go list first as explained earlier.

Is it possible to influence which version of Go this action installs? I don't see mention of such a parameter in the action.yaml file.

[ifree92]

As a "very hot" solution I have just disabled package caching:

skip-pkg-cache: true

[directionless]

This appears to still be present in v1.42

My build depends on generated files. As several linters seem to examine the full call tree, they will always fail without the generated content. Creating something of a chicken and egg problem.

[andig]

Looks like this issue has not really caught attention with the maintainers ;)

[SVilgelm]

there is a special skip-go-installation parameter to use already installed go

      - uses: actions/checkout@v2
      - uses: actions/setup-go@v2
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v2
        with:
          version: latest
          skip-go-installation: true

[SVilgelm]

This appears to still be present in v1.42

My build depends on generated files. As several linters seem to examine the full call tree, they will always fail without the generated content. Creating something of a chicken and egg problem.

you are right, there are several linters that require whole project, like unused. In this case you need to generate all files first, then run golangci-lint. And I don't see the chicken and egg problem here. Your goal is in checking that code is compilable and is valid. So if you run go generate first and if fails, then it is an expected and valid result of a workflow.
if the files were successfully generated, then linter can do its work

[andig]

there is a special skip-go-installation parameter to use already installed go

      - uses: actions/checkout@v2
      - uses: actions/setup-go@v2
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v2
        with:
          version: latest
          skip-go-installation: true

In evcc-io/evcc@1557bb0 I've added skip-go-installation: true, but build log is still full or errors: https://github.com/evcc-io/evcc/runs/4081084055?check_suite_focus=true, see Lint step.

In fact, the errors are not caused by the Go installation:

  /usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/8b8190cf-ca27-4b77-9d43-823bd24af1dd/cache.tzst -P -C /home/runner/work/evcc/evcc
  Error: /usr/bin/tar: ../../../go/pkg/mod/gosrc.io/xmpp@v0.5.1/xmpp_transport.go: Cannot open: File exists
  Error: /usr/bin/tar: ../../../go/pkg/mod/gosrc.io/xmpp@v0.5.1/router.go: Cannot open: File exists

[andig]

@sg0hsmt @abennett any chance to look at this issue?

[SVilgelm]

I checked the logs, so the linter works fine, no issues at all. The errors are related to usage of cache and I found that you are caching the go folder in another step, so I would recommend you to remove that step, it is not needed: https://github.com/evcc-io/evcc/blob/master/.github/workflows/build.yml#L18-L26

so, your step restores the cache, then you call lint action that tries to restore the cache as well and of course it fails due to the files already exist.

[andig]

Thank you for the comment. I still feel this is bad behaviour.

Use case: My code contained generated files and I'd also like to lint those (might have gotten the templates wrong). I can only generate them when the cache is already populated.
I might also simply chose to run the linter as a later step in the build when the cache would already be populated.

Why does the linter think that it needs to restore the cache? Imho that's not its task and if it's optional behaviour then it would help have control over it?

[SVilgelm]

have you tried to disable cache?:

          skip-pkg-cache: true
          skip-build-cache: true

the action is designed to be run alone first, since you have more advanced usage you need to tweak the action's and linter's configs accordingly

[andig]

That leads to:

Run golangci/golangci-lint-action@v2
prepare environment
run golangci-lint
  Running [/home/runner/golangci-lint-1.42.1-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
  level=error msg="Timeout exceeded: try increasing it by passing --timeout option"
  Error: golangci-lint exit with code 4
  Ran golangci-lint in 76614ms

with the manual caching step before looking like this:

- name: Cache Go modules
  uses: actions/cache@v2
  with:
    path: |
      ~/go/pkg/mod
      ~/.cache/go-build
    key: ${{ runner.os }}-${{ matrix.go-version }}-go-${{ hashFiles('**/go.sum') }}
    restore-keys: |
      ${{ runner.os }}-${{ matrix.go-version }}-go-

This cache will of course be cold when the go.sum changes.

I've finally moved the linter to a later position after the first build step and then it succeeds.

[StevenACoffman]

Why can we not add --overwrite to tar?

[SVilgelm]

Because it is not our code, we use GitHub cache package

[StevenACoffman]

Looks like someone tried to fix this upstream for this very package here: actions/toolkit#717 but that PR was superseded by actions/toolkit#807 which was merged on May 21, 2021.

However, the last release of @actions/cache was 1.0.7 which was Tuesday, April 13, 2021, prior to the merge of the fix. 😞

As a result, this will not be fixed until a new upstream release is cut and this updates to use it, but no other code needs to be changed.

[andig]

@maintainers could we please reopen? IMHO we have the bad choices of slow CI due to excessive logging or slow CI due do omitting the caches, both are not good options.

[ldez]

duplicate of #135