Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow ignoring errors when exporting remote cache #2578

Closed
tonistiigi opened this issue Jan 25, 2022 · 3 comments · Fixed by #3430
Closed

Allow ignoring errors when exporting remote cache #2578

tonistiigi opened this issue Jan 25, 2022 · 3 comments · Fixed by #3430

Comments

@tonistiigi
Copy link
Member

previous issue docker/buildx#786
related docker/build-push-action#515

Cache export should take an option for controlling what to do when cache exporting fails. There are cases where there are temporary issues with the cache backend and the user would like it to not fail the build.

@ellisio
Copy link

ellisio commented Mar 1, 2022

For reference, I am posting what I posted on the previous issue here.

I know this issue is closed, but I would like to bring up that GitHub Support is claiming these are timeouts on their API. Their recommendation is to patch this code to retry on 503s, with the backoff strategy.

While that kind of patch would probably resolve this for more cases than not, it still leaves a possibility that the builds will fail. I am trying to push back and have them provide more information on why these requests are timing out as, to me, that is a GitHub Infrastructure issue, not OSS issue.

@ellisio
Copy link

ellisio commented Mar 10, 2022

GitHub support has requested that anyone experiencing this issue, please open a support ticket and reference ticket #1469808. They are wanting to build a case around this for prioritization for some infrastructure changes involving the caching layer.

@tjhiggins
Copy link

tjhiggins commented Jul 14, 2022

Hitting an issue about half of the time when using the local cache. Would be super useful to gracefully ignore a failure when writing to the cache.

#35 [app] exporting cache
#35 preparing build cache for export
#35 preparing build cache for export 0.4s done
#35 writing layer sha256:000f02d5bab2e1769bf724dc6a99cf1882267577981c5eac519ac786ad88da57 done
#35 writing layer sha256:0ab5ec7719940ff567e31ea7bcb202c5a70cf97551c77fd8a9ffcffe8d8c6a57 0.1s done
#35 writing layer sha256:1acf565088aae3ef2159885f29853bce88eb16082b0c98fcacd08fc9008c84b9
#35 writing layer sha256:1acf565088aae3ef2159885f29853bce88eb16082b0c98fcacd08fc9008c84b9 0.1s done
#35 writing layer sha256:1c6f497c8d6139eaf9da229089ce3330071db73cef17ebedcd933ae5b02d0e49
#35 writing layer sha256:1c6f497c8d6139eaf9da229089ce3330071db73cef17ebedcd933ae5b02d0e49 2.9s done
#35 writing layer sha256:37219ced0f729d39f75fe142ea182463cd5a1ec7cd8666319264523093ad7432 done
#35 writing layer sha256:4c25b3090c2685271afcffc2a4db73f15ab11a0124bfcde6085c934a4e6f4a51
#35 writing layer sha256:4c25b3090c2685271afcffc2a4db73f15ab11a0124bfcde6085c934a4e6f4a51 1.0s done
#35 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 done
#35 writing layer sha256:5084fa7ebd744165b15df008a9c14db7fc3d6af34cce64ba85bbaa348af594a3 done
#35 ERROR: error writing layer blob: Canceled: grpc: the client connection is closing: unknown
------
 > [app] exporting cache:
------
error: failed to solve: error writing layer blob: Canceled: grpc: the client connection is closing: unknown
Docker buildx bake failed. Please make sure docker is running.
Command failed with exit code 1: docker bake -f /home/runner/docker-compose.yml --push
    Error: Command failed with exit code 1: docker 
     bake -f
     /home/runner/docker-compose.yml --push
Error: Process completed with exit code 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants