-
Notifications
You must be signed in to change notification settings - Fork 546
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix windows race condition when writing image with duplicate layers #1921
Fix windows race condition when writing image with duplicate layers #1921
Conversation
Signed-off-by: David Gannon <19214156+dgannon991@users.noreply.github.com>
I'd say a more robust solution is "try to rename, if it fails, but destination exists, just remove the source and pretend you're fine". This would also cover the case when multiple |
I had a very similar issue in git-lfs in the past. Here's the analysis: git-lfs/git-lfs#3880 (comment) and here's the final fix that solved the issue: git-lfs/git-lfs@662a624 |
I'll give that a go, thank you for the feedback. |
@slonopotamus I've written this up and given it a test, and I can confirm it works: err = os.Rename(w.Name(), renamePath)
var badInputErr *os.LinkError
if err != nil && errors.As(err, &badInputErr) && badInputErr.Err == syscall.ERROR_ACCESS_DENIED {
if _, err := os.Stat(renamePath); err == nil {
// windows os.Rename() issue #1920
// the destination already exists, just remove the target and continue
return os.Remove(w.Name())
}
}
return err I do have to say though of the two solutions I think I'm leaning towards the first one, as it makes os.Rename behave more like linux (in that it's an atomic transaction.) Would you mind if I left this up for a second opinion? I'm more than happy to swap over to this approach, but I worry it would mask other issues outside of the scope of this ticket (as you mentioned, running two crane processes at the same time, it got me thinking that realistically that /should/ be an error!) |
Sure, I'm just passing by, and simply gave my thoughts. |
@slonopotamus - thank you for taking the time, the thoughts were definitely appreciated, and it was great to have a confirm that it was an issue in other places! |
Hi Guys, |
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | Type | Update | |---|---|---|---|---|---|---|---| | [deps.dev/api/v3](https://github.com/google/deps.dev) | `v3.0.0-20240611045547-af20eef0f1eb` -> `v3.0.0-20240617015216-b147e04533eb` | [![age](https://developer.mend.io/api/mc/badges/age/go/deps.dev%2fapi%2fv3/v3.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/deps.dev%2fapi%2fv3/v3.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/deps.dev%2fapi%2fv3/v3.0.0-20240611045547-af20eef0f1eb/v3.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/deps.dev%2fapi%2fv3/v3.0.0-20240611045547-af20eef0f1eb/v3.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | patch | | [deps.dev/util/maven](https://github.com/google/deps.dev) | `af20eef` -> `b147e04` | [![age](https://developer.mend.io/api/mc/badges/age/go/deps.dev%2futil%2fmaven/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/deps.dev%2futil%2fmaven/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/deps.dev%2futil%2fmaven/v0.0.0-20240611045547-af20eef0f1eb/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/deps.dev%2futil%2fmaven/v0.0.0-20240611045547-af20eef0f1eb/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | digest | | [deps.dev/util/resolve](https://github.com/google/deps.dev) | `af20eef` -> `b147e04` | [![age](https://developer.mend.io/api/mc/badges/age/go/deps.dev%2futil%2fresolve/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/deps.dev%2futil%2fresolve/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/deps.dev%2futil%2fresolve/v0.0.0-20240611045547-af20eef0f1eb/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/deps.dev%2futil%2fresolve/v0.0.0-20240611045547-af20eef0f1eb/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | digest | | [deps.dev/util/semver](https://github.com/google/deps.dev) | `af20eef` -> `b147e04` | [![age](https://developer.mend.io/api/mc/badges/age/go/deps.dev%2futil%2fsemver/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/deps.dev%2futil%2fsemver/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/deps.dev%2futil%2fsemver/v0.0.0-20240611045547-af20eef0f1eb/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/deps.dev%2futil%2fsemver/v0.0.0-20240611045547-af20eef0f1eb/v0.0.0-20240617015216-b147e04533eb?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | digest | | [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) | `v0.19.1` -> `v0.19.2` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgoogle%2fgo-containerregistry/v0.19.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fgoogle%2fgo-containerregistry/v0.19.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fgoogle%2fgo-containerregistry/v0.19.1/v0.19.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgoogle%2fgo-containerregistry/v0.19.1/v0.19.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | patch | | golang.org/x/exp | `fc45aab` -> `7f521ea` | [![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fexp/v0.0.0-20240613232115-7f521ea00fb8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/golang.org%2fx%2fexp/v0.0.0-20240613232115-7f521ea00fb8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/golang.org%2fx%2fexp/v0.0.0-20240604190554-fc45aab8b7f8/v0.0.0-20240613232115-7f521ea00fb8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fexp/v0.0.0-20240604190554-fc45aab8b7f8/v0.0.0-20240613232115-7f521ea00fb8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | digest | | [google.golang.org/protobuf](https://github.com/protocolbuffers/protobuf-go) | `v1.34.1` -> `v1.34.2` | [![age](https://developer.mend.io/api/mc/badges/age/go/google.golang.org%2fprotobuf/v1.34.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/google.golang.org%2fprotobuf/v1.34.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/google.golang.org%2fprotobuf/v1.34.1/v1.34.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/google.golang.org%2fprotobuf/v1.34.1/v1.34.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | require | patch | --- ### Release Notes <details> <summary>google/go-containerregistry (github.com/google/go-containerregistry)</summary> ### [`v0.19.2`](https://github.com/google/go-containerregistry/releases/tag/v0.19.2) [Compare Source](https://github.com/google/go-containerregistry/compare/v0.19.1...v0.19.2) #### What's Changed - Add JSON marshalling funcs for Digest. by [@​wlynch](https://github.com/wlynch) in [https://github.com/google/go-containerregistry/pull/1915](https://github.com/google/go-containerregistry/pull/1915) - registry: Implement Range requests for blobs by [@​jonjohnsonjr](https://github.com/jonjohnsonjr) in [https://github.com/google/go-containerregistry/pull/1917](https://github.com/google/go-containerregistry/pull/1917) - Support podman auth file REGISTRY_AUTH_FILE. by [@​zhaoyonghe](https://github.com/zhaoyonghe) in [https://github.com/google/go-containerregistry/pull/1914](https://github.com/google/go-containerregistry/pull/1914) - feat: crane mutate platform by [@​joshwlewis](https://github.com/joshwlewis) in [https://github.com/google/go-containerregistry/pull/1919](https://github.com/google/go-containerregistry/pull/1919) - Add Context support to auth methods by [@​jonjohnsonjr](https://github.com/jonjohnsonjr) in [https://github.com/google/go-containerregistry/pull/1949](https://github.com/google/go-containerregistry/pull/1949) - Fix windows race condition when writing image with duplicate layers by [@​dgannon991](https://github.com/dgannon991) in [https://github.com/google/go-containerregistry/pull/1921](https://github.com/google/go-containerregistry/pull/1921) - Add -O shorthand for --omit-digest-tags to crane. by [@​smoser](https://github.com/smoser) in [https://github.com/google/go-containerregistry/pull/1958](https://github.com/google/go-containerregistry/pull/1958) #### New Contributors - [@​wlynch](https://github.com/wlynch) made their first contribution in [https://github.com/google/go-containerregistry/pull/1915](https://github.com/google/go-containerregistry/pull/1915) - [@​zhaoyonghe](https://github.com/zhaoyonghe) made their first contribution in [https://github.com/google/go-containerregistry/pull/1914](https://github.com/google/go-containerregistry/pull/1914) - [@​joshwlewis](https://github.com/joshwlewis) made their first contribution in [https://github.com/google/go-containerregistry/pull/1919](https://github.com/google/go-containerregistry/pull/1919) - [@​dgannon991](https://github.com/dgannon991) made their first contribution in [https://github.com/google/go-containerregistry/pull/1921](https://github.com/google/go-containerregistry/pull/1921) - [@​smoser](https://github.com/smoser) made their first contribution in [https://github.com/google/go-containerregistry/pull/1958](https://github.com/google/go-containerregistry/pull/1958) **Full Changelog**: google/go-containerregistry@v0.19.1...v0.19.2 </details> <details> <summary>protocolbuffers/protobuf-go (google.golang.org/protobuf)</summary> ### [`v1.34.2`](https://github.com/protocolbuffers/protobuf-go/releases/tag/v1.34.2) [Compare Source](https://github.com/protocolbuffers/protobuf-go/compare/v1.34.1...v1.34.2) Minor feature: - [CL/589336](https://go.dev/cl/589336): gofeatures: allow setting legacy_unmarshal_json_enum feature at file level Minor bug fixes: - [CL/588875](https://go.dev/cl/588875): types/descriptorpb: regenerate using latest protobuf v27.0 release - [CL/586396](https://go.dev/cl/586396): internal/impl: fix size cache semantics with lazy decoding - [CL/585736](https://go.dev/cl/585736): reflect/protodesc: remove obsolete JSON name check from desc validator - [CL/588976](https://go.dev/cl/588976): reflect/protoreflect: FieldDescriptor.Kind should never be GroupKind for maps or fields of map entry </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 6am on monday" in timezone Australia/Sydney, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/google/osv-scanner). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zOTMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjM5My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
This is a potential solution for #1920, any feedback would be greatly appreciated.