Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No empty layers #293

Merged
merged 2 commits into from
May 1, 2019
Merged

No empty layers #293

merged 2 commits into from
May 1, 2019

Conversation

tych0
Copy link
Member

@tych0 tych0 commented Apr 2, 2019

Here's some code to avoid inserting empty layers, and instead just creating a history entry to log that something happened.

test/repack.bats Outdated
@@ -231,7 +231,7 @@ function teardown() {
bundle-verify "$BUNDLE"

# Make some small change.
touch "$BUNDLE/a_small_change"
touch "$BUNDLE/rootfs/a_small_change"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use $ROOTFS -- I thought I fixed all of these incorrect $BUNDLE usages...

test/repack.bats Outdated

layers0=$(cat "${IMAGE}/oci/blobs/sha256/$manifest0" | jq -r .layers)
layers1=$(cat "${IMAGE}/oci/blobs/sha256/$manifest1" | jq -r .layers)
[ "$layers0" == "$layers1" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent tabs and spaces -- use tabs everywhere.

test/repack.bats Outdated

@test "umoci repack (empty diff)" {
# Unpack the original image
new_bundle_rootfs && BUNDLE_A="$BUNDLE" ROOTFS_A="$ROOTFS"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're not going to use BUNDLE_A and ROOTFS_A you can drop the && ... section of this line.

@tych0
Copy link
Member Author

tych0 commented Apr 3, 2019

Huh. No idea why this one failed. Maybe a race? There's lots of goroutines in that code path...

@cyphar
Copy link
Member

cyphar commented Apr 3, 2019

Yeah, I'm not sure. Every once in a while we see a flaky test (and this one was odd -- a return code was 0 when it shouldn't have been). At some point I should run some stress-like tests to see what falls out.

Anyway, I re-ran it and it passes. I'll review this one tomorrow. The main thing that strikes me is that mutator.Set appears to be quite ugly to use if you are changing only one piece of the metadata -- sucks there's no Option typeclass in Go and we'd need to do that with pointers. But we can fix that in a future PR.

@tych0
Copy link
Member Author

tych0 commented Apr 17, 2019

Ping, any word on this?

1 similar comment
@tych0
Copy link
Member Author

tych0 commented Apr 30, 2019

Ping, any word on this?

@cyphar
Copy link
Member

cyphar commented May 1, 2019

Yup sorry, this looks okay. I might prefer having a separate config.go which meant we didn't need to have a copy of this logic here but I don't mind either way. For OCIv2 (which I have started working on again) we will need to restructure quite a bit of the wrappers so that the same interfaces can support both layered rootfs-es (OCIv1) and tree-based rootfs-es (OCIv2).

LGTM.

tych0 added 2 commits May 1, 2019 17:13
$BUNDLE is really the path to the top level thing, and we only detect
changes to $BUNDLE/rootfs. So when we make changes, let's actually make
them to the rootfs.

The reason this worked before is that we always insert a layer, even if
it's empty. With the next patch, we wouldn't do this any more, causing this
test to fail.

Signed-off-by: Tycho Andersen <tycho@tycho.ws>
Let's not insert an empty gzip layer if there is no diff.

It might make sense to not even insert a history entry, but it also seems
reasonable to want to log that umoci ran at some point, so we do that, but
just leave everything unchanged.

Signed-off-by: Tycho Andersen <tycho@tycho.ws>
@cyphar
Copy link
Member

cyphar commented May 1, 2019

I think I've figured out the root cause of the failure. I think it's because the current trick of returning errors through write.CloseWithError seems to have some sort of race condition involved...

@cyphar
Copy link
Member

cyphar commented May 1, 2019

LGTM.

@cyphar cyphar merged commit b1df418 into opencontainers:master May 1, 2019
cyphar added a commit that referenced this pull request May 1, 2019
  repack: don't insert a tar gzip layer if there is no diff
  tests: fix typo

LGTMs: @cyphar
Closes #293
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants