Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hard link disappeared when built docker image #412

Open
lvii opened this issue Nov 16, 2017 · 4 comments · May be fixed by #455
Open

Hard link disappeared when built docker image #412

lvii opened this issue Nov 16, 2017 · 4 comments · May be fixed by #455

Comments

@lvii
Copy link

lvii commented Nov 16, 2017

After I built a docker image, docker image tar file size was more than VM's raw disk size :

# qemu-img info /var/lib/imagefactory/storage/422c0e0e-c027-4e5b-ab6f-5506b0680d25.body
image: /var/lib/imagefactory/storage/422c0e0e-c027-4e5b-ab6f-5506b0680d25.body
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 300M

# ll -s /var/lib/imagefactory/storage/cc5b5cbc-5eeb-471e-967e-c2a3a00704bb.body
555M -rw-r--r--. 1 root root 555M | 2017-11-16 03:58 | /var/lib/imagefactory/storage/cc5b5cbc-5eeb-471e-967e-c2a3a00704bb.body

I decompressed docker tar file, and I find that /usr/libexec/git-core directory occupied space :

# du -sh usr/libexec/*|sort  -h|tail
...
536K    usr/libexec/sudo
908K    usr/libexec/openssh
265M    usr/libexec/git-core

Binary files in /usr/libexec/git-core directory were almost hard link.

@juhp
Copy link

juhp commented Oct 19, 2023

Well at least today (F39) /usr/libexec/git-core/ seems full of symlinks

@owtaylor
Copy link

The way that the Docker plugin works is that it mounts the image with libguestfs (using FUSE), and then runs tar on the result. However, FUSE, by default, makes up the inodes without regard to the underlying inodes, so tars hardlink detection in tar can't work.

There would be two fixes for this:

A) pass the use_ino option to the mount call, Something like:

diff --git a/imagefactory_plugins/Docker/Docker.py b/imagefactory_plugins/Docker/Docker.py
index 68e12c3..3b44603 100644
--- a/imagefactory_plugins/Docker/Docker.py
+++ b/imagefactory_plugins/Docker/Docker.py
@@ -317,7 +317,7 @@ class Docker(object):
         try:
             tempdir = tempfile.mkdtemp(dir=storagedir)
             self.log.debug("Mounting input image locally at (%s)" % (tempdir))
-            guestfs_handle.mount_local(tempdir)
+            guestfs_handle.mount_local(tempdir, options="use_ino")
             def _run_guestmount(g):
                 g.mount_local_run()
             self.log.debug("Launching mount_local_run thread")

(not tested since I'm having trouble getting imagefactory to work...)

B) Skip the guestmount and use guestfs_tar_out_opts() which would simplify the code quite a bit (no need to start a thread) and likely perform considerably better. The computation of layer size would take a bit of adoption, but I think this is some old obsolete thing - if you use skopeo to copy to 'docker-archive:' you don't get that field.

@juhp
Copy link

juhp commented Oct 20, 2023

Perhaps your one-line patch/workaround could be tested for Fedora Rawhide?
I opened https://pagure.io/releng/issue/11735 to see.

owtaylor added a commit to owtaylor/imagefactory that referenced this issue Oct 20, 2023
When use libguestfs to mount the intermediate image to create a tarball,
pass the "use_ino" option so that original inode numbers are preserved.
This fixes a problem where hardlinks were not properly preserved in the
output.

This is a minimal-code-change fix - an edge case bug and better approach are
described in comments.

Resolves redhat-imaging#412
@owtaylor owtaylor linked a pull request Oct 20, 2023 that will close this issue
@owtaylor
Copy link

Turned my pseudo-patch into a a pull request. It's still not actually tested for an image build, but I created a test program to verify the syntax and make sure that it has the intended result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants