Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kic: fix var race condition #8657

Closed
medyagh opened this issue Jul 6, 2020 · 6 comments
Closed

kic: fix var race condition #8657

medyagh opened this issue Jul 6, 2020 · 6 comments
Assignees
Labels
co/docker-driver Issues related to kubernetes in container co/podman-driver podman driver issues kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone

Comments

@medyagh
Copy link
Member

medyagh commented Jul 6, 2020

No description provided.

@medyagh medyagh added this to the v1.13.0-candidate milestone Jul 6, 2020
@afbjorklund afbjorklund added co/docker-driver Issues related to kubernetes in container co/podman-driver podman driver issues labels Jul 6, 2020
@afbjorklund
Copy link
Collaborator

The race condition is because we have /var both in the kicbase image (from ubuntu), and /var mounted as a volume.

And if the volume wins the race, then we won't copy those files from the image to the volume - and miss some important ones...

@afbjorklund afbjorklund added kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Jul 6, 2020
@afbjorklund
Copy link
Collaborator

afbjorklund commented Jul 6, 2020

This is related to #8151 and #8458 as well, since we don't mount the directories needed for the storage provisioner.

But if we add a minikube-automount to the KIC, then it can take care of adding all of these directories to the volume.

@medyagh
Copy link
Member Author

medyagh commented Jul 13, 2020

dupe of #8458

@medyagh medyagh closed this as completed Jul 13, 2020
@afbjorklund afbjorklund reopened this Jul 20, 2020
@afbjorklund
Copy link
Collaborator

afbjorklund commented Jul 20, 2020

The #8458 and #8151 are about making sure that the storage provisioner mounts the correct directories.
It is similar to the same issue for the "none" driver, where the user is supposed to mount them: #7511

However, this particular issue is about the race condition that happens between "start" and "preload".
We have one container that uses a named volume (copying contents), and the creation of that volume.

So we are going to fix the storage mounts in that other issue, and this is about fixing the start up order.
Initially I was going to change the /var mount point, but since that will break with KIND we will delay that.

Instead we are going to make sure that the volume is created by "docker run", and not create an empty one.
This way we make sure that the image contents are present, such as /var/lib/dpkg/alternative which is needed.

It's a bit unfortunate that we opened a new "blank" story for the new milestone, but the issue is: #8179 #8100
We saw this already with the podman driver, but there the workaround was to disable the parallell preload: #8056

So the error we are trying to fix here, is the bootup error:

update-alternatives: error: no alternatives for iptables

  • oci.PrepareContainerNode (docker volume create)

  • oci.CreateContainerNode (docker run, /var)

  • oci.ExtractTarballToVolume (docker run, /extractDir)

Or we have some more pretty way to make sure it happens...

Maybe changing using /var also for tar, would be enough ?

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jul 20, 2020

As mentioned in the other issues, we want to copy:

  • /var/lib/dpkg (etc )from the ubuntu image

backups cache lib local lock log mail opt run spool tmp

  • /var/lib/docker (etc) from the minikube preload

lib

And we need to have both of them in the final volume.

@afbjorklund
Copy link
Collaborator

Now that the real issues have been added, we don't need this (redundant) placeholder

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/docker-driver Issues related to kubernetes in container co/podman-driver podman driver issues kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

No branches or pull requests

2 participants