Skip to content

Commit

Permalink
fix: Update ExtractSingleSquash, adding policy.
Browse files Browse the repository at this point in the history
Overall, there are 3 "good things" done by this change:
1. Fix bug in the current code which tries mounting with each option
   every time.  The problem with doing that is really that the kernel
   mount option didn't work very well.  It would fail with "already
   mounted", and then squashfuse would end up getting used to mount over
   the top.

2. Fix race conditions in the current code.

   overlay.Unpack starts a thread pool and tries to unpack all layers
   at once.  That is fine if there are no duplicate layers.  But
      if there are duplicate layers used by a stacker.yaml file, then
   there were races on extraction.  The end result really was that things
   would get mounted more than once.

   Example stacker that shows this:

    l1:
      from:
        type: docker
        url: docker://busybox:latest
      run: |
        echo build layer 1

    l2:
      from:
        type: docker
        url: docker://busybox:latest
      run: |
        echo build layer 1

  There, the busybox layer would get extracted multiple times.

  The code here has a single lock on ExtractSingleSquash, it would
  be better to have lock being taken per extractDir.

3. Allow the user to control the list of extractors.

   If they knew that they could not use kernel mounts (or could, but
   didn't want to) or wanted to use unsquashfs they can now do that.

   STACKER_SQUASHFS_EXTRACT_POLICY=kmount stacker build ..

   or

   STACKER_SQUASHFS_EXTRACT_POLICY="squashfuse kmount" stacker build ...

   This adds a SquashExtractor interface, with 3 implementers
   (KernelExtractor, SquashFuseExtractor, UnsquashfsExtractor).

   A ExtractPolicy is basically a list of Extractors to try.
   The first time ExtractPolicy is used it will try each of the Extractors
   in order.  It then stores the result in .Extractor and uses that
   subsequently.

Signed-off-by: Scott Moser <smoser@brickies.net>
  • Loading branch information
smoser committed Oct 4, 2023
1 parent f7bb116 commit 521ee24
Show file tree
Hide file tree
Showing 3 changed files with 342 additions and 59 deletions.
3 changes: 2 additions & 1 deletion pkg/overlay/pack.go
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,8 @@ func (o *overlay) initializeBasesInOutput(name string, layerTypes []types.LayerT
return err
}
} else {
log.Debugf("converting between %v and %v", sourceLayerType, layerType)
log.Debugf("creating layer %s (type=%s) by converting layer %s (type=%s)",
layerType.LayerName(name), layerType, sourceLayerType.LayerName(name), sourceLayerType)
err = ConvertAndOutput(o.config, cacheTag, name, layerType)
if err != nil {
return err
Expand Down
Loading

0 comments on commit 521ee24

Please sign in to comment.