feat: Squash builds

[gmpinder]

Buildah/Podman support

Buildah and podman can make heavy use of the squash feature. Something that I've noticed when trying to build from inside of a container, requiring intermediate layers with mounts causes build times to skyrocket. Build times are much faster when using the --squash functionality (seen as --layers=false).

Here are the following results from my personal build using both squash and non-squash functionality.

Squash upgrade:

$> rpm-ostree upgrade
Pulling manifest: ostree-image-signed:docker://registry.gitlab.com/wunker-bunker/wunker-os/jp-laptop
Importing: ostree-image-signed:docker://registry.gitlab.com/wunker-bunker/wunker-os/jp-laptop (digest: sha256:60f743ba322041918d302e7e7f10438c59502e19343c294064bacb676c8eb7b7)
ostree chunk layers already present: 65
custom layers already present: 3
custom layers needed: 1 (814.0 MB)

All changes appear to show as a single custom layer. Any small change even at the end of the build appears to require completely downloading the new layer (squash only squashes additional layers on top of the base layer). This makes sense as layers cannot currently be downloaded by diff.

Non-squash upgrade:

$> rpm-ostree upgrade
Pulling manifest: ostree-image-signed:docker://registry.gitlab.com/wunker-bunker/wunker-os/jp-desktop-gaming:latest
Importing: ostree-image-signed:docker://registry.gitlab.com/wunker-bunker/wunker-os/jp-desktop-gaming:latest (digest: sha256:0658b51febfcbaa1722961b7a6d2b197d3823a6228e330f45dd1e1aaefd145c5)
ostree chunk layers already present: 65
custom layers already present: 4
custom layers needed: 15 (942.4 MB)

As expected, there are more layers when not squashing and the size is slightly bigger. Most likely due to there being extra information stored in the layers that is subsequently removed.

Docker support

Docker is apparently no longer supporting the use of the --squash arg. The use of squash will not be available for the docker driver in this case.

[xynydev]

I'm not really able to review Rust code quality, but it looks alright to me.

[dperson]

@gmpinder just wanted to check that I'm calling it correctly from a modified bluebuild/github-action based on v1.1.1:

podman run \                                                             
  -v buildah-vol:/var/run/containerd \                                   
  -v $PWD:/bluebuild \                                                   
  --env-host \                                                           
  --network=host \                                                       
  --privileged \                                                         
  --device /dev/fuse \                                                                 
  ghcr.io/blue-build/cli:latest-alpine \                                               
  build -vv --squash --push ./config/${{ inputs.recipe }} \                            
  --registry ${{inputs.registry}} \                                                    
  --registry-namespace ${{inputs.registry_namespace}}                                  

Thanks so much!

[gmpinder]

@gmpinder just wanted to check that I'm calling it correctly from a modified bluebuild/github-action based on v1.1.1:

podman run \                                                             
  -v buildah-vol:/var/run/containerd \                                   
  -v $PWD:/bluebuild \                                                   
  --env-host \                                                           
  --network=host \                                                       
  --privileged \                                                         
  --device /dev/fuse \                                                                 
  ghcr.io/blue-build/cli:latest-alpine \                                               
  build -vv --squash --push ./config/${{ inputs.recipe }} \                            
  --registry ${{inputs.registry}} \                                                    
  --registry-namespace ${{inputs.registry_namespace}}                                  

Thanks so much!

That looks correct from what I can see.

I'd also suggest trying the latest github action with cache enabled and move your recipe files into ./recipes. We've found that to work really well for incremental changes as it will only pull the layers you've changed since the last build. You can see my findings in this PR #157. Any upstream image changes will cause the entire build to be rebuilt purely due to how docker images work though.

[gmpinder]

So some clarification @dperson , we don't quite have the cache abilities released yet but we're working on it. We are also adding the ability to squash in the GHA (blue-build/github-action#41), so you won't have to manage the action yourself soon.