Skip to content

Commit

Permalink
Improve dmverity-vhd -d performance (microsoft#2089)
Browse files Browse the repository at this point in the history
The current implementation of dmverity-vhd -d has to make one of the
following tradeoffs:

Runtime: By default this option calls the docker daemon to fetch the
entire image for each layer as it doesn't provide an endpoint to get
a specific layer
Memory: The user can include a -b option that makes this call buffered,
keeping the image in memory the whole time, this is much faster but
at the cost of keeping the whole image in memory, which is a problem
with runners with low memory

microsoft#2086 Proposed a new tradeoff of disk space, by saving the image to
disk and accessing the layers locally, this is a problem for runners with
smaller disks as the image is stored twice.

This solution makes a single request to the docker daemon, and
processes both the layer hashes and the manifest to assign layer
numbers in a single pass, making it performant in all three aspects.

---------

Signed-off-by: Dominic Ayre <dominicayre@microsoft.com>
Signed-off-by: Dominic Ayre <domayre@outlook.com>
  • Loading branch information
DomAyre authored Apr 12, 2024
1 parent c09ae9d commit f9a5c7b
Show file tree
Hide file tree
Showing 8 changed files with 376 additions and 698 deletions.
510 changes: 375 additions & 135 deletions cmd/dmverity-vhd/main.go

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
github.com/containerd/protobuild v0.3.0
github.com/containerd/ttrpc v1.2.3
github.com/containerd/typeurl/v2 v2.1.1
github.com/docker/docker v24.0.9+incompatible
github.com/google/go-cmp v0.6.0
github.com/google/go-containerregistry v0.19.1
github.com/josephspurrier/goversioninfo v1.4.0
Expand Down Expand Up @@ -56,7 +57,6 @@ require (
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/docker/cli v24.0.0+incompatible // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/docker/docker v24.0.9+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
Expand Down

This file was deleted.

17 changes: 0 additions & 17 deletions vendor/github.com/google/go-containerregistry/pkg/v1/daemon/doc.go

This file was deleted.

350 changes: 0 additions & 350 deletions vendor/github.com/google/go-containerregistry/pkg/v1/daemon/image.go

This file was deleted.

Loading

0 comments on commit f9a5c7b

Please sign in to comment.