Some OCI Optimizations #1764

Noxsios · 2023-06-01T07:26:46Z

Description

zarf deploy oci:// --components=... Given a call to zarf package deploy from an oci:// URL, with a list of --component options, I'd like Zarf to pull only those layers necessary to deploy the selected components.
zarf package create . -o oci://... calling this guy inline publising. This is essentially the same thing as doing a zarf package create && zarf package publish, but in the future this could also be smart about images, and only fetch images if their ref from the OCI package is different.
re-architect + restructure our OCI library code to actually be usable as a library
zarf package remove oci://

Related Issue

Closes #1779
Relates to #1763
Closes #956

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Other (security config, docs update, etc)

Checklist before merging

Test, docs, adr added or updated as needed
Contributor Guide Steps followed

Signed-off-by: razzle <harry@razzle.cloud>

netlify · 2023-06-01T07:26:49Z

✅ Deploy Preview for zarf-docs canceled.

Name	Link
🔨 Latest commit	`06c5d04`
🔍 Latest deploy log	https://app.netlify.com/sites/zarf-docs/deploys/64890bf178653200088253ec

Signed-off-by: razzle <harry@razzle.cloud>

src/pkg/packager/common.go

src/pkg/packager/network.go

Signed-off-by: razzle <harry@razzle.cloud>

src/pkg/utils/oras.go

Signed-off-by: razzle <harry@razzle.cloud>

Racer159

Overall looks good, but did find one issue.

src/pkg/oci/pull.go

Signed-off-by: razzle <harry@razzle.cloud>

src/pkg/oci/pull.go

src/cmd/package.go

src/cmd/viper.go

src/internal/api/components/list.go

src/pkg/oci/common.go

src/pkg/oci/pull.go

src/pkg/packager/common.go

src/pkg/utils/checksum.go

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>

Signed-off-by: razzle <harry@razzle.cloud>

Racer159

lint

src/internal/api/components/list.go

src/pkg/oci/pull.go

Signed-off-by: razzle <harry@razzle.cloud>

Racer159

lgtm

wip

6bf8449

Signed-off-by: razzle <harry@razzle.cloud>

Noxsios linked an issue Jun 1, 2023 that may be closed by this pull request

OCI Layer Optimizations #1763

Closed

Noxsios added 3 commits June 1, 2023 02:26

Merge branch 'main' into 1763-oci-layer-optimizations

2e47273

deploying a single component via OCI is working

b070c48

Signed-off-by: razzle <harry@razzle.cloud>

handle required components needing to be always pulled

9866b10

Signed-off-by: razzle <harry@razzle.cloud>

Noxsios changed the title ~~OCI layer optimizations~~ zarf deploy oci:// components=... Jun 1, 2023

lint

7c63e10

Signed-off-by: razzle <harry@razzle.cloud>

Noxsios removed a link to an issue Jun 1, 2023

OCI Layer Optimizations #1763

Closed

Noxsios marked this pull request as ready for review June 1, 2023 14:33

Noxsios requested review from jeff-mccoy, YrrepNoj and Racer159 as code owners June 1, 2023 14:33

Merge branch 'main' into 1763-oci-layer-optimizations

87b129f

Noxsios changed the title ~~zarf deploy oci:// components=...~~ zarf deploy oci:// --components=... Jun 1, 2023

Merge branch 'main' into 1763-oci-layer-optimizations

7db7554

Racer159 requested changes Jun 2, 2023

View reviewed changes

src/pkg/packager/common.go Outdated Show resolved Hide resolved

src/pkg/packager/network.go Outdated Show resolved Hide resolved

Racer159 mentioned this pull request Jun 2, 2023

Allow Zarf to accept an OCI URL on package removal #1779

Closed

Noxsios added 10 commits June 2, 2023 23:09

merge main

6da2fab

Signed-off-by: razzle <harry@razzle.cloud>

committing in case i lose power in a sec

b4d20e4

Signed-off-by: razzle <harry@razzle.cloud>

should be working now

5e826a7

Signed-off-by: razzle <harry@razzle.cloud>

naming

c81f1a7

Signed-off-by: razzle <harry@razzle.cloud>

simpler calculate size

1742243

Signed-off-by: razzle <harry@razzle.cloud>

fix deploy w/ components after refactor

48c642f

Signed-off-by: razzle <harry@razzle.cloud>

comments

875b9a9

Signed-off-by: razzle <harry@razzle.cloud>

cleanup

d55ac15

Signed-off-by: razzle <harry@razzle.cloud>

Merge branch 'main' into 1763-oci-layer-optimizations

3e6826d

cleanup

2290d4f

Signed-off-by: razzle <harry@razzle.cloud>

YrrepNoj reviewed Jun 5, 2023

View reviewed changes

src/pkg/utils/oras.go Outdated Show resolved Hide resolved

src/pkg/utils/oras.go Outdated Show resolved Hide resolved

Merge branch 'main' into 1763-oci-layer-optimizations

ef940f9

Noxsios added 6 commits June 12, 2023 02:59

fix test 50

b18de26

Signed-off-by: razzle <harry@razzle.cloud>

message tweak

8b48fd5

Signed-off-by: razzle <harry@razzle.cloud>

message tweak

ef5eb56

Signed-off-by: razzle <harry@razzle.cloud>

message tweak

82b259b

Signed-off-by: razzle <harry@razzle.cloud>

message tweak

d743f78

Signed-off-by: razzle <harry@razzle.cloud>

testing github group

f6a4ae0

Signed-off-by: razzle <harry@razzle.cloud>

Racer159 requested changes Jun 12, 2023

View reviewed changes

src/pkg/oci/pull.go Outdated Show resolved Hide resolved

Noxsios added 2 commits June 12, 2023 13:09

feedback from review

aadcfd5

Signed-off-by: razzle <harry@razzle.cloud>

Merge branch 'main' into 1763-oci-layer-optimizations

63ea4a5

Racer159 requested changes Jun 12, 2023

View reviewed changes

src/pkg/oci/pull.go Outdated Show resolved Hide resolved

Merge branch 'main' into 1763-oci-layer-optimizations

9137417

Racer159 requested changes Jun 12, 2023

View reviewed changes

Noxsios and others added 12 commits June 12, 2023 22:01

Update src/pkg/oci/pull.go

7357b3e

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>

Update src/internal/api/components/list.go

5740e81

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>

Update src/pkg/oci/pull.go

1f357d5

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>

Update src/pkg/packager/common.go

6e04b29

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>

Update src/pkg/utils/checksum.go

4a398da

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>

feedback from review

a6a7a1b

Signed-off-by: razzle <harry@razzle.cloud>

feedback from review

1e176f1

Signed-off-by: razzle <harry@razzle.cloud>

Merge branch 'main' into 1763-oci-layer-optimizations

d2588b1

feedback from review

2dbac28

Signed-off-by: razzle <harry@razzle.cloud>

verify agg checksums exist before validating

44611f0

Signed-off-by: razzle <harry@razzle.cloud>

check agg checksum within validate

d4e1bec

Signed-off-by: razzle <harry@razzle.cloud>

dont validate checksums on packages that do not have agg checksums

4e76a6f

Signed-off-by: razzle <harry@razzle.cloud>

Racer159 requested changes Jun 13, 2023

View reviewed changes

src/internal/api/components/list.go Show resolved Hide resolved

src/pkg/oci/pull.go Show resolved Hide resolved

lint

06c5d04

Signed-off-by: razzle <harry@razzle.cloud>

Racer159 approved these changes Jun 14, 2023

View reviewed changes

Racer159 mentioned this pull request Jun 14, 2023

Specify registry scopes separately per package publish and package pull #1821

Closed

Racer159 merged commit ee62b29 into main Jun 14, 2023

Racer159 deleted the 1763-oci-layer-optimizations branch June 14, 2023 13:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some OCI Optimizations #1764

Some OCI Optimizations #1764

Noxsios commented Jun 1, 2023 •

edited

Loading

netlify bot commented Jun 1, 2023 •

edited

Loading

Racer159 left a comment

Racer159 left a comment

Racer159 left a comment

Some OCI Optimizations #1764

Some OCI Optimizations #1764

Conversation

Noxsios commented Jun 1, 2023 • edited Loading

Description

Related Issue

Type of change

Checklist before merging

netlify bot commented Jun 1, 2023 • edited Loading

✅ Deploy Preview for zarf-docs canceled.

Racer159 left a comment

Choose a reason for hiding this comment

Racer159 left a comment

Choose a reason for hiding this comment

Racer159 left a comment

Choose a reason for hiding this comment

Noxsios commented Jun 1, 2023 •

edited

Loading

netlify bot commented Jun 1, 2023 •

edited

Loading