Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manual Workflow For Publishing Application Packages to GHCR #1856

Merged
merged 29 commits into from
Jul 27, 2023
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5580380
add version to the dos-games example
YrrepNoj Jun 27, 2023
7e9236e
add workflow for publishing the dos-games package to GHCR
YrrepNoj Jun 27, 2023
9eed4cd
Merge branch 'main' into 758-publish-example-packages
Racer159 Jun 27, 2023
0dd0587
update dos games verion to 1.0.0
YrrepNoj Jun 28, 2023
6c276ff
Merge branch 'main' into 758-publish-example-packages
YrrepNoj Jun 28, 2023
431e8f9
Merge branch 'main' into 758-publish-example-packages
YrrepNoj Jun 28, 2023
df73613
Merge branch 'main' into 758-publish-example-packages
YrrepNoj Jun 29, 2023
95f2252
Merge branch 'main' into 758-publish-example-packages
Racer159 Jun 30, 2023
9c88aa9
update workflow for publishing packages
YrrepNoj Jun 30, 2023
cd59bf4
Merge branch 'main' into 758-publish-example-packages
Racer159 Jul 2, 2023
111219b
add warnings about depercating sget functionality within Zarf
YrrepNoj Jul 3, 2023
58a3b23
add permissions to the package publishing workflow
YrrepNoj Jul 11, 2023
2a91eab
add depreication notice to the 'CosignKeyPath' field description with…
YrrepNoj Jul 11, 2023
1c43f3e
update docs and schema
YrrepNoj Jul 11, 2023
721e26e
Merge branch 'main' into 758-publish-example-packages
YrrepNoj Jul 11, 2023
24cc1fa
update docs and schema
YrrepNoj Jul 11, 2023
82da0e9
Merge branch 'main' into 758-publish-example-packages
Racer159 Jul 22, 2023
25cc874
use kms key to sign packages that are created during every release
YrrepNoj Jul 24, 2023
a094d3f
update docs and schema
YrrepNoj Jul 24, 2023
490e0fd
Update docs/2-the-zarf-cli/100-cli-commands/zarf_package_deploy.md
YrrepNoj Jul 24, 2023
207177a
remove unnecessary nodejs import for publish workflow
YrrepNoj Jul 24, 2023
1b599c3
update deprecation warning and update docs and schema
YrrepNoj Jul 24, 2023
49024ec
update deprecation warning and update a TODO comment
YrrepNoj Jul 24, 2023
c6433ab
update docs and schema
YrrepNoj Jul 24, 2023
cd974e6
publish the release applications to the 'packages' sub namespace
YrrepNoj Jul 24, 2023
a3ab42f
reduce flux overrides for the helm controler in the bigbang extension
YrrepNoj Jul 24, 2023
e7069ca
Merge branch 'main' into 758-publish-example-packages
Racer159 Jul 27, 2023
b73f515
Merge branch 'main' into 758-publish-example-packages
Racer159 Jul 27, 2023
f6df16b
revert that change where we reduced flux overrides for the helm contr…
YrrepNoj Jul 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/publish-application-packages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Zarf Application Package Publishing

permissions:
contents: read

on:
workflow_dispatch:
inputs:
branchName:
description: "Branch to build the packages from"
required: true
default: "main"

jobs:
publish-packages:
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- name: "Checkout Repo"
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2
with:
ref: ${{ github.event.inputs.branchName }}

- name: Install The Latest Release Version of Zarf
uses: defenseunicorns/setup-zarf@main

- name: "Login to GHCR"
uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # v2.1.0
with:
registry: ghcr.io
username: dummy
password: ${{ github.token }}

- name: Build And Publish Application Packages
# TODO: Add package signing to this step
YrrepNoj marked this conversation as resolved.
Show resolved Hide resolved
run: |
make build-release-packages KEY=awskms:///${{ secrets.COSIGN_AWS_KMS_KEY }}
make publish-release-packages
15 changes: 13 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

# Provide a default value for the operating system architecture used in tests, e.g. " APPLIANCE_MODE=true|false make test-e2e ARCH=arm64"
ARCH ?= amd64
KEY ?= ""
######################################################################################

# Figure out which Zarf binary we should use based on the operating system we are on
Expand Down Expand Up @@ -127,10 +128,20 @@ ib-init-package:
--set REGISTRY_IMAGE="ironbank/opensource/docker/registry-v2" \
--set REGISTRY_IMAGE_TAG="2.8.2"

# INTERNAL used to build the dos games packages for release
build-release-packages:
$(ZARF_BIN) package create -o build -a amd64 examples/dos-games --key=$(KEY) --confirm
$(ZARF_BIN) package create -o build -a arm64 examples/dos-games --key=$(KEY) --confirm

# INTERNAL used to publish the dos games packages to GHCR
publish-release-packages:
$(ZARF_BIN) package publish ./build/zarf-package-dos-games-amd64-1.0.0.tar.zst oci://ghcr.io/defenseunicorns/zarf-package
$(ZARF_BIN) package publish ./build/zarf-package-dos-games-arm64-1.0.0.tar.zst oci://ghcr.io/defenseunicorns/zarf-package

build-examples: ## Build all of the example packages
@test -s $(ZARF_BIN) || $(MAKE) build-cli

@test -s ./build/zarf-package-dos-games-$(ARCH).tar.zst || $(ZARF_BIN) package create examples/dos-games -o build -a $(ARCH) --confirm
@test -s ./build/zarf-package-dos-games-$(ARCH)-1.0.0.tar.zst || $(ZARF_BIN) package create examples/dos-games -o build -a $(ARCH) --confirm

@test -s ./build/zarf-package-manifests-$(ARCH)-0.0.1.tar.zst || $(ZARF_BIN) package create examples/manifests -o build -a $(ARCH) --confirm

Expand Down Expand Up @@ -196,7 +207,7 @@ test-ui-dev-server:

.PHONY: test-ui-build-server
# INTERNAL: used to start the built version of the API server for the Zarf Web UI (in CI)
test-ui-build-server:
test-ui-build-server:
API_PORT=3333 API_TOKEN=insecure $(ZARF_BIN) dev ui

# INTERNAL: used to test that a dev has ran `make docs-and-schema` in their PR
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ zarf package deploy [ PACKAGE ] [flags]
-h, --help help for deploy
-k, --key string Path to public key file for validating signed packages
--set stringToString Specify deployment variables to set on the command line (KEY=value) (default [])
--sget string Path to public sget key file for remote packages signed via cosign
--sget string [Deprecated] Path to public sget key file for remote packages signed via cosign. This flag will be removed in v0.31.0 please use the --key flag instead.
--shasum string Shasum of the package to deploy. Required if deploying a remote package and "--insecure" is not provided
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ zarf tools registry catalog REGISTRY [flags]

# list the repos internal to Zarf
$ zarf tools registry catalog

# list the repos for reg.example.com
$ zarf tools registry catalog reg.example.com

Expand Down
16 changes: 0 additions & 16 deletions docs/3-create-a-zarf-package/4-zarf-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -765,22 +765,6 @@ Must be one of:
</blockquote>
</details>

<details>
<summary>
<strong> <a name="components_items_cosignKeyPath"></a>cosignKeyPath</strong>
</summary>
&nbsp;
<blockquote>

**Description:** Specify a path to a public key to validate signed online resources

| | |
| -------- | -------- |
| **Type** | `string` |

</blockquote>
</details>

<details open>
<summary>
<strong> <a name="components_items_import"></a>import</strong>
Expand Down
1 change: 1 addition & 0 deletions examples/dos-games/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ kind: ZarfPackageConfig
metadata:
name: dos-games
description: Simple example to load classic DOS games into K8s in the airgap
version: 1.0.0

components:
- name: baseline
Expand Down
2 changes: 1 addition & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ var cosignPublicKey string

func main() {
config.UIAssets = assets
config.SGetPublicKey = cosignPublicKey
config.CosignPublicKey = cosignPublicKey
cmd.Execute()
}
4 changes: 2 additions & 2 deletions src/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ var (
// SkipLogFile is a flag to skip logging to a file
SkipLogFile bool

SGetPublicKey string
UIAssets embed.FS
CosignPublicKey string
UIAssets embed.FS

// Timestamp of when the CLI was started
operationStartTime = time.Now().Unix()
Expand Down
9 changes: 7 additions & 2 deletions src/config/lang/english.go
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ const (
CmdPackageDeployFlagSet = "Specify deployment variables to set on the command line (KEY=value)"
CmdPackageDeployFlagComponents = "Comma-separated list of components to install. Adding this flag will skip the init prompts for which components to install"
CmdPackageDeployFlagShasum = "Shasum of the package to deploy. Required if deploying a remote package and \"--insecure\" is not provided"
CmdPackageDeployFlagSget = "Path to public sget key file for remote packages signed via cosign"
CmdPackageDeployFlagSget = "[Deprecated] Path to public sget key file for remote packages signed via cosign. This flag will be removed in v0.31.0 please use the --key flag instead."
CmdPackageDeployFlagPublicKey = "Path to public key file for validating signed packages"
CmdPackageDeployValidateArchitectureErr = "this package architecture is %s, but the target cluster has the %s architecture. These architectures must be the same"
CmdPackageDeployErr = "Failed to deploy package: %s"
Expand Down Expand Up @@ -328,7 +328,7 @@ const (
CmdToolsRegistryCatalogExample = `
# list the repos internal to Zarf
$ zarf tools registry catalog

# list the repos for reg.example.com
$ zarf tools registry catalog reg.example.com
`
Expand Down Expand Up @@ -515,3 +515,8 @@ var (
ErrUnableToCheckArch = errors.New("unable to get the configured cluster's architecture")
ErrInterrupt = errors.New("Failed due to interrupt")
)

// Collection of reusable warn messages.
var (
WarnSGetDeprecation = "Using sget to download resources is being deprecated and will removed in the v0.31.0 release of Zarf. Please publish the packages as OCI artifacts instead."
)
4 changes: 3 additions & 1 deletion src/pkg/packager/network.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"strings"

"github.com/defenseunicorns/zarf/src/config"
"github.com/defenseunicorns/zarf/src/config/lang"
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/utils"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
Expand Down Expand Up @@ -103,6 +104,7 @@ func (p *Packager) handlePackagePath() (partialPaths []string, err error) {

func (p *Packager) handleSgetPackage() error {
message.Debug("packager.handleSgetPackage()")
message.Warn(lang.WarnSGetDeprecation)

opts := p.cfg.DeployOpts

Expand All @@ -119,7 +121,7 @@ func (p *Packager) handleSgetPackage() error {

// If this is a DefenseUnicorns package, use an internal sget public key
if strings.HasPrefix(opts.PackagePath, fmt.Sprintf("%s://defenseunicorns", utils.SGETURLScheme)) {
os.Setenv("DU_SGET_KEY", config.SGetPublicKey)
os.Setenv("DU_SGET_KEY", config.CosignPublicKey)
p.cfg.DeployOpts.SGetKeyPath = "env://DU_SGET_KEY"
}

Expand Down
5 changes: 4 additions & 1 deletion src/pkg/utils/sget.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"strings"

"github.com/defenseunicorns/zarf/src/config"
"github.com/defenseunicorns/zarf/src/config/lang"
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
Expand All @@ -36,9 +37,11 @@ import (

// Sget performs a cosign signature verification on a given image using the specified public key.
func Sget(ctx context.Context, image, key string, out io.Writer) error {
message.Warnf(lang.WarnSGetDeprecation)

// If this is a DefenseUnicorns package, use an internal sget public key
if strings.HasPrefix(image, fmt.Sprintf("%s://defenseunicorns", SGETURLScheme)) {
os.Setenv("DU_SGET_KEY", config.SGetPublicKey)
os.Setenv("DU_SGET_KEY", config.CosignPublicKey)
key = "env://DU_SGET_KEY"
}

Expand Down
2 changes: 1 addition & 1 deletion src/test/e2e/06_create_sbom_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ func TestCreateSBOM(t *testing.T) {
cachePath := filepath.Join(tmpdir, ".cache-location")
sbomPath := filepath.Join(tmpdir, ".sbom-location")

pkgName := fmt.Sprintf("zarf-package-dos-games-%s.tar.zst", e2e.Arch)
pkgName := fmt.Sprintf("zarf-package-dos-games-%s-1.0.0.tar.zst", e2e.Arch)

stdOut, stdErr, err := e2e.Zarf("package", "create", "examples/dos-games", "--zarf-cache", cachePath, "--sbom-out", sbomPath, "--confirm")
require.NoError(t, err, stdOut, stdErr)
Expand Down
4 changes: 2 additions & 2 deletions src/test/e2e/25_helm_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ func testHelmEscaping(t *testing.T) {
func testHelmUninstallRollback(t *testing.T) {
t.Log("E2E: Helm Uninstall and Rollback")

goodPath := fmt.Sprintf("build/zarf-package-dos-games-%s.tar.zst", e2e.Arch)
goodPath := fmt.Sprintf("build/zarf-package-dos-games-%s-1.0.0.tar.zst", e2e.Arch)
evilPath := fmt.Sprintf("zarf-package-dos-games-%s.tar.zst", e2e.Arch)

// Create the evil package (with the bad configmap).
Expand Down Expand Up @@ -128,7 +128,7 @@ func testHelmUninstallRollback(t *testing.T) {
func testHelmAdoption(t *testing.T) {
t.Log("E2E: Helm Adopt a Deployment")

packagePath := fmt.Sprintf("build/zarf-package-dos-games-%s.tar.zst", e2e.Arch)
packagePath := fmt.Sprintf("build/zarf-package-dos-games-%s-1.0.0.tar.zst", e2e.Arch)
deploymentManifest := "src/test/packages/25-manifest-adoption/deployment.yaml"

// Deploy dos-games manually into the cluster without Zarf
Expand Down
2 changes: 1 addition & 1 deletion src/test/e2e/26_simple_packages_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ func TestDosGames(t *testing.T) {
t.Log("E2E: Dos games")
e2e.SetupWithCluster(t)

path := fmt.Sprintf("build/zarf-package-dos-games-%s.tar.zst", e2e.Arch)
path := fmt.Sprintf("build/zarf-package-dos-games-%s-1.0.0.tar.zst", e2e.Arch)

// Deploy the game
stdOut, stdErr, err := e2e.Zarf("package", "deploy", path, "--confirm")
Expand Down
2 changes: 1 addition & 1 deletion src/test/e2e/29_mismatched_architectures_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ func TestMismatchedArchitectures(t *testing.T) {

var (
mismatchedArch = e2e.GetMismatchedArch()
mismatchedGamesPackage = fmt.Sprintf("zarf-package-dos-games-%s.tar.zst", mismatchedArch)
mismatchedGamesPackage = fmt.Sprintf("zarf-package-dos-games-%s-1.0.0.tar.zst", mismatchedArch)
expectedErrorMessage = fmt.Sprintf("this package architecture is %s", mismatchedArch)
)

Expand Down
2 changes: 1 addition & 1 deletion src/test/e2e/32_checksum_and_signature_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ func TestChecksumAndSignature(t *testing.T) {
e2e.SetupWithCluster(t)

testPackageDirPath := "examples/dos-games"
pkgName := fmt.Sprintf("zarf-package-dos-games-%s.tar.zst", e2e.Arch)
pkgName := fmt.Sprintf("zarf-package-dos-games-%s-1.0.0.tar.zst", e2e.Arch)
privateKeyFlag := "--key=src/test/packages/zarf-test.prv-key"
publicKeyFlag := "--key=src/test/packages/zarf-test.pub"

Expand Down
2 changes: 1 addition & 1 deletion src/test/e2e/50_oci_package_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func (suite *RegistryClientTestSuite) Test_0_Publish() {
suite.Contains(stdErr, "Published "+ref)

// Publish w/ package missing `metadata.version` field.
example = filepath.Join(suite.PackagesDir, fmt.Sprintf("zarf-package-dos-games-%s.tar.zst", e2e.Arch))
example = filepath.Join(suite.PackagesDir, fmt.Sprintf("zarf-package-component-actions-%s.tar.zst", e2e.Arch))
_, stdErr, err = e2e.Zarf("package", "publish", example, "oci://"+ref, "--insecure")
suite.Error(err, stdErr)

Expand Down
2 changes: 1 addition & 1 deletion src/types/component.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ type ZarfComponent struct {
Group string `json:"group,omitempty" jsonschema:"description=Create a user selector field based on all components in the same group"`

//Path to cosign publickey for signed online resources
CosignKeyPath string `json:"cosignKeyPath,omitempty" jsonschema:"description=Specify a path to a public key to validate signed online resources"`
CosignKeyPath string `json:"cosignKeyPath,omitempty" jsonschema:"description=[Deprecated] Specify a path to a public key to validate signed online resources"`
YrrepNoj marked this conversation as resolved.
Show resolved Hide resolved

// Import refers to another zarf.yaml package component.
Import ZarfComponentImport `json:"import,omitempty" jsonschema:"description=Import a component from another Zarf package"`
Expand Down
2 changes: 1 addition & 1 deletion src/ui/lib/api-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ export interface ZarfComponent {
*/
charts?: ZarfChart[];
/**
* Specify a path to a public key to validate signed online resources
* [Deprecated] Specify a path to a public key to validate signed online resources
*/
cosignKeyPath?: string;
/**
Expand Down
2 changes: 1 addition & 1 deletion zarf.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@
},
"cosignKeyPath": {
"type": "string",
"description": "Specify a path to a public key to validate signed online resources"
"description": "[Deprecated] Specify a path to a public key to validate signed online resources"
},
"import": {
"$schema": "http://json-schema.org/draft-04/schema#",
Expand Down