Basic composability, starting with Flux #208

RothAndrew · 2021-12-15T00:48:25Z

Add support for "composable zarf packages". This will require syntax change to the zarf package definitions as well as code changes to support.

Tasks:

Define zarf.yaml specification change, should be agreed upon by the team overall
Require component-to-component mapping to keep things sane
Syntax on the producer side (package authors) should clearly state this is an import/inclusion of other package component data
Should be completely transparent on the consumer (package deployer) side. The component tree should be flattened during zarf package create so that no changes are needed for zarf package deploy.
There should be no makefile or other dependencies for package producers or consumers, it should all be done natively through the zarf cli
This feature can be limited in scope to local packages/components only for now and a new issue added for remote package/component support
E2E tests should be created to demonstrate a basic composite package

The text was updated successfully, but these errors were encountered:

…ng with flux. Consider extraction into seperately maintained repos for composable packages in near future.

…h package-deploy and have the flux resources populate within kind cluster.

mike-winberry · 2022-02-02T18:42:45Z

Need to discuss the initial hosting of composable packages such as flux. Right now it is working as an external package and I am going to create some examples and documentation on using it, but without a current way to host the package (external repo, s3 etc...) the package will have to be created locally.

jeff-mccoy · 2022-02-02T18:45:06Z

One thing I've been thinking about for that is we try to require shasums for other remote resources...until we have some type of singling or attestation, we may need to stick with local only. Realize that's not ideal from UX, but I think we have a responsibility to protect producers/consumers too.

RothAndrew · 2022-02-02T19:09:28Z

Where do we want to put pre-built packages? I see this as the beginnings of the "Zarf Appstore". What do you guys think about either:

A new repo just for this Flux package
- Would be less DRY but easier to have separate release lifecycles for different pre-built packages
A new repo that will have all pre-built packages
- More DRY, but more "monorepo" which is a pattern I'm not a huge fan of

…on Mac. Updated prrtrwuidid. Change build-package in Makefile to package

mike-winberry · 2022-02-04T17:00:12Z

So I have updated the way it works in the packages folder at the zarf root for now. I think that going with the manual creation and hosting in a single repo may be a good way for now until we are able to host and update the packages with sha256. Check out the readme and the Makefile. I made it easy to build specific packages and add them to the components in a zarf file. At least for mvp behavior.

…ng with flux. Consider extraction into seperately maintained repos for composable packages in near future.

…h package-deploy and have the flux resources populate within kind cluster.

…on Mac. Updated prrtrwuidid. Change build-package in Makefile to package

mike-winberry · 2022-02-10T15:05:27Z

Moved to https://github.com/defenseunicorns/zarf-package-flux

jeff-mccoy · 2022-02-11T22:20:41Z

Met with @mike-winberry and we rewrote the issue to not use makefile and instead be a code change solution.

…e private config. package/create.Create: Add logic to expand locally imported components within the parent build config. types: Add Import field to Zarf.Component as expirement (expected to change).

…ponents and nested import components. UPDATE types.go: Added ZarfImport type with a path variable as the associated type with ZarfComponent.Import field (this sets up sha verification additions in the future, still need team imput). UPDATE Packager/Common.go: pulled confirming optional component logic out of getValidComponents to play with composed package validation (need to validate expected behavior). UPDATE Packager/Create.go: Replaced the initial compose logic with call to GetComposedAssets(). ADD examples/compose-example: a simple composition example for explicit example.

…er needed. UPDATE packager/compose: removed the logic working with SeedImages as it is not necessary for composition. UPDATE packager/create.go: seperate the GetComposedAssets call from the seed images. Set seedImages variable using the config.GetSeedImages() method.

…he composed packages flux and games are properly deployed together. UPDATE examples/Makefile with the package-example-composability command and added that command to the package-examples command. RENAME examples/composable-example -> examples/composable-packages

…ethod for use with compose.go. UPDATE packager/compose: add the hasValidSubPackage method and replaced the hasSubPackage calls. Updated the hasSubPackage method to validate only that the Component.Import field exists. Clean up index names in prepComponentToCompose.

…ackages. ADD examples/composable-packages/README: with instructions on running doom using composition from the games example zarf.yaml. UPDATE examples/composable-packages/zarf.yaml: removed the flux package as the game is a more clear example and still shows the same composability without the need of flux since the games package does not rely on flux

…ge method when pulling in imported components. Updated shouldAddImportedPackage method to no longer prompt if --confirm flag is passed

…sed package when the user chooses to not import that composed packages.

… due to remove of flux composed component in the composability example.

…s. Update .github/workflows/test*: add package-compose-example to make packages stage. UPDATE packager/compose: rename GetComposedAssets to GetComposedComponents UPDATE reference in packager/create. refactored and rename hasValidSubPackage to validateOrBail. Rename shouldAddImportedPackage to shouldComposePackage. Extracted component required logic to componentConfirmedForInclusion. UPDATE e2e/e2e_composability_test: to work with new testing strategy.

* #208. config: Add SetComponents method to update the components in the private config. package/create.Create: Add logic to expand locally imported components within the parent build config. types: Add Import field to Zarf.Component as expirement (expected to change). * #208. ADD packager/compose.go: Created compose logic for imported components and nested import components. UPDATE types.go: Added ZarfImport type with a path variable as the associated type with ZarfComponent.Import field (this sets up sha verification additions in the future, still need team imput). UPDATE Packager/Common.go: pulled confirming optional component logic out of getValidComponents to play with composed package validation (need to validate expected behavior). UPDATE Packager/Create.go: Replaced the initial compose logic with call to GetComposedAssets(). ADD examples/compose-example: a simple composition example for explicit example. * #208. UPDATE cli/config: Delete SetSeedImages method as it is no longer needed. UPDATE packager/compose: removed the logic working with SeedImages as it is not necessary for composition. UPDATE packager/create.go: seperate the GetComposedAssets call from the seed images. Set seedImages variable using the config.GetSeedImages() method. * #208. ADD e2e_composability_test: set up basic tests to ensure that the composed packages flux and games are properly deployed together. UPDATE examples/Makefile with the package-example-composability command and added that command to the package-examples command. RENAME examples/composable-example -> examples/composable-packages * #208, #351. UPDATE packager/validate: add the ValidateImportPackage method for use with compose.go. UPDATE packager/compose: add the hasValidSubPackage method and replaced the hasSubPackage calls. Updated the hasSubPackage method to validate only that the Component.Import field exists. Clean up index names in prepComponentToCompose. * #208, #351. UPDATE docer/components: Added new section on composing packages. ADD examples/composable-packages/README: with instructions on running doom using composition from the games example zarf.yaml. UPDATE examples/composable-packages/zarf.yaml: removed the flux package as the game is a more clear example and still shows the same composability without the need of flux since the games package does not rely on flux * #208, #351. UPDATE packager/compose: added the shouldAddImportedPackage method when pulling in imported components. Updated shouldAddImportedPackage method to no longer prompt if --confirm flag is passed * #208, #351. FIX packager/compose: to no longer add the optional composed package when the user chooses to not import that composed packages. * #208. UPDATE e2e_composability_test: removed the tests verifying flux due to remove of flux composed component in the composability example. * #208, #351. UPDATE Makefile: add package-example-compose for e2e tests. Update .github/workflows/test*: add package-compose-example to make packages stage. UPDATE packager/compose: rename GetComposedAssets to GetComposedComponents UPDATE reference in packager/create. refactored and rename hasValidSubPackage to validateOrBail. Rename shouldAddImportedPackage to shouldComposePackage. Extracted component required logic to componentConfirmedForInclusion. UPDATE e2e/e2e_composability_test: to work with new testing strategy.

tukaanimhaka · 2024-01-26T17:31:52Z

Maybe an example on how to use composable packages would be nice to have if not already added.

RothAndrew changed the title ~~Flux should have its own package instead of deploying it in the gitops examples~~ Basic composability, starting with Flux Dec 15, 2021

RothAndrew added this to Zarf Project Board Jan 10, 2022

RothAndrew moved this to New Requests in Zarf Project Board Jan 10, 2022

RothAndrew moved this from New Requests to Ready to Start in Zarf Project Board Jan 10, 2022

mike-winberry self-assigned this Jan 31, 2022

mike-winberry moved this from Ready to Start to Doing Now in Zarf Project Board Jan 31, 2022

mike-winberry added a commit that referenced this issue Jan 31, 2022

#208. set up initial folder structure for composable packages. Starti…

6a1f80d

…ng with flux. Consider extraction into seperately maintained repos for composable packages in near future.

mike-winberry added a commit that referenced this issue Jan 31, 2022

#208. Initial flux package creation. Currently was able to deploy wit…

c5928ae

…h package-deploy and have the flux resources populate within kind cluster.

mike-winberry linked a pull request Feb 2, 2022 that will close this issue

208 basic composability #250

Closed

mike-winberry added the needs-more-info label Feb 2, 2022

mike-winberry added a commit that referenced this issue Feb 3, 2022

#208. Cleaned up instructions and Makefile. Fixed zarf missing error …

c5c04a2

…on Mac. Updated prrtrwuidid. Change build-package in Makefile to package

mike-winberry added a commit that referenced this issue Feb 4, 2022

#208. Fix makefile example and .PHONY build-package changed to package

5c5ff56

RothAndrew mentioned this issue Feb 4, 2022

Real package composability #212

Closed

mike-winberry added a commit that referenced this issue Feb 9, 2022

#208. set up initial folder structure for composable packages. Starti…

3c4bd2c

…ng with flux. Consider extraction into seperately maintained repos for composable packages in near future.

mike-winberry added a commit that referenced this issue Feb 9, 2022

#208. Initial flux package creation. Currently was able to deploy wit…

26252cc

…h package-deploy and have the flux resources populate within kind cluster.

mike-winberry added a commit that referenced this issue Feb 9, 2022

#208. Cleaned up instructions and Makefile. Fixed zarf missing error …

86bf65b

…on Mac. Updated prrtrwuidid. Change build-package in Makefile to package

mike-winberry added a commit that referenced this issue Feb 9, 2022

#208. Fix makefile example and .PHONY build-package changed to package

c731686

mike-winberry added a commit that referenced this issue Feb 9, 2022

#208. #250. Attempting reopen

9375187

mike-winberry added enhancement ✨ New feature or request and removed needs-more-info labels Feb 22, 2022

mike-winberry added a commit that referenced this issue Mar 9, 2022

#208, #351. UPDATE packager/compose: added the shouldAddImportedPacka…

1731cea

…ge method when pulling in imported components. Updated shouldAddImportedPackage method to no longer prompt if --confirm flag is passed

mike-winberry added a commit that referenced this issue Mar 9, 2022

#208, #351. FIX packager/compose: to no longer add the optional compo…

b1c3ade

…sed package when the user chooses to not import that composed packages.

mike-winberry added a commit that referenced this issue Mar 9, 2022

#208. UPDATE e2e_composability_test: removed the tests verifying flux…

01a7b22

… due to remove of flux composed component in the composability example.

mike-winberry closed this as completed in #351 Mar 10, 2022

Repository owner moved this from Doing Now to Done in Zarf Project Board Mar 10, 2022

Racer159 moved this to Done in Zarf Project Board Apr 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basic composability, starting with Flux #208

Basic composability, starting with Flux #208

RothAndrew commented Dec 15, 2021 •

edited by mike-winberry

Loading

mike-winberry commented Feb 2, 2022

jeff-mccoy commented Feb 2, 2022

RothAndrew commented Feb 2, 2022

mike-winberry commented Feb 4, 2022

mike-winberry commented Feb 10, 2022

jeff-mccoy commented Feb 11, 2022

tukaanimhaka commented Jan 26, 2024 •

edited

Loading

Basic composability, starting with Flux #208

Basic composability, starting with Flux #208

Comments

RothAndrew commented Dec 15, 2021 • edited by mike-winberry Loading

mike-winberry commented Feb 2, 2022

jeff-mccoy commented Feb 2, 2022

RothAndrew commented Feb 2, 2022

mike-winberry commented Feb 4, 2022

mike-winberry commented Feb 10, 2022

jeff-mccoy commented Feb 11, 2022

tukaanimhaka commented Jan 26, 2024 • edited Loading

RothAndrew commented Dec 15, 2021 •

edited by mike-winberry

Loading

tukaanimhaka commented Jan 26, 2024 •

edited

Loading