diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index cc68f6dc2..89ef65ffd 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -26,7 +26,7 @@ jobs: - name: Install pack uses: buildpacks/github-actions/setup-pack@v5.2.0 with: - pack-version: '0.30.0-rc1' + pack-version: '0.30.0-rc1' # FIXME: update to 0.30.0 when available - name: Test run: make test env: diff --git a/content/docs/extension-guide/create-extension/_index.md b/content/docs/extension-guide/create-extension/_index.md index 4a9144184..2ec44ee1d 100644 --- a/content/docs/extension-guide/create-extension/_index.md +++ b/content/docs/extension-guide/create-extension/_index.md @@ -24,7 +24,7 @@ This is a step-by-step tutorial for creating and using CNB image extensions. - [See a build that requires base image extension in order to succeed](/docs/extension-guide/create-extension/why-dockerfiles) - [Building blocks of an extension](/docs/extension-guide/create-extension/building-blocks-extension) - [Generating a build.Dockerfile for your application](/docs/extension-guide/create-extension/build-dockerfile) -- [Generating a run.Dockerfile for your application](/docs/extension-guide/create-extension/run-dockerfile) +- [Generating a run.Dockerfile for your application](/docs/extension-guide/create-extension/run-dockerfile-switch) --- diff --git a/content/docs/extension-guide/create-extension/build-dockerfile.md b/content/docs/extension-guide/create-extension/build-dockerfile.md index a568c178d..b92761994 100644 --- a/content/docs/extension-guide/create-extension/build-dockerfile.md +++ b/content/docs/extension-guide/create-extension/build-dockerfile.md @@ -8,6 +8,9 @@ aliases = [ +Builder images can be kept lean if image extensions are used to dynamically install the needed dependencies +for the current application. + ### Examine `vim` extension #### detect @@ -28,7 +31,9 @@ cat $PWD/samples/extensions/vim/bin/generate The extension generates a `build.Dockerfile` that installs `vim` on the builder image. -### Re-build the application image +### Configure the `hello-extensions` buildpack to require `vim` + +Set the `BP_REQUIRES` build-time environment variable to configure the `hello-extensions` buildpack to require `vim` (review the `./bin/detect` script to see why this works). ``` @@ -47,6 +52,7 @@ Note that `--network host` is necessary when publishing to a local registry. You should see: ``` +... [detector] ======== Results ======== [detector] pass: samples/vim@0.0.1 [detector] pass: samples/hello-extensions@0.0.1 @@ -55,12 +61,12 @@ You should see: [detector] samples/hello-extensions 0.0.1 [detector] Running generate for extension samples/vim@0.0.1 ... -[extender] Found build Dockerfile for extension 'samples/vim' -[extender] Applying the Dockerfile at /layers/generated/build/samples_vim/Dockerfile... +[extender (build)] Found build Dockerfile for extension 'samples/vim' +[extender (build)] Applying the Dockerfile at /layers/generated/build/samples_vim/Dockerfile... ... -[extender] Running build command -[extender] ---> Hello Extensions Buildpack -[extender] vim v1.8.0 (c) 1996 - 2018 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro +[extender (build)] Running build command +[extender (build)] ---> Hello Extensions Buildpack +[extender (build)] VIM - Vi IMproved 9.0 (2022 Jun 28, compiled May 19 2023 16:28:36) ... Successfully built image hello-extensions ``` @@ -85,5 +91,4 @@ Let's take a look at how the `samples/curl` extension fixes the error by switchi --- -Next Step - +Next Step diff --git a/content/docs/extension-guide/create-extension/building-blocks-extension.md b/content/docs/extension-guide/create-extension/building-blocks-extension.md index 04a749c38..3cd2322b4 100644 --- a/content/docs/extension-guide/create-extension/building-blocks-extension.md +++ b/content/docs/extension-guide/create-extension/building-blocks-extension.md @@ -12,10 +12,10 @@ aliases = [ ```bash -vim --help +tree $PWD/samples/extensions/vim ``` -(That's right, we're using the very tool we will later be installing!) You should see something akin to the following: +You should see something akin to the following: ``` . @@ -38,11 +38,6 @@ vim --help * Only a limited set of Dockerfile instructions is supported - consult the [spec](https://github.com/buildpacks/spec/blob/main/image_extension.md) for further details. - * In the [initial implementation](/docs/features/dockerfiles#phased-approach), `run.Dockerfile` instructions are - limited to a single `FROM` instruction (effectively, it is only possible to switch the run-time base image to a - pre-created image i.e., no dynamic image modification is allowed). Consult - the [spec](https://github.com/buildpacks/spec/blob/main/image_extension.md) - for further details. We'll take a closer look at the executables for the `vim` extension in the next step. diff --git a/content/docs/extension-guide/create-extension/run-dockerfile-extend.md b/content/docs/extension-guide/create-extension/run-dockerfile-extend.md new file mode 100644 index 000000000..fce8b3b8c --- /dev/null +++ b/content/docs/extension-guide/create-extension/run-dockerfile-extend.md @@ -0,0 +1,107 @@ ++++ +title="Generating a run.Dockerfile that extends the runtime base image" +weight=406 ++++ + + + +Run images can be kept lean if image extensions are used to dynamically install the needed dependencies +for the current application. + +### Examine `cowsay` extension + +#### detect + + +```bash +cat $PWD/samples/extensions/cowsay/bin/detect +``` + +The extension always detects (because its exit code is `0`) and provides a dependency called `cowsay`. + +#### generate + + +```bash +cat $PWD/samples/extensions/cowsay/bin/generate +``` + +The extension generates a `run.Dockerfile` that installs `cowsay` on the current run image. + +### Configure the `hello-extensions` buildpack to require `cowsay` + +Set the `BP_REQUIRES` build-time environment variable to configure the `hello-extensions` buildpack to require both `vim` and `curl` (review the `./bin/detect` script to see why this works). + + +```bash +pack build hello-extensions \ + --builder localhost:5000/extensions-builder \ + --env BP_EXT_DEMO=1 \ + --env BP_REQUIRES=vim,curl,cowsay \ + --path $PWD/samples/apps/java-maven \ + --pull-policy always \ + --network host \ + --verbose +``` + +Note that `--network host` is necessary when publishing to a local registry. + +You should see: + +``` +... +[detector] ======== Results ======== +[detector] pass: samples/vim@0.0.1 +[detector] pass: samples/curl@0.0.1 +[detector] pass: samples/cowsay@0.0.1 +[detector] pass: samples/hello-extensions@0.0.1 +[detector] Resolving plan... (try #1) +[detector] samples/vim 0.0.1 +[detector] samples/curl 0.0.1 +[detector] samples/cowsay 0.0.1 +[detector] samples/hello-extensions 0.0.1 +[detector] Running generate for extension samples/vim@0.0.1 +... +[detector] Running generate for extension samples/curl@0.0.1 +... +[detector] Running generate for extension samples/cowsay@0.0.1 +... +[detector] Found a run.Dockerfile from extension 'samples/curl' setting run image to 'localhost:5000/run-image-curl' +... +[extender (build)] Found build Dockerfile for extension 'samples/vim' +[extender (build)] Applying Dockerfile at /layers/generated/build/samples_vim/Dockerfile... +[extender (run)] Found run Dockerfile for extension 'samples/curl' +[extender (run)] Found run Dockerfile for extension 'samples/cowsay' +[extender (run)] Applying Dockerfile at /layers/generated/run/samples_curl/Dockerfile... +... +[extender (run)] Applying Dockerfile at /layers/generated/run/samples_cowsay/Dockerfile +... +[extender (build)] Running build command +[extender (build)] ---> Hello Extensions Buildpack +[extender (build)] VIM - Vi IMproved 9.0 (2022 Jun 28, compiled May 19 2023 16:28:36) +... +Successfully built image hello-extensions +``` + +Note: build image extension and run image extension are done in parallel, +so the log lines for those phases may print in a different order from that shown above. + +### See the image run successfully + + +```bash +docker run --rm --entrypoint cowsay hello-extensions +``` + +You should see something akin to: + +``` + ________ +< MOOOO! > + -------- + \ ^__^ + \ (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` diff --git a/content/docs/extension-guide/create-extension/run-dockerfile.md b/content/docs/extension-guide/create-extension/run-dockerfile-switch.md similarity index 55% rename from content/docs/extension-guide/create-extension/run-dockerfile.md rename to content/docs/extension-guide/create-extension/run-dockerfile-switch.md index b7e94f813..9c36b3ee3 100644 --- a/content/docs/extension-guide/create-extension/run-dockerfile.md +++ b/content/docs/extension-guide/create-extension/run-dockerfile-switch.md @@ -1,5 +1,5 @@ +++ -title="Generating a run.Dockerfile" +title="Generating a run.Dockerfile that switches the runtime base image" weight=405 aliases = [ "/docs/extension-author-guide/run-dockerfile/" @@ -8,6 +8,10 @@ aliases = [ +Platforms can have several run images available, each tailored to a specific language family - thus limiting the number +of installed dependencies for each image to the minimum necessary to support the targeted language. Image extensions +can be used to switch the run image to that most appropriate for the current application. + ### Examine `curl` extension #### detect @@ -26,7 +30,7 @@ The extension always detects (because its exit code is `0`) and provides a depen cat $PWD/samples/extensions/curl/bin/generate ``` -The extension generates a `run.Dockerfile` that switches the run image to reference `run-image-curl`. +The extension generates a `run.Dockerfile` that switches the run image to reference `localhost:5000/run-image-curl`. ### Build a run image for `curl` extension to use @@ -46,9 +50,13 @@ Build the run image: docker build \ --file $PWD/samples/stacks/alpine/run/curl.Dockerfile \ --tag localhost:5000/run-image-curl . + +docker push localhost:5000/run-image-curl ``` -### Re-build the application image +### Configure the `hello-extensions` buildpack to require `curl` + +Set the `BP_REQUIRES` build-time environment variable to configure the `hello-extensions` buildpack to require both `vim` and `curl` (review the `./bin/detect` script to see why this works). ```bash @@ -70,8 +78,11 @@ You should see: [detector] ======== Results ======== [detector] pass: samples/vim@0.0.1 [detector] pass: samples/curl@0.0.1 +[detector] pass: samples/cowsay@0.0.1 [detector] pass: samples/hello-extensions@0.0.1 [detector] Resolving plan... (try #1) +[detector] skip: samples/cowsay@0.0.1 provides unused cowsay +[detector] 3 of 4 buildpacks participating [detector] samples/vim 0.0.1 [detector] samples/curl 0.0.1 [detector] samples/hello-extensions 0.0.1 @@ -80,14 +91,14 @@ You should see: [detector] Running generate for extension samples/curl@0.0.1 ... [detector] Checking for new run image -[detector] Found a run.Dockerfile configuring image 'run-image-curl' from extension with id 'samples/curl' +[detector] Found a run.Dockerfile from extension 'samples/curl' setting run image to 'localhost:5000/run-image-curl' ... -[extender] Found build Dockerfile for extension 'samples/vim' -[extender] Applying the Dockerfile at /layers/generated/build/samples_vim/Dockerfile... +[extender (build)] Found build Dockerfile for extension 'samples/vim' +[extender (build)] Applying the Dockerfile at /layers/generated/build/samples_vim/Dockerfile... ... -[extender] Running build command -[extender] ---> Hello Extensions Buildpack -[extender] vim v1.8.0 (c) 1996 - 2018 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro +[extender (build)] Running build command +[extender (build)] ---> Hello Extensions Buildpack +[extender (build)] VIM - Vi IMproved 9.0 (2022 Jun 28, compiled May 19 2023 16:28:36) ... Successfully built image hello-extensions ``` @@ -108,20 +119,31 @@ curl 7.85.0-DEV (x86_64-pc-linux-musl) ... more stuff here ... What happened: now that `hello-extensions` requires both `vim` and `curl` in its build plan, both extensions are included in the build and provide the needed dependencies for build and launch, respectively * The `vim` extension installs `vim` at build time, as before -* The `curl` extension switches the run image to `run-image-curl`, which has `curl` installed +* The `curl` extension switches the run image to `localhost:5000/run-image-curl`, which has `curl` installed Now our `curl` process can succeed! -## What's next? +### Next steps -The `vim` and `curl` examples are very simple, but we can unlock powerful new features with this functionality. +Our `curl` process succeeded, but there is another process type defined on our image: -Platforms could have several run images available, each tailored to a specific language family, thus limiting the number -of installed dependencies for each image to the minimum necessary to support the targeted language. Image extensions -could be used to switch the run image to that most appropriate for the current application. +``` +docker run --rm --entrypoint cowsay hello-extensions +``` + +You should see: + +``` +ERROR: failed to launch: path lookup: exec: "cowsay": executable file not found in $PATH +``` + +Our run image, `localhost:5000/run-image-curl`, has `curl` installed, but it doesn't have `cowsay`. + +In general, we may not always have a preconfigured run image available with all the needed dependencies for the current application. +Luckily, we can also use image extensions to dynamically install runtime dependencies at build time. Let's look at that next. -Similarly, builder images could be kept lean if image extensions are used to dynamically install the needed dependencies -for each application. + +--- -In the future, both run image switching and run image modification will be supported, opening the door to other use -cases. Consult the [RFC](https://github.com/buildpacks/rfcs/pull/173) for further information. +Next Step + diff --git a/content/docs/extension-guide/create-extension/setup-local-environment.md b/content/docs/extension-guide/create-extension/setup-local-environment.md index 0931d364a..74daf7fb1 100644 --- a/content/docs/extension-guide/create-extension/setup-local-environment.md +++ b/content/docs/extension-guide/create-extension/setup-local-environment.md @@ -55,7 +55,7 @@ cd pack version ``` -The version should be at least `0.28.0` +The version should be at least `0.30.0` ### Update pack configuration diff --git a/content/docs/extension-guide/create-extension/why-dockerfiles.md b/content/docs/extension-guide/create-extension/why-dockerfiles.md index 6ef94e7b1..ef8d02437 100644 --- a/content/docs/extension-guide/create-extension/why-dockerfiles.md +++ b/content/docs/extension-guide/create-extension/why-dockerfiles.md @@ -19,7 +19,9 @@ Let's see a build that requires base image extension in order to succeed. cat $PWD/samples/buildpacks/hello-extensions/bin/detect ``` -The buildpack always detects (because its exit code is `0`) but doesn't require any dependencies (as the output build plan is empty). +The buildpack opts-out of the build (exits with non-zero code) unless the `BP_EXT_DEMO` environment variable is set. + +If the `BP_EXT_DEMO` environment variable is set, the buildpack detects (exits with code `0`), but doesn't require any dependencies through a build plan unless the `BP_REQUIRES` environment variable is set. #### build @@ -28,7 +30,7 @@ The buildpack always detects (because its exit code is `0`) but doesn't require cat $PWD/samples/buildpacks/hello-extensions/bin/build ``` -The buildpack tries to use `tree` at build-time, and defines a launch process called `curl` that runs `curl --version` at runtime. +The buildpack tries to use `vim` at build-time, and defines a launch process called `curl` that runs `curl --version` at runtime. ### Create a builder with extensions and publish it @@ -72,28 +74,33 @@ Note that `--network host` is necessary when publishing to a local registry. You should see: ``` +... [detector] ======== Results ======== -[detector] pass: samples/tree@0.0.1 +[detector] pass: samples/vim@0.0.1 +[detector] pass: samples/curl@0.0.1 +[detector] pass: samples/cowsay@0.0.1 [detector] pass: samples/hello-extensions@0.0.1 [detector] Resolving plan... (try #1) -[detector] skip: samples/tree@0.0.1 provides unused tree -[detector] 1 of 2 buildpacks participating +[detector] skip: samples/vim@0.0.1 provides unused vim +[detector] skip: samples/curl@0.0.1 provides unused curl +[detector] skip: samples/cowsay@0.0.1 provides unused cowsay +[detector] 1 of 4 buildpacks participating [detector] samples/hello-extensions 0.0.1 ... -[extender] Running build command -[extender] ---> Hello Extensions Buildpack -[extender] /cnb/buildpacks/samples_hello-extensions/0.0.1/bin/build: line 6: tree: command not found -[extender] ERROR: failed to build: exit status 127 +[extender (build)] Running build command +[extender (build)] ---> Hello Extensions Buildpack +[extender (build)] /cnb/buildpacks/samples_hello-extensions/0.0.1/bin/build: line 6: vim: command not found +[extender (build)] ERROR: failed to build: exit status 127 ``` -What happened: our builder doesn't have `tree` installed, so the `hello-extensions` buildpack failed to build (as it -tries to run `tree --version` in its `./bin/build` script). +What happened: our builder doesn't have `vim` installed, so the `hello-extensions` buildpack failed to build (as it +tries to run `vim --version` in its `./bin/build` script). -Even though there is a `samples/tree` extension that passed detection (`pass: samples/tree@0.0.1`), because -the `hello-extensions` buildpack didn't require `tree` in the build plan, the extension was omitted from the detected -group (`skip: samples/tree@0.0.1 provides unused tree`). +Even though there is a `samples/vim` extension that passed detection (`pass: samples/vim@0.0.1`), because +the `hello-extensions` buildpack didn't require `vim` in the build plan, the extension was omitted from the detected +group (`skip: samples/vim@0.0.1 provides unused vim`). -Let's take a look at how the `samples/tree` extension installs `tree` on the builder image... +Let's take a look at how the `samples/vim` extension installs `vim` on the builder image... --- diff --git a/content/docs/features/dockerfiles.md b/content/docs/features/dockerfiles.md index 0919f03a9..705113c7e 100644 --- a/content/docs/features/dockerfiles.md +++ b/content/docs/features/dockerfiles.md @@ -56,8 +56,7 @@ An image extension could be defined with the following directory: * `./bin/detect` is invoked during the `detect` phase. It analyzes application source code to determine if the extension is needed and contributes build plan entries. * `./bin/generate` is invoked during the `generate` phase (a new lifecycle phase that happens after `detect`). It - outputs either or both of `build.Dockerfile` or `run.Dockerfile` for extending the builder or run image, - respectively (in the [initial implementation](#phased-approach), only limited `run.Dockerfile`s are allowed). + outputs either or both of `build.Dockerfile` or `run.Dockerfile` for extending the builder or run image. For more information and to see a build in action, see [authoring an image extension](/docs/extension-guide/create-extension). @@ -78,30 +77,9 @@ should be **used with great care**. Platform operators should be mindful that: may not have all the mixins required by buildpacks that detected. Platforms may wish to optionally re-validate mixins prior to `build` when using extensions. -### Phased approach +### Putting it all together -Some limitations of the initial implementation of the Dockerfiles feature have already been mentioned, and we'll expand -on them here. As this is a large and complicated feature, the implementation has been split into phases in order to -deliver incremental value and gather feedback. - -#### Phase 1 (supported in lifecycle `0.15.0` or greater) - -One or more `run.Dockerfile`s each containing a single `FROM` instruction can be used to switch the original run image -to a new image (as no image modifications are permitted, there is no need to run `extend` on the run image) - -#### Phase 2 (supported in lifecycle `0.15.0` or greater) - -One or more `build.Dockerfile`s can be used to extend the builder image - -* A new `extend` lifecycle phase is introduced to apply `build.Dockerfile`s from `generate` to the builder image - -#### Phase 3 (future) - -One or more `run.Dockerfile`s can be used to extend the run image - -* The `extend` lifecycle phase can be run in parallel for the builder and run images - -The final ordering of lifecycle phases will look something like the following: +The ordering of lifecycle phases looks like the following: * `analyze` * `detect` - after standard detection, `detect` will also run extensions' `./bin/generate`; output Dockerfiles are @@ -116,9 +94,9 @@ For more information, consult the [migration guide](/docs/reference/spec/migrati #### Platform support for Dockerfiles -Supported (phases 1 and 2): +Supported: -* [pack cli](https://github.com/buildpacks/pack) (version `0.28.0` and above) +* [pack cli](https://github.com/buildpacks/pack) (prefer version `0.30.0` and above) Needs support: