From d8f3a6e82969a92f456c473c0ac97356ec31fd73 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Sun, 22 Sep 2024 21:20:07 -0400 Subject: [PATCH 01/12] expands platform lifecycle tutorial Signed-off-by: Hanan Younes --- .../concepts/lifecycle/analyze.md | 2 +- .../tutorials/lifecycle/_index.md | 175 +++++++++++++++++- 2 files changed, 175 insertions(+), 2 deletions(-) diff --git a/content/docs/for-platform-operators/concepts/lifecycle/analyze.md b/content/docs/for-platform-operators/concepts/lifecycle/analyze.md index 5f8505af6..48c1423c0 100644 --- a/content/docs/for-platform-operators/concepts/lifecycle/analyze.md +++ b/content/docs/for-platform-operators/concepts/lifecycle/analyze.md @@ -15,7 +15,7 @@ This information is used during the `export` phase in order to avoid re-uploadin Starting from `Platform API 0.7`, the `analyze` phase runs before the `detect` phase in order to validate registry access for all images that are used during the `build` as early as possible. In this way it provides faster failures for end users. The other responsibilities of the `analyzer` were moved to the `restorer`.\ For more information, please see [this migration guide][platform-api-06-07-migration]. -The `lifecycle` should attempt to locate a reference to the latest `OCI image` from a previous build that is readable and was created by the `lifecycle` using the same application source code. If no such reference is found, the `analysis` is skipped.\ +The `lifecycle` should attempt to locate a reference to the latest `OCI image` from a previous build that is readable and was created by the `lifecycle` using the same application source code. If no such reference is found, the `analysis` is skipped. The `lifecycle` must write [analysis metadata][analyzedtoml-toml] to ``, where: diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 90f0f1148..00fbd73c3 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -5,4 +5,177 @@ expand=true include_summaries=true +++ -Coming soon! In the meantime check out this excellent [blog post](https://medium.com/buildpacks/unpacking-cloud-native-buildpacks-ff51b5a767bf) from our community. +A `platform` orchestrates builds by invoking the [lifecycle][lifecycle] binary together with buildpacks and application source code to produce a runnable `OCI image`. + + + +The majority of Buildpack users use platforms, such as [pack][pack] and [kpack][kpack], to run Buildpacks and create `OCI images`. However this might not be desireable especially for users maintaining their own platforms and seeking more control over how the underlying Buildpack `lifecycle phases` are executed. + +In this step-by-step tutorial, you will build a `Bash` application without using any `platform` tooling like `pack` or `kpack`. You will also leverage the individual `lifecycle phases` to produce a runnable application image. + +## Prerequisites + +You'll need to clone a local copy of the following to get started: + +* The `lifecycle`: `git clone https://github.com/buildpacks/lifecycle` +* The official `Buildpack.io` samples repo: `git clone https://github.com/buildpacks/samples` + +As previously mentioned, the `lifecycle` orchestrates `Buildpacks` then assembles the resulting artifacts into an `OCI image`. The `lifecycle` is composed of a series of distinct `phases` that need to be executed to have the final image built and exported. + +## Overview + +Now that you’re set up, let’s build our `Bash` application and dive deeper into the `lifecycle` and its phases. + +### Build the lifecycle + +As a starting step, you need to build the `lifecycle` in order to use its phases. This could be done by navigating to the `lifecycle` directory and executing one of the following commands, depending on your system architecture. + +* `make build` for `AMD64` architectures +* `make build-darwin-arm64` for `ARM64` architectures + +### Set environment variables + +In order to execute the various `lifecycle phases` correctly, you first need to set the values of few important environment variables by running the following commands in the terminal: + +```text + +export CNB_USER_ID=1000 CNB_GROUP_ID=1000 CNB_PLATFORM_API=0.14 +export CNB_SAMPLES_PATH="//samples" +export CNB_LIFECYCLE_PATH="//lifecycle"` + +``` + +Where + +* `CNB_USER_ID` and `CNB_GROUP_ID` are arbitrary values that need to be consistent, which both have a default value of `1000`. +* `CNB_PLATFORM_API` or the `Platform API` version, varies depending on the use case. This tutorial uses `v0.14`, which is the latest [Platform API][Platform API] version. +* `CNB_SAMPLES_PATH` represents the path of our local copy of the `samples` directory. +* `CNB_LIFECYCLE_PATH` represents the path of our local compiled `lifecycle` directory. + +### Examine lifecycle phases + +A single app image build consists of the following phases: + +1. [Analysis](#analyze) +2. [Detection](#detect) +3. [Cache Restoration](#restore) +4. [Build](#build) +5. [Export](#export) + +> Note that a `platform` executes the phases above either by invoking phase-specific lifecycle binaries in order or by executing `/cnb/lifecycle/creator`. + +Let's expand each `lifecycle` phase to explain how the `lifecycle` orchestrates buildpacks: + +#### Analyze + +The `analyze` phase runs before the `detect` phase in order to validate registry access for all images used during the `build` as early as possible. In this way it provides faster failures for end users. + +Prior to executing `/cnb/lifecycle/analyzer`, you need to create two directories in the `root` directory as follows: + +```text + +mkdir -p apps/bash-script +mkdir -p layers + +``` + +* `apps` directory that contains a `bash-script` directory +* `layers` directory that contains subdirectories representing each layer created by the Buildpack in the final image or build cache. + +Next, you need to copy the `bash-script` samples into our `apps/bash-script` directory, which will host our app's source code. + +```text + +cp -r "$CNB_SAMPLES_PATH/apps/bash-script/" ./apps/bash-script + +``` + +Now, you can invoke the `analyzer` for `AMD64` and `ARM64` architectures respectively + +```text + +$CNB_LIFECYCLE_PATH/analyzer -log-level debug -daemon -layers="./layers" -run-image cnbs/sample-stack-run:bionic apps/bash-script + +``` + +```text + +$CNB_LIFECYCLE_PATH/analyzer -log-level debug -daemon -layers="./layers" -run-image arm64v8/ubuntu:latest apps/bash-script + +``` + +The commands above run the `analyzer` with: + +* A `debug` logging level +* Pointing to the local `Docker daemon` +* Pointing to the `layers` directory, which is the main `lifecycle` working directory +* Running the specified image +* The path to the app that you are analyzing + +Now the `analyzer`: + +* Checks a registry for previous images called `apps/bash-script`. +* Resolves the image metadata making it available to the subsequent `restore` phase. +* Verifies that you have write access to the registry to create or update the image called `apps/bash-script`. + +In this tutorial, there is no previous `apps/bash-script` image, and the output produced should be similar to the following: + +```text + +PLACEHOLDER + +``` + +Now checking the `layers` directory you should have a `analyzer.toml` file with a bunch of null entries. + +#### Detect + +#### Restore + +The `restorer` retrieves cache contents, if it contains any, into the build container. During this phase, the `restorer` looks for layers that could be reused or should be replaced while building the application image. + +First, you need to create the `cache` directory, and then run the `restorer` binary as follows: + +```text + +mkdir cache + +``` + +```text + +$CNB_LIFECYCLE_PATH/restorer -log-level debug -layers="./layers" -group="./layers/group.toml" -cache-dir="./cache" -analyzed="./layers/analyzed.toml" + +``` + +The `cache` directory should now be populated by two sub-directories, `committed` and `staging` as shown in the output below: + +```text + +Starting restorer... +Parsing inputs... +Ensuring privileges... +Executing command... +Timer: Restorer started at 2023-08-30T17:10:54+02:00 +Restoring Layer Metadata +Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-world +Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-moon +Reading Buildpack Layers directory /github.com/dlion/unpackingCNB/layers +Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-world +Reading Buildpack Layers directory /github.com/dlion/unpackingCNB/layers +Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-moon +Timer: Restorer ran for 832.458µs and ended at 2023-08-30T17:10:54+02:00 + +``` + +#### Build + +#### Export + +> This tutorial is based on a [blog post][blog post] contributed by one of our community. + +[pack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ +[kpack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/kpack/ +[lifecycle]: https://buildpacks.io/docs/for-platform-operators/concepts/lifecycle/ +[Platform API]: https://github.com/buildpacks/spec/releases?q=platform +[blog post]: https://medium.com/buildpacks/unpacking-cloud-native-buildpacks-ff51b5a767bf From 6ccc5c98f083b31446d7c6c3f43cf81feff6aded Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Mon, 23 Sep 2024 19:42:00 -0400 Subject: [PATCH 02/12] adds more content & reviewer feedback Signed-off-by: Hanan Younes --- .../tutorials/lifecycle/_index.md | 138 +++++++++++++----- 1 file changed, 101 insertions(+), 37 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 00fbd73c3..9aa476de8 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -11,14 +11,25 @@ A `platform` orchestrates builds by invoking the [lifecycle][lifecycle] binary t The majority of Buildpack users use platforms, such as [pack][pack] and [kpack][kpack], to run Buildpacks and create `OCI images`. However this might not be desireable especially for users maintaining their own platforms and seeking more control over how the underlying Buildpack `lifecycle phases` are executed. +> This tutorial is derived from a [blog post][blog post] contributed by one of our community. + In this step-by-step tutorial, you will build a `Bash` application without using any `platform` tooling like `pack` or `kpack`. You will also leverage the individual `lifecycle phases` to produce a runnable application image. ## Prerequisites You'll need to clone a local copy of the following to get started: -* The `lifecycle`: `git clone https://github.com/buildpacks/lifecycle` -* The official `Buildpack.io` samples repo: `git clone https://github.com/buildpacks/samples` +* The `lifecycle` + + ```text + git clone https://github.com/buildpacks/lifecycle + ``` + +* The official `Buildpack.io` samples repo + + ```text + git clone https://github.com/buildpacks/samples + ``` As previously mentioned, the `lifecycle` orchestrates `Buildpacks` then assembles the resulting artifacts into an `OCI image`. The `lifecycle` is composed of a series of distinct `phases` that need to be executed to have the final image built and exported. @@ -30,8 +41,10 @@ Now that you’re set up, let’s build our `Bash` application and dive deeper i As a starting step, you need to build the `lifecycle` in order to use its phases. This could be done by navigating to the `lifecycle` directory and executing one of the following commands, depending on your system architecture. -* `make build` for `AMD64` architectures -* `make build-darwin-arm64` for `ARM64` architectures +* `make build` for `AMD64` architectures (for Linux users) +* `make build-darwin-arm64` for `ARM64` architectures (for Mac users) + +> Please note that the entire process is most easily followed on Linux systems ### Set environment variables @@ -73,10 +86,8 @@ The `analyze` phase runs before the `detect` phase in order to validate registry Prior to executing `/cnb/lifecycle/analyzer`, you need to create two directories in the `root` directory as follows: ```text - mkdir -p apps/bash-script mkdir -p layers - ``` * `apps` directory that contains a `bash-script` directory @@ -85,23 +96,17 @@ mkdir -p layers Next, you need to copy the `bash-script` samples into our `apps/bash-script` directory, which will host our app's source code. ```text - -cp -r "$CNB_SAMPLES_PATH/apps/bash-script/" ./apps/bash-script - +cp -r "${CNB_SAMPLES_PATH}/apps/bash-script/" ./apps/bash-script ``` Now, you can invoke the `analyzer` for `AMD64` and `ARM64` architectures respectively ```text - -$CNB_LIFECYCLE_PATH/analyzer -log-level debug -daemon -layers="./layers" -run-image cnbs/sample-stack-run:bionic apps/bash-script - +${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run-image cnbs/sample-stack-run:bionic apps/bash-script ``` ```text - -$CNB_LIFECYCLE_PATH/analyzer -log-level debug -daemon -layers="./layers" -run-image arm64v8/ubuntu:latest apps/bash-script - +${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run-image arm64v8/ubuntu:latest apps/bash-script ``` The commands above run the `analyzer` with: @@ -121,9 +126,7 @@ Now the `analyzer`: In this tutorial, there is no previous `apps/bash-script` image, and the output produced should be similar to the following: ```text - -PLACEHOLDER - +OUTPUT PLACEHOLDER ``` Now checking the `layers` directory you should have a `analyzer.toml` file with a bunch of null entries. @@ -137,42 +140,103 @@ The `restorer` retrieves cache contents, if it contains any, into the build cont First, you need to create the `cache` directory, and then run the `restorer` binary as follows: ```text - mkdir cache +``` +```text +${CNB_LIFECYCLE_PATH}/restorer -log-level debug -layers="./layers" -group="./layers/group.toml" -cache-dir="./cache" -analyzed="./layers/analyzed.toml" ``` +The `cache` directory should now be populated by two sub-directories, `committed` and `staging` as shown in the output below: + ```text +OUTPUT PLACEHOLDER +``` + +#### Build + +The `builder` transforms application source code into runnable artifacts that can be packaged into a container. + +Before running the `builder`, the following steps are required: + +1. Create two directories: + * `platform` directory to store configurations and environment variables + * `workspace` directory to store application source code and where you build it + + ```text + mkdir -p platform + mkdir -p workspace + ``` + +2. Copy the source code from the `app` directory to the `workspace` directory -$CNB_LIFECYCLE_PATH/restorer -log-level debug -layers="./layers" -group="./layers/group.toml" -cache-dir="./cache" -analyzed="./layers/analyzed.toml" + ```text + cp -r apps/bash-script/* ./workspace + ``` +3. Create a `launcher` file with instructions to run your application + + ```text + cat << EOF > ./layers/samples_hello-moon/launch.toml + [[processes]] + type = "shell" + command = ["./app.sh"] + EOF + ``` + +Now you're ready to run the `builder` as follows: + +```text +${CNB_LIFECYCLE_PATH}/builder -log-level debug -layers="./layers" -group="./layers/group.toml" -analyzed="./layers/analyzed.toml" -plan="./layers/plan.toml" -buildpacks="./buildpacks" -app="./workspace" -platform="./platform" ``` -The `cache` directory should now be populated by two sub-directories, `committed` and `staging` as shown in the output below: +Taking a deep look at the following output shows that you have built the two buildpacks that we need in order to run our `bash-script` application. In addition, checking the `layers` directory should show other directories like the two from the buildpacks, a `config` and a `sbom` ones. + +```text +OUTPUT PLACEHOLDER +``` + +#### Export + +The purpose of the `export` phase is to create a new `OCI` image using a combination of remote layers, local `/` layers, and the processed `app` directory. + +To export the artifacts built by the `builder`, you first need to specify the path of the `launcher` that your image is going to run: + +* For AMD64 architectures + + ```text + export {CNB_LINUX_LAUNCHER_PATH}=//lifecycle/out/linux-amd64/lifecycle/launcher + ``` + +* For ARM64 Architectures + + ```text + export {CNB_LINUX_LAUNCHER_PATH}=//lifecycle/out/linux-arm64/lifecycle/launcher + ``` + +Now you can run the `exporter` ```text +${CNB_LIFECYCLE_PATH}/exporter --log-level debug -launch-cache "./cache" -daemon -cache-dir "./cache" -analyzed "./layers/analyzed.toml" -group "./layers/group.toml" -layers="./layers" -app "./workspace" -launcher="${CNB_LINUX_LAUNCHER_PATH}" -process-type="shell" apps/bash-script +``` + +You can verify that the image was successfully exported by running the `docker images` command. -Starting restorer... -Parsing inputs... -Ensuring privileges... -Executing command... -Timer: Restorer started at 2023-08-30T17:10:54+02:00 -Restoring Layer Metadata -Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-world -Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-moon -Reading Buildpack Layers directory /github.com/dlion/unpackingCNB/layers -Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-world -Reading Buildpack Layers directory /github.com/dlion/unpackingCNB/layers -Reading buildpack directory: /github.com/dlion/unpackingCNB/layers/samples_hello-moon -Timer: Restorer ran for 832.458µs and ended at 2023-08-30T17:10:54+02:00 +### Run the app image +Finally, you can run the exported image as follows: + +```text +docker run -it apps/bash-script ./app.sh ``` -#### Build +```text +OUTPUT PLACEHOLDER +``` -#### Export +## Wrapping up -> This tutorial is based on a [blog post][blog post] contributed by one of our community. +At the end of this tutorial, we hope that you have a better overview of the process of using `Buildpacks` to create container images. You are now ready to explore this technology further and adapt it to your application development and deployment needs. [pack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ [kpack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/kpack/ From 751f86bed8a68841d47577aad0a037dcad14e494 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Mon, 23 Sep 2024 22:34:49 -0400 Subject: [PATCH 03/12] adds detect section & updates few links Signed-off-by: Hanan Younes --- .../tutorials/lifecycle/_index.md | 54 +++++++++++++++++-- 1 file changed, 51 insertions(+), 3 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 9aa476de8..ee1148846 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -129,15 +129,62 @@ In this tutorial, there is no previous `apps/bash-script` image, and the output OUTPUT PLACEHOLDER ``` -Now checking the `layers` directory you should have a `analyzer.toml` file with a bunch of null entries. +Now if you check the `layers` directory, you should have a `analyzer.toml` file with a few null entries. #### Detect +In this phase, the `detector` looks for an ordered group of buildpacks that will be used during the `build` phase. The `detector` requires an `order.toml` file being present in the `root` directory, which you could derived from `builder.toml` in the `samples` directory while removing the deprecated `stack` section as follows: + +```text +cat "${CNB_SAMPLES_PATH}/builders/jammy/builder.toml" | grep -v -i "stack" | sed 's/\.\.\/\.\./\./' > order.toml + +``` + +`order.toml` files contain a list of groups with each group containing a list of buildpacks. The `detector` reads `order.toml` and looks for the first group that passes the detection process. + +##### Set buildpacks layout directory + +Before running the `detector`, you need to: + +1. Create a `buildpacks` directory in the `root` directory + + ```text + mkdir -p buildpacks + ``` + +2. Then you must populate the `buildpacks` directory with your buildpacks of interest. + + > You have to follow the [directory layout][directory layout] defined in the buildpack spec, where each top-level directory is a `buildpack ID` and each second-level directory is a `buildpack version`. + + Let’s do that for every buildpack in the `samples/buildpacks` directory: + + ```text + for f in `ls --color=no $CNB_SAMPLES_PATH/buildpacks | grep -v README` + do + mkdir -p ./buildpacks/samples_"$f"/0.0.1 + cp -r "$CNB_SAMPLES_PATH/buildpacks/$f/" ./buildpacks/samples_"$f"/0.0.1/ + done + ``` + +Now, you can run the `detector` binary: + +```text +${CNB_LIFECYCLE_PATH}/detector -log-level debug -layers="./layers" -order="./order.toml" -buildpacks="./buildpacks" -app apps/bash-script +``` + +The output of the above command should have `group.toml` and `plan.toml` output files (i.e., the groups that have passed the detection have been written into the `group.toml` file writing its build plan into the `plan.toml` file) + +```text +OUTPUT PLACEHOLDER +``` + +You can view more details about the [order](https://buildpacks.io/docs/for-platform-operators/concepts/lifecycle/detect/#ordertoml), [group](https://buildpacks.io/docs/for-platform-operators/concepts/lifecycle/detect/#grouptoml) and [plan](https://buildpacks.io/docs/concepts/components/lifecycle/detect/#plantoml) toml files in the platform documentation. + #### Restore The `restorer` retrieves cache contents, if it contains any, into the build container. During this phase, the `restorer` looks for layers that could be reused or should be replaced while building the application image. -First, you need to create the `cache` directory, and then run the `restorer` binary as follows: +First, you need to create the `cache` directory, and then run the `restorer` binary as added below: ```text mkdir cache @@ -214,7 +261,7 @@ To export the artifacts built by the `builder`, you first need to specify the pa export {CNB_LINUX_LAUNCHER_PATH}=//lifecycle/out/linux-arm64/lifecycle/launcher ``` -Now you can run the `exporter` +Now you can run the `exporter`: ```text ${CNB_LIFECYCLE_PATH}/exporter --log-level debug -launch-cache "./cache" -daemon -cache-dir "./cache" -analyzed "./layers/analyzed.toml" -group "./layers/group.toml" -layers="./layers" -app "./workspace" -launcher="${CNB_LINUX_LAUNCHER_PATH}" -process-type="shell" apps/bash-script @@ -241,5 +288,6 @@ At the end of this tutorial, we hope that you have a better overview of the proc [pack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ [kpack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/kpack/ [lifecycle]: https://buildpacks.io/docs/for-platform-operators/concepts/lifecycle/ +[directory layout]: https://github.com/buildpacks/spec/blob/main/platform.md#buildpacks-directory-layout [Platform API]: https://github.com/buildpacks/spec/releases?q=platform [blog post]: https://medium.com/buildpacks/unpacking-cloud-native-buildpacks-ff51b5a767bf From bb2241bde703bc95cb60129cdee930cad13ed09b Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Mon, 23 Sep 2024 22:35:48 -0400 Subject: [PATCH 04/12] edits the last section Signed-off-by: Hanan Younes --- .../docs/for-platform-operators/tutorials/lifecycle/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index ee1148846..843dd6c7f 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -283,7 +283,7 @@ OUTPUT PLACEHOLDER ## Wrapping up -At the end of this tutorial, we hope that you have a better overview of the process of using `Buildpacks` to create container images. You are now ready to explore this technology further and adapt it to your application development and deployment needs. +At the end of this tutorial, we hope that you now have a better understanding on how you could use `Buildpacks` to create container images. You are now ready to explore this technology further and customize it to fit your application development and deployment needs. [pack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ [kpack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/kpack/ From 363b82ed43c09dd3325d5d41a65e21c6bdf94e59 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Mon, 30 Sep 2024 07:30:30 -0400 Subject: [PATCH 05/12] adds reviewer feedback Signed-off-by: Hanan Younes --- .../tutorials/lifecycle/_index.md | 44 +++++++++++++------ 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 843dd6c7f..f46a8a400 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -51,16 +51,14 @@ As a starting step, you need to build the `lifecycle` in order to use its phases In order to execute the various `lifecycle phases` correctly, you first need to set the values of few important environment variables by running the following commands in the terminal: ```text - -export CNB_USER_ID=1000 CNB_GROUP_ID=1000 CNB_PLATFORM_API=0.14 +export CNB_USER_ID=$(id -u) CNB_GROUP_ID=$(id -g) CNB_PLATFORM_API=0.14 export CNB_SAMPLES_PATH="//samples" -export CNB_LIFECYCLE_PATH="//lifecycle"` - +export CNB_LIFECYCLE_PATH="//lifecycle" ``` Where -* `CNB_USER_ID` and `CNB_GROUP_ID` are arbitrary values that need to be consistent, which both have a default value of `1000`. +* `CNB_USER_ID` and `CNB_GROUP_ID` are arbitrary values that need to be consistent. This example re-uses our user id and group id for the `CNB` user. In a production system, these are commonly set to `1000`. * `CNB_PLATFORM_API` or the `Platform API` version, varies depending on the use case. This tutorial uses `v0.14`, which is the latest [Platform API][Platform API] version. * `CNB_SAMPLES_PATH` represents the path of our local copy of the `samples` directory. * `CNB_LIFECYCLE_PATH` represents the path of our local compiled `lifecycle` directory. @@ -96,15 +94,17 @@ mkdir -p layers Next, you need to copy the `bash-script` samples into our `apps/bash-script` directory, which will host our app's source code. ```text -cp -r "${CNB_SAMPLES_PATH}/apps/bash-script/" ./apps/bash-script +cp -r "${CNB_SAMPLES_PATH}/apps/bash-script/*" ./apps/bash-script ``` -Now, you can invoke the `analyzer` for `AMD64` and `ARM64` architectures respectively +Now, you can invoke the `analyzer` for `AMD64` architecture ```text -${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run-image cnbs/sample-stack-run:bionic apps/bash-script +${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run-image cnbs/sample-stack-run:jammy apps/bash-script ``` +Or if you are on an `ARM64` platform + ```text ${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run-image arm64v8/ubuntu:latest apps/bash-script ``` @@ -126,14 +126,24 @@ Now the `analyzer`: In this tutorial, there is no previous `apps/bash-script` image, and the output produced should be similar to the following: ```text -OUTPUT PLACEHOLDER +sample-stack-run:jammy apps/bash-script +Starting analyzer... +Parsing inputs... +Ensuring privileges... +Executing command... +Timer: Analyzer started at 2024-09-30T07:38:14Z +Image with name "apps/bash-script" not found +Image with name "cnbs/sample-stack-run:jammy" not found +Timer: Analyzer ran for 41.92µs and ended at 2024-09-30T07:38:14Z +Run image info in analyzed metadata is: +{"Reference":"","Image":"cnbs/sample-stack-run:jammy","Extend":false,"target":{"os":"linux","arch":"amd64"}} ``` -Now if you check the `layers` directory, you should have a `analyzer.toml` file with a few null entries. +Now if you check the `layers` directory, you should have a `analyzed.toml` file with a few null entries. #### Detect -In this phase, the `detector` looks for an ordered group of buildpacks that will be used during the `build` phase. The `detector` requires an `order.toml` file being present in the `root` directory, which you could derived from `builder.toml` in the `samples` directory while removing the deprecated `stack` section as follows: +In this phase, the `detector` looks for an ordered group of buildpacks that will be used during the `build` phase. The `detector` requires an `order.toml` file being present in the `root` directory, which you could derive from `builder.toml` in the `samples` directory while removing the deprecated `stack` section as follows: ```text cat "${CNB_SAMPLES_PATH}/builders/jammy/builder.toml" | grep -v -i "stack" | sed 's/\.\.\/\.\./\./' > order.toml @@ -156,13 +166,19 @@ Before running the `detector`, you need to: > You have to follow the [directory layout][directory layout] defined in the buildpack spec, where each top-level directory is a `buildpack ID` and each second-level directory is a `buildpack version`. + We will use [`dasel`](http://github.com/tomwright/dasel/) to help us parse toml files. + + ```command + $ go install github.com/tomwright/dasel/v2/cmd/dasel@master + Let’s do that for every buildpack in the `samples/buildpacks` directory: ```text - for f in `ls --color=no $CNB_SAMPLES_PATH/buildpacks | grep -v README` + for f in $(ls --color=no ${CNB_SAMPLES_PATH}/buildpacks | grep -v README) do - mkdir -p ./buildpacks/samples_"$f"/0.0.1 - cp -r "$CNB_SAMPLES_PATH/buildpacks/$f/" ./buildpacks/samples_"$f"/0.0.1/ + bp_version=$(cat $f | dasel -r toml "${CNB_SAMPLES_PATH}/buildpacks/${f}/buildpack.version" | sed s/\'//g) + mkdir -p ./buildpacks/samples_"${f}"/${bp_version} + cp -r "$CNB_SAMPLES_PATH/buildpacks/${f}/" ./buildpacks/samples_"${f}"/${bp_version}/ done ``` From d17e96ab6bb9764b6ac921651f80dbb8db9611a1 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Tue, 1 Oct 2024 08:25:25 -0400 Subject: [PATCH 06/12] adds more reviewer feedback Signed-off-by: Hanan Younes --- .../tutorials/lifecycle/_index.md | 145 +++++++++++++++++- 1 file changed, 137 insertions(+), 8 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index f46a8a400..ecff6ec9c 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -176,9 +176,9 @@ Before running the `detector`, you need to: ```text for f in $(ls --color=no ${CNB_SAMPLES_PATH}/buildpacks | grep -v README) do - bp_version=$(cat $f | dasel -r toml "${CNB_SAMPLES_PATH}/buildpacks/${f}/buildpack.version" | sed s/\'//g) + bp_version=$(cat ${CNB_SAMPLES_PATH}/buildpacks/$f/buildpack.toml | dasel -r toml buildpack.version | sed s/\'//g); mkdir -p ./buildpacks/samples_"${f}"/${bp_version} - cp -r "$CNB_SAMPLES_PATH/buildpacks/${f}/" ./buildpacks/samples_"${f}"/${bp_version}/ + cp -r "$CNB_SAMPLES_PATH/buildpacks/${f}/*" ./buildpacks/samples_"${f}"/${bp_version}/ done ``` @@ -188,10 +188,36 @@ Now, you can run the `detector` binary: ${CNB_LIFECYCLE_PATH}/detector -log-level debug -layers="./layers" -order="./order.toml" -buildpacks="./buildpacks" -app apps/bash-script ``` -The output of the above command should have `group.toml` and `plan.toml` output files (i.e., the groups that have passed the detection have been written into the `group.toml` file writing its build plan into the `plan.toml` file) +The output of the above command should have `layers/group.toml` and `layers/plan.toml` output files (i.e., the groups that have passed the detection have been written into the `group.toml` file writing its build plan into the `plan.toml` file) ```text -OUTPUT PLACEHOLDER +Starting detector... +Parsing inputs... +Ensuring privileges... +Executing command... +Timer: Detector started at 2024-10-01T07:00:50Z +Checking for match against descriptor: {linux []} +target distro name/version labels not found, reading /etc/os-release file +Checking for match against descriptor: {linux []} +target distro name/version labels not found, reading /etc/os-release file +Checking for match against descriptor: {linux []} +target distro name/version labels not found, reading /etc/os-release file +Checking for match against descriptor: {linux []} +target distro name/version labels not found, reading /etc/os-release file +Checking for match against descriptor: {linux []} +======== Results ======== +fail: samples/java-maven@0.0.2 +======== Results ======== +fail: samples/kotlin-gradle@0.0.2 +======== Results ======== +fail: samples/ruby-bundler@0.0.1 +======== Results ======== +pass: samples/hello-world@0.0.1 +pass: samples/hello-moon@0.0.1 +Resolving plan... (try #1) +samples/hello-world 0.0.1 +samples/hello-moon 0.0.1 +Timer: Detector ran for 26.011769ms and ended at 2024-10-01T07:00:50Z ``` You can view more details about the [order](https://buildpacks.io/docs/for-platform-operators/concepts/lifecycle/detect/#ordertoml), [group](https://buildpacks.io/docs/for-platform-operators/concepts/lifecycle/detect/#grouptoml) and [plan](https://buildpacks.io/docs/concepts/components/lifecycle/detect/#plantoml) toml files in the platform documentation. @@ -213,7 +239,22 @@ ${CNB_LIFECYCLE_PATH}/restorer -log-level debug -layers="./layers" -group="./lay The `cache` directory should now be populated by two sub-directories, `committed` and `staging` as shown in the output below: ```text -OUTPUT PLACEHOLDER +Starting restorer... +Parsing inputs... +Ensuring privileges... +Executing command... +No run metadata found at path "/cnb/run.toml" +Run image info in analyzed metadata is: +{"Reference":"","Image":"cnbs/sample-stack-run:jammy","Extend":false,"target":{"os":"linux","arch":"amd64"}} +Timer: Restorer started at 2024-10-01T07:03:47Z +Restoring Layer Metadata +Reading buildpack directory: /tmp/example/layers/samples_hello-world +Reading buildpack directory: /tmp/example/layers/samples_hello-moon +Reading Buildpack Layers directory /tmp/example/layers +Reading buildpack directory: /tmp/example/layers/samples_hello-world +Reading Buildpack Layers directory /tmp/example/layers +Reading buildpack directory: /tmp/example/layers/samples_hello-moon +Timer: Restorer ran for 274.41µs and ended at 2024-10-01T07:03:47Z ``` #### Build @@ -240,6 +281,7 @@ Before running the `builder`, the following steps are required: 3. Create a `launcher` file with instructions to run your application ```text + mkdir -p layers/samples_hello-moon cat << EOF > ./layers/samples_hello-moon/launch.toml [[processes]] type = "shell" @@ -256,7 +298,94 @@ ${CNB_LIFECYCLE_PATH}/builder -log-level debug -layers="./layers" -group="./laye Taking a deep look at the following output shows that you have built the two buildpacks that we need in order to run our `bash-script` application. In addition, checking the `layers` directory should show other directories like the two from the buildpacks, a `config` and a `sbom` ones. ```text -OUTPUT PLACEHOLDER +Starting builder... +Parsing inputs... +Ensuring privileges... +Executing command... +Timer: Builder started at 2024-10-01T07:07:46Z +target distro name/version labels not found, reading /etc/os-release file +Running build for buildpack samples/hello-world@0.0.1 +Looking up buildpack +Finding plan +Creating plan directory +Preparing paths +Running build command +---> Hello World buildpack + platform_dir files: + /tmp/example/platform: + total 0 + drwxr-xr-x 2 gitpod gitpod 40 Oct 1 07:04 . + drwxr-xr-x 8 gitpod gitpod 180 Oct 1 07:04 .. + env_dir: /tmp/example/platform/env + env vars: + declare -x CNB_BP_PLAN_PATH="/tmp/samples_hello-world-576309032/samples_hello-world/plan.toml" + declare -x CNB_BUILDPACK_DIR="/tmp/example/buildpacks/samples_hello-world/0.0.1" + declare -x CNB_LAYERS_DIR="/tmp/example/layers/samples_hello-world" + declare -x CNB_PLATFORM_DIR="/tmp/example/platform" + declare -x CNB_TARGET_ARCH="amd64" + declare -x CNB_TARGET_DISTRO_NAME="ubuntu" + declare -x CNB_TARGET_DISTRO_VERSION="22.04" + declare -x CNB_TARGET_OS="linux" + declare -x HOME="/home/gitpod" + declare -x HOSTNAME="buildpacks-docs-dusxugo5ehi" + declare -x OLDPWD + declare -x PATH="/bin:/usr/bin:/usr/local/bin" + declare -x PWD="/tmp/example/workspace" + declare -x SHLVL="1" + layers_dir: /tmp/example/layers/samples_hello-world + plan_path: /tmp/samples_hello-world-576309032/samples_hello-world/plan.toml + plan contents: + [[entries]] + name = "some-world" + + [[entries]] + name = "some-world" + [entries.metadata] + world = "Earth-616" +---> Done +Processing layers +Updating environment +Reading output files +Updating buildpack processes +Updating process list +Finished running build for buildpack samples/hello-world@0.0.1 +Running build for buildpack samples/hello-moon@0.0.1 +Looking up buildpack +Finding plan +Creating plan directory +Preparing paths +Running build command +---> Hello Moon buildpack + env_dir: /tmp/example/platform/env + env vars: + declare -x CNB_BP_PLAN_PATH="/tmp/samples_hello-moon-3356528850/samples_hello-moon/plan.toml" + declare -x CNB_BUILDPACK_DIR="/tmp/example/buildpacks/samples_hello-moon/0.0.1" + declare -x CNB_LAYERS_DIR="/tmp/example/layers/samples_hello-moon" + declare -x CNB_PLATFORM_DIR="/tmp/example/platform" + declare -x CNB_TARGET_ARCH="amd64" + declare -x CNB_TARGET_DISTRO_NAME="ubuntu" + declare -x CNB_TARGET_DISTRO_VERSION="22.04" + declare -x CNB_TARGET_OS="linux" + declare -x HOME="/home/gitpod" + declare -x HOSTNAME="buildpacks-docs-dusxugo5ehi" + declare -x OLDPWD + declare -x PATH="/bin:/usr/bin:/usr/local/bin" + declare -x PWD="/tmp/example/workspace" + declare -x SHLVL="1" + layers_dir: /tmp/example/layers/samples_hello-moon + plan_path: /tmp/samples_hello-moon-3356528850/samples_hello-moon/plan.toml + plan contents: +---> Done +Processing layers +Updating environment +Reading output files +Updating buildpack processes +Updating process list +Finished running build for buildpack samples/hello-moon@0.0.1 +Copying SBOM files +Creating SBOM files for legacy BOM +Listing processes +Timer: Builder ran for 20.200892ms and ended at 2024-10-01T07:07:46Z ``` #### Export @@ -268,13 +397,13 @@ To export the artifacts built by the `builder`, you first need to specify the pa * For AMD64 architectures ```text - export {CNB_LINUX_LAUNCHER_PATH}=//lifecycle/out/linux-amd64/lifecycle/launcher + export CNB_LINUX_LAUNCHER_PATH=//lifecycle/out/linux-amd64/lifecycle/launcher ``` * For ARM64 Architectures ```text - export {CNB_LINUX_LAUNCHER_PATH}=//lifecycle/out/linux-arm64/lifecycle/launcher + export CNB_LINUX_LAUNCHER_PATH=//lifecycle/out/linux-arm64/lifecycle/launcher ``` Now you can run the `exporter`: From 14353cd20624de7d636719bde96029d4d389c240 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 4 Oct 2024 08:26:49 +0100 Subject: [PATCH 07/12] Fix recursive copy of samples Make the recursive copy of samples work correctly in the expected tutorial directory structure. Co-authored-by: Natalie Arellano Signed-off-by: Aidan Delaney --- .../docs/for-platform-operators/tutorials/lifecycle/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index ecff6ec9c..bbc0f9f26 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -94,7 +94,7 @@ mkdir -p layers Next, you need to copy the `bash-script` samples into our `apps/bash-script` directory, which will host our app's source code. ```text -cp -r "${CNB_SAMPLES_PATH}/apps/bash-script/*" ./apps/bash-script +cp -r "${CNB_SAMPLES_PATH}/apps/bash-script" ./apps/ ``` Now, you can invoke the `analyzer` for `AMD64` architecture From 493f3bca10a461e7bb4cab5b2a586c6945642ec3 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Fri, 4 Oct 2024 18:24:47 -0400 Subject: [PATCH 08/12] implements reviewer feedback Signed-off-by: Hanan Younes --- .../tutorials/lifecycle/_index.md | 50 +++++++++---------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index ecff6ec9c..f28460f47 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -9,17 +9,17 @@ A `platform` orchestrates builds by invoking the [lifecycle][lifecycle] binary t -The majority of Buildpack users use platforms, such as [pack][pack] and [kpack][kpack], to run Buildpacks and create `OCI images`. However this might not be desireable especially for users maintaining their own platforms and seeking more control over how the underlying Buildpack `lifecycle phases` are executed. +The majority of Buildpack users use community-maintained platforms, such as [pack][pack] and [kpack][kpack], to run Buildpacks and create `OCI images`. However this might not be desireable especially for users maintaining their own platforms and seeking more control over how the underlying Buildpack `lifecycle phases` are executed. -> This tutorial is derived from a [blog post][blog post] contributed by one of our community. +> This tutorial is derived from a [blog post][blog post] contributed by one of our community members. In this step-by-step tutorial, you will build a `Bash` application without using any `platform` tooling like `pack` or `kpack`. You will also leverage the individual `lifecycle phases` to produce a runnable application image. ## Prerequisites -You'll need to clone a local copy of the following to get started: +This tutorial has been tested on linux amd64 and darwin arm64 platforms. You'll need to clone a local copy of the following to get started: -* The `lifecycle` +* The CNB `lifecycle` ```text git clone https://github.com/buildpacks/lifecycle @@ -41,7 +41,7 @@ Now that you’re set up, let’s build our `Bash` application and dive deeper i As a starting step, you need to build the `lifecycle` in order to use its phases. This could be done by navigating to the `lifecycle` directory and executing one of the following commands, depending on your system architecture. -* `make build` for `AMD64` architectures (for Linux users) +* `make build-linux-amd64` for `AMD64` architectures (for Linux users) * `make build-darwin-arm64` for `ARM64` architectures (for Mac users) > Please note that the entire process is most easily followed on Linux systems @@ -53,7 +53,7 @@ In order to execute the various `lifecycle phases` correctly, you first need to ```text export CNB_USER_ID=$(id -u) CNB_GROUP_ID=$(id -g) CNB_PLATFORM_API=0.14 export CNB_SAMPLES_PATH="//samples" -export CNB_LIFECYCLE_PATH="//lifecycle" +export CNB_LIFECYCLE_PATH="//lifecycle" ``` Where @@ -112,16 +112,16 @@ ${CNB_LIFECYCLE_PATH}/analyzer -log-level debug -daemon -layers="./layers" -run- The commands above run the `analyzer` with: * A `debug` logging level -* Pointing to the local `Docker daemon` -* Pointing to the `layers` directory, which is the main `lifecycle` working directory -* Running the specified image -* The path to the app that you are analyzing +* Pointing to the local `Docker daemon` as the repository where the lifecycle should look for images +* Pointing to the `layers` directory, which is the `lifecycle`'s main working directory +* Pointing to a `run` image, the base image for the application +* Specifying a name for the final application image Now the `analyzer`: -* Checks a registry for previous images called `apps/bash-script`. -* Resolves the image metadata making it available to the subsequent `restore` phase. -* Verifies that you have write access to the registry to create or update the image called `apps/bash-script`. +* Checks the image repository (`daemon` in this case) for a previous image called `apps/bash-script` +* Reads metadata from the previous image (if it exists) for later use +* Verifies that have permission to create or update the image called `apps/bash-script` in the image repository In this tutorial, there is no previous `apps/bash-script` image, and the output produced should be similar to the following: @@ -139,11 +139,11 @@ Run image info in analyzed metadata is: {"Reference":"","Image":"cnbs/sample-stack-run:jammy","Extend":false,"target":{"os":"linux","arch":"amd64"}} ``` -Now if you check the `layers` directory, you should have a `analyzed.toml` file with a few null entries. +Now if you `cat ./layers/analyzed.toml`, you should see a few null entries, a `run-image` section that records the provided name provided, and the found `os/arch`. #### Detect -In this phase, the `detector` looks for an ordered group of buildpacks that will be used during the `build` phase. The `detector` requires an `order.toml` file being present in the `root` directory, which you could derive from `builder.toml` in the `samples` directory while removing the deprecated `stack` section as follows: +In this phase, the `detector` looks for an ordered group of buildpacks that will be used during the `build` phase. The `detector` requires an `order.toml` file to be provided. We can derive an order from `builder.toml` in the `samples` directory while removing the deprecated `stack` section as follows: ```text cat "${CNB_SAMPLES_PATH}/builders/jammy/builder.toml" | grep -v -i "stack" | sed 's/\.\.\/\.\./\./' > order.toml @@ -178,7 +178,7 @@ Before running the `detector`, you need to: do bp_version=$(cat ${CNB_SAMPLES_PATH}/buildpacks/$f/buildpack.toml | dasel -r toml buildpack.version | sed s/\'//g); mkdir -p ./buildpacks/samples_"${f}"/${bp_version} - cp -r "$CNB_SAMPLES_PATH/buildpacks/${f}/*" ./buildpacks/samples_"${f}"/${bp_version}/ + cp -r "$CNB_SAMPLES_PATH/buildpacks/${f}/" ./buildpacks/samples_"${f}"/${bp_version}/ done ``` @@ -224,7 +224,7 @@ You can view more details about the [order](https://buildpacks.io/docs/for-platf #### Restore -The `restorer` retrieves cache contents, if it contains any, into the build container. During this phase, the `restorer` looks for layers that could be reused or should be replaced while building the application image. +The `restorer` copies cache contents, if there is a cache, into the build container. This avoids buildpacks having to re-download build-time dependencies that were downloaded during a previous build. First, you need to create the `cache` directory, and then run the `restorer` binary as added below: @@ -233,7 +233,7 @@ mkdir cache ``` ```text -${CNB_LIFECYCLE_PATH}/restorer -log-level debug -layers="./layers" -group="./layers/group.toml" -cache-dir="./cache" -analyzed="./layers/analyzed.toml" +${CNB_LIFECYCLE_PATH}/restorer -log-level debug -daemon -layers="./layers" -group="./layers/group.toml" -cache-dir="./cache" -analyzed="./layers/analyzed.toml" ``` The `cache` directory should now be populated by two sub-directories, `committed` and `staging` as shown in the output below: @@ -259,13 +259,13 @@ Timer: Restorer ran for 274.41µs and ended at 2024-10-01T07:03:47Z #### Build -The `builder` transforms application source code into runnable artifacts that can be packaged into a container. +The `builder` run buildpacks, that do the actual work of transforming application source code into runnable artifacts. Before running the `builder`, the following steps are required: 1. Create two directories: - * `platform` directory to store configurations and environment variables - * `workspace` directory to store application source code and where you build it + * `platform` directory to store platform configuration and environment variables + * `workspace` directory to store the application source code ```text mkdir -p platform @@ -295,7 +295,7 @@ Now you're ready to run the `builder` as follows: ${CNB_LIFECYCLE_PATH}/builder -log-level debug -layers="./layers" -group="./layers/group.toml" -analyzed="./layers/analyzed.toml" -plan="./layers/plan.toml" -buildpacks="./buildpacks" -app="./workspace" -platform="./platform" ``` -Taking a deep look at the following output shows that you have built the two buildpacks that we need in order to run our `bash-script` application. In addition, checking the `layers` directory should show other directories like the two from the buildpacks, a `config` and a `sbom` ones. +Taking a look at the following output shows that you have invoked the two buildpacks that we need in order to run our `bash-script` application. ```text Starting builder... @@ -392,7 +392,7 @@ Timer: Builder ran for 20.200892ms and ended at 2024-10-01T07:07:46Z The purpose of the `export` phase is to create a new `OCI` image using a combination of remote layers, local `/` layers, and the processed `app` directory. -To export the artifacts built by the `builder`, you first need to specify the path of the `launcher` that your image is going to run: +To export the artifacts built by the `builder`, you first need to specify where to find the `launcher` executable that will be bundled into your image as the entrypoint to run: * For AMD64 architectures @@ -409,7 +409,7 @@ To export the artifacts built by the `builder`, you first need to specify the pa Now you can run the `exporter`: ```text -${CNB_LIFECYCLE_PATH}/exporter --log-level debug -launch-cache "./cache" -daemon -cache-dir "./cache" -analyzed "./layers/analyzed.toml" -group "./layers/group.toml" -layers="./layers" -app "./workspace" -launcher="${CNB_LINUX_LAUNCHER_PATH}" -process-type="shell" apps/bash-script +${CNB_LIFECYCLE_PATH}/exporter --log-level debug -launch-cache "./cache" -daemon -cache-dir "./cache" -analyzed "./layers/analyzed.toml" -group "./layers/group.toml" -layers="./layers" -app "./workspace" -launcher="${CNB_LINUX_LAUNCHER_PATH}" apps/bash-script ``` You can verify that the image was successfully exported by running the `docker images` command. @@ -428,7 +428,7 @@ OUTPUT PLACEHOLDER ## Wrapping up -At the end of this tutorial, we hope that you now have a better understanding on how you could use `Buildpacks` to create container images. You are now ready to explore this technology further and customize it to fit your application development and deployment needs. +At the end of this tutorial, we hope that you now have a better understanding of what happens during each lifecycle phase, and how you could use `Buildpacks` to create container images. You are now ready to explore this technology further and customize it to fit your application development and deployment needs. [pack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ [kpack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/kpack/ From 84c25c715c5126a29628491e00861e8a016b1f85 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Mon, 7 Oct 2024 09:14:30 -0400 Subject: [PATCH 09/12] adds latest reviewer comments Signed-off-by: Hanan Younes --- .../for-platform-operators/tutorials/lifecycle/_index.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index efdf356fb..880468b46 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -44,6 +44,7 @@ As a starting step, you need to build the `lifecycle` in order to use its phases * `make build-linux-amd64` for `AMD64` architectures (for Linux users) * `make build-darwin-arm64` for `ARM64` architectures (for Mac users) +It's recommended to check the [lifecycle releases][releases] page to download binaries based on your system. > Please note that the entire process is most easily followed on Linux systems ### Set environment variables @@ -226,7 +227,9 @@ You can view more details about the [order](https://buildpacks.io/docs/for-platf The `restorer` copies cache contents, if there is a cache, into the build container. This avoids buildpacks having to re-download build-time dependencies that were downloaded during a previous build. -First, you need to create the `cache` directory, and then run the `restorer` binary as added below: +This tutorial doesn't have any previous builds, i.e., the `analyze` phase didn't return any `cached` image. Therefore the `restore` phase will not be copying any `cache` contents at this stage. Feel free to inspect the `cache` when the `build` is done, and re-run the tutorial using the cache created to see how this speeds things up. + +Meanwhile you can start by creating a `cache` directory, and then run the `restorer` binary as added below: ```text mkdir cache @@ -390,7 +393,7 @@ Timer: Builder ran for 20.200892ms and ended at 2024-10-01T07:07:46Z #### Export -The purpose of the `export` phase is to create a new `OCI` image using a combination of remote layers, local `/` layers, and the processed `app` directory. +The purpose of the `export` phase is to take the output from buildpacks and package it into an `OCI` image using a combination of remote layers, local buildpack-contributed layers (under ``), and the processed `app` directory. To export the artifacts built by the `builder`, you first need to specify where to find the `launcher` executable that will be bundled into your image as the entrypoint to run: @@ -436,3 +439,4 @@ At the end of this tutorial, we hope that you now have a better understanding of [directory layout]: https://github.com/buildpacks/spec/blob/main/platform.md#buildpacks-directory-layout [Platform API]: https://github.com/buildpacks/spec/releases?q=platform [blog post]: https://medium.com/buildpacks/unpacking-cloud-native-buildpacks-ff51b5a767bf +[releases]: https://github.com/buildpacks/lifecycle/releases From 21a4b89f2fd7456f8daffa2c6dedfe45d2a75b71 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Thu, 10 Oct 2024 18:35:10 -0400 Subject: [PATCH 10/12] adds reviewer final feedback Signed-off-by: Hanan Younes --- .../docs/for-platform-operators/tutorials/lifecycle/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 880468b46..9b6d85c49 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -64,6 +64,8 @@ Where * `CNB_SAMPLES_PATH` represents the path of our local copy of the `samples` directory. * `CNB_LIFECYCLE_PATH` represents the path of our local compiled `lifecycle` directory. +> Please note that we only run the commands above on a host machine for the purpose of this tutorial, which is not a common practice for buildpacks. + ### Examine lifecycle phases A single app image build consists of the following phases: From 78ba3ce38a7b8d67acc9008c076cf0fe73d2dda9 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Thu, 10 Oct 2024 18:39:44 -0400 Subject: [PATCH 11/12] adds minor update Signed-off-by: Hanan Younes --- .../docs/for-platform-operators/tutorials/lifecycle/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 9b6d85c49..29602fc24 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -433,7 +433,7 @@ OUTPUT PLACEHOLDER ## Wrapping up -At the end of this tutorial, we hope that you now have a better understanding of what happens during each lifecycle phase, and how you could use `Buildpacks` to create container images. You are now ready to explore this technology further and customize it to fit your application development and deployment needs. +At the end of this tutorial, we hope that you now have a better understanding of what happens during each `lifecycle phase`, and how you could use `Buildpacks` to create container images. You are now ready to explore this technology further and customize it to fit your application development and deployment needs. [pack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ [kpack]: https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/kpack/ From 4c7092c986052f0f3f03ae4d622b39af3c104978 Mon Sep 17 00:00:00 2001 From: Hanan Younes Date: Wed, 16 Oct 2024 21:20:55 -0400 Subject: [PATCH 12/12] adds reviewer code suggestions Signed-off-by: Hanan Younes --- .../for-platform-operators/tutorials/lifecycle/_index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md index 29602fc24..0a09a2d74 100644 --- a/content/docs/for-platform-operators/tutorials/lifecycle/_index.md +++ b/content/docs/for-platform-operators/tutorials/lifecycle/_index.md @@ -84,9 +84,11 @@ Let's expand each `lifecycle` phase to explain how the `lifecycle` orchestrates The `analyze` phase runs before the `detect` phase in order to validate registry access for all images used during the `build` as early as possible. In this way it provides faster failures for end users. -Prior to executing `/cnb/lifecycle/analyzer`, you need to create two directories in the `root` directory as follows: +Prior to executing `/cnb/lifecycle/analyzer`, you need to create a parent directory for this tutorial and two other directories inside it as follows: ```text +mkdir /tmp/tutorial # or your preferred directory +cd /tmp/tutorial mkdir -p apps/bash-script mkdir -p layers ``` @@ -283,7 +285,7 @@ Before running the `builder`, the following steps are required: cp -r apps/bash-script/* ./workspace ``` -3. Create a `launcher` file with instructions to run your application +3. Create a `launcher` file with instructions to run your application. Note that in a real buildpacks build, the `platform` does not create this file! The `samples/hello-moon` buildpack would create it. In our case, the `samples/hello-moon` buildpack hasn't been updated with this functionality, so we are faking that behavior. ```text mkdir -p layers/samples_hello-moon