Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Buildpack environment related content #754

Closed
wants to merge 14 commits into from
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ title="Add labels to the application image"
weight=99
+++

<!--more-->

Labels are key-value pairs, stored as strings, that are attached to an image (i.e., arbitrary metadata). Labels are used to add helpful descriptions or attributes to an application image, which are meaningful to users.

<!--more-->

Labels are usually added at the time an image is created. Images can have multiple labels; however, each key must be unique.

## Key Points
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,20 @@ title="Clear the buildpack environment"
weight=99
+++

"Clearing" the buildpack environment with `clear-env` is the process of preventing end-users from customizing a buildpack's behavior through environment variables.

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
Buildpack authors may elect to clear user-provided environment variables when `bin/detect` and `bin/build` are executed. This is achieved by setting `clear-env` to `true` in [buildpack.toml](https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml); by default `clear-env` is set to `false`.

* When `clear-env` is set to `true` for a given buildpack, the `lifecycle` will not set user-provided environment variables when running `/bin/detect` or `/bin/build`.
* If a buildpack does allow customization by the end-user through the environment (`clear-env` is `false`), there is a special convention for naming the environment variables recognized by the buildpack, shown in the following table:

| Env Variable | Description | Detect | Build | Launch |
|------------------------|---------------------------------------------------|--------|-------|--------|
| `BP_*` | User-provided variable for buildpack | [x] | [x] | |
| `BPL_*` | User-provided variable for exec.d | | | [x] |

### Further Reading

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
For more about how environment variables are specified by end-users, see the page for how to [customize buildpack behavior with build-time environment variables](https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/).
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@ A buildpack can control how a layer will be used by creating a `<layer>.toml` wi

A buildpack might create a `$CNB_LAYERS_DIR/python` directory and a `$CNB_LAYERS_DIR/python.toml` with the following contents:

```
```toml
launch = true
cache = true
build = true
```

In this example:

* the final app image will contain a layer with `python`, as this is needed to run the app
* the `$CNB_LAYERS_DIR/python` directory will be pre-created for future builds, avoiding the need to re-download this large dependency
* buildpacks that follow in the build will be able to use `python`
Expand All @@ -36,7 +37,7 @@ In this example:

This is a simple `./bin/build` script for a buildpack that runs Python's `pip` package manager to resolve dependencies:

```
```bash
#!/bin/sh

PIP_LAYER="$CNB_LAYERS_DIR/pip"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ A buildpack must contain a `buildpack.toml` file in its root directory.

### Example

```
```toml
api = "0.10"

[buildpack]
Expand All @@ -46,7 +46,7 @@ For more information, see [buildpack config](/docs/reference/config/buildpack-co

### Usage

```
```txt
bin/detect
```

Expand All @@ -72,7 +72,7 @@ Other exit codes indicate an error during detection.
This is a simple example of a buildpack that detects a Python application
by checking for the presence of a `requirements.txt` file:

```
```bash
#!/bin/sh

if [ -f requirements.txt ]; then
Expand All @@ -87,7 +87,7 @@ fi

### Usage

```
```txt
bin/build
```

Expand All @@ -108,4 +108,4 @@ It is important to note that multiple buildpacks may work together to create the
each contributing a subset of the dependencies or configuration needed to run the application.
In this way, buildpacks are modular and composable.

[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan
[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,112 @@ title="Specify the environment"
weight=99
+++

Environment variables are a common way to configure buildpacks at build-time and the application at runtime.

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
### Preparing the environment at build time

When the `lifecycle` runs each buildpack, it first tears down any environment variables defined on the `build-time` base image. It only allows a [specific set](https://github.com/buildpacks/lifecycle/blob/a43d5993a4f2cc23c44b6480ba2ab09fe81d57ed/env/build.go#L9-L19) of pre-configured environment variables through.

For the `detect` phase, the `lifecycle` then applies user-provided environment variables, followed by platform-provided environment variables. For more information, see the page for how to [customize buildpack behavior with build-time environment variables](https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/).

For the `build` phase, the process is more complex. Before applying user-provided environment variables, the `lifecycle` applies buildpack-provided environment variables, which is anything that a previous buildpack (a buildpack that ran earlier in the `build` phase) might have configured in its `layers` directory.

>Note that buildpacks cannot set environment variables for other buildpacks during the `detect` phase.

#### Example

Let's look at the following directory tree to see how layers created by a previous buildpack (with id `some-buildpack-id`) would affect the environment for the current buildpack.

```text

layers/
└── some-buildpack-id
├── some-build-layer
│   ├── bin
│   │   └── some-binary
│   ├── env
│   │   └── SOME_VAR # contents foo
│   └── lib
│   └── some-static-library
└── some-build-layer.toml # has build = true in the [types] table

hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
```

With this tree:

* The current buildpack will see `SOME_VAR=foo` in its environment
* The current buildpack will find `some-binary` in `PATH`
* The current buildpack will find `some-static-library` in `LIBRARY_PATH`

Thus, any `<layers>/<layer>/<env>` directory is for setting environment variables directly, and `<layers>/<layer>/<bin>`, `<layers>/<layer>/<lib>`, etc. offer a convenient way to modify `POSIX` path variables.

The full list of convenience directories is summarized in the table below:

| Env Variable | Layer Path | Contents | Build | Launch |
|--------------------------------------------|--------------|------------------|-------|--------|
| `PATH` | `/bin` | binaries | [x] | [x] |
| `LD_LIBRARY_PATH` | `/lib` | shared libraries | [x] | [x] |
| `LIBRARY_PATH` | `/lib` | static libraries | [x] | |
| `CPATH` | `/include` | header files | [x] | |
| `PKG_CONFIG_PATH` | `/pkgconfig` | pc files | [x] | |

* User-provided variables are then applied, meaning that it's possible for the end-user to override buildpack-provided variables.
* Finally, `platform-defined` variables are applied, which eventually override any previous values.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

### Preparing the environment at runtime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can flesh out this section a little more

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you elaborate on this? thanks

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to land this change now, and flesh out this discussion in subsequent changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AidanDelaney

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natalieparellano still waiting for you


At `runtime`, the `lifecycle` (or rather, the piece of the lifecycle known as the `launcher` that gets included in the application image) prepares the environment for the application process.
For setting up environment variables at `runtime`, the [tree above](#example) is still applicable except that the layer must be a `launch` layer, i.e., `some-launch-layer.toml` has

```yaml

[types]
launch = true

```

>Note that the `launcher` binary is found inside the application image at `/cnb/lifecycle/launcher` and is the entrypoint for any `CNB-built` image.

### When multiple buildpacks define the same variable

When multiple buildpacks define the same variable, the ["environment modification rules"](https://github.com/buildpacks/spec/blob/main/buildpack.md#environment-variable-modification-rules) come into play.

Let's say buildpack A (which runs first) defines `SOME_VAR=foo` and buildpack B defines `SOME_VAR=bar`. The `lifecycle` can perform different modifications when setting up the environment for buildpack C (which runs last).

* The `lifecycle` can `append` the second value to the first, so that buildpack C sees something like `SOME_VAR=foo:bar`.
* The `lifecycle can` `prepend` the second value to the first, so that buildpack C sees something like `SOME_VAR=bar:foo`.
* The `lifecycle` can `override` the first value with the second value, so that buildpack C sees `SOME_VAR=bar`.
* The `lifecycle` can treat the second value as a `default` (the value to set when no other entity defines this variable), so that buildpack C sees `SOME_VAR=foo`.
* In all cases, the behavior of the `lifecycle` is governed by the file suffix for `<layers>/<layer>/<env>/SOME_VAR<.suffix>`. The suffix is optional, and the assumed behavior when no suffix is provided is `override`.

>Note that whenever the suffix is `append` or `prepend` an additional file, `<layers>/<layer>/<env>/SOME_VAR.delim`, is needed to specify the delimiter used during concatenation. If no delimiter is provided, none will be used.

To better understand the above modification rules, let's take a look at the tree below:

```text

layers/
├── some-buildpack-id
│   ├── some-layer
│   │   └── env
│   │   ├── SOME_VAR # contents foo
│   │   └── SOME_VAR.append # contents :
│   └── some-layer.toml
└── some-other-buildpack-id
├── some-layer
│   └── env
│   ├── SOME_VAR # contents bar
│   └── SOME_VAR.append # contents :
└── some-layer.toml

```

Assuming that `some-buildpack-id` comes before `some-other-buildpack-id` in the buildpack group, the final value of `SOME_VAR` shown above would be `foo:bar`.

Note that the examples shown on this page are relatively simple. It is possible for a buildpack to double-specify the same variable (i.e., within two or more different layers), and for a buildpack to specify a variable for a particular phase (build or launch) when the layer has type both `build = true` and `launch = true`. Additionally, for `runtime` variables, buildpacks can specify a variable for a particular process.

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
In these cases, the `lifecycle` determines the final value of the variable according to a process outlined in the specification.

To cover:
* POSIX paths
* Build env
* Process env
* Runtime env
* Modifiers
>For more information on modifying environment variables, see the [Environment Variable Modification Rules](https://github.com/buildpacks/spec/blob/main/buildpack.md#environment-variable-modification-rules) section of the specification.
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ title="Use exec.d binaries to configure the application at runtime"
weight=99
+++

The [buildpacks `exec.d` interface](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd) allows buildpack authors to execute custom scripts or binaries when the application image is started.

<!--more-->

The [buildpacks `exec.d` interface](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd) allows buildpack authors to execute custom scripts or binaries when the application image is started. This interface can be particularly useful for injecting dynamic behavior or environment variables into the runtime environment of an application.
This interface can be particularly useful for injecting dynamic behavior or environment variables into the runtime environment of an application.

## Key Points

Expand Down Expand Up @@ -51,23 +53,23 @@ And a `Go` example is:
package main

import (
"fmt"
"os"
"fmt"
"os"
)

func main() {
// Open file descriptor 3 for writing
fd3 := os.NewFile(3, "fd3")
if fd3 == nil {
fmt.Println("Failed to open file descriptor 3")
return
}

// Write the environment variable to file descriptor 3
_, err := fd3.WriteString(`EXAMPLE="test"\n`)
if err != nil {
fmt.Println("Error writing to file descriptor 3:", err)
}
// Open file descriptor 3 for writing
fd3 := os.NewFile(3, "fd3")
if fd3 == nil {
fmt.Println("Failed to open file descriptor 3")
return
}

// Write the environment variable to file descriptor 3
_, err := fd3.WriteString(`EXAMPLE="test"\n`)
if err != nil {
fmt.Println("Error writing to file descriptor 3:", err)
}
}
```

Expand Down
Loading