Skip to content

Commit

Permalink
Update plugin loading documentation
Browse files Browse the repository at this point in the history
Packer checks a number of directories for plugins upon initialization,
with the introduction of multi-component plugins and underlying changes
to the Packer SDK the ordering changed slightly. These changes update
the related documentation to reflect the new ordering, and adds a plugin
loading ordering section to the docs to help users discover how plugin
loading works.

Include in this change are updates to the PACKER_CONFIG_DIR environment
variables to reflect the XDG base directory specification used as the
default Packer configuration directory layout.

* Update website/content/docs/configure.mdx
  • Loading branch information
nywilken committed Jul 6, 2023
1 parent 8982d4f commit 275c262
Show file tree
Hide file tree
Showing 3 changed files with 107 additions and 158 deletions.
80 changes: 30 additions & 50 deletions website/content/docs/configure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,69 +13,42 @@ so you generally don't have to worry about it until you want to tweak a
configuration. If you're just getting started with Packer, don't worry about
core configuration for now.

## Packer's home directory

Plugins and core configuration files can exist in the home directory of Packer.
The home directory of Packer will be the first one of the following env values
to be set :

| Unix | Windows |
| ---------------------- | --------------------- |
| `${PACKER_CONFIG_DIR}` | `%PACKER_CONFIG_DIR%` |
| `${APPDATA}` | `%APPDATA%` |
| `${HOME}` | `%HOME%` |
| user dir of `${USER}` | user dir of `${USER}` |

-> Note: On this page "Packer's home directory" will be referenced as
`PACKER_HOME_DIR`.

## Packer's config file

Packer can optionally read a JSON file for the end user to set core settings.
The config file of Packer will be looked up on the following paths:

| Unix | Windows |
| -------------------------------- | -------------------------------- |
| `${PACKER_CONFIG}` | `%PACKER_CONFIG%` |
| `PACKER_HOME_DIR/.packerconfig` | `PACKER_HOME_DIR/packer.config/` |
| `${XDG_CONFIG_HOME}/packer` | |
| `PACKER_HOME_DIR/.config/packer` | |

## Packer's config directory

Packer's configuration directory can potentially contain plugins and internal
Packer files. The config dir of Packer will be looked up on the following paths:
Packer files. The Packer config directory will be looked up on the following paths:

| Unix | Windows |
| --------------------------- | --------------------------- |
| `PACKER_HOME_DIR/.packer.d` | `PACKER_HOME_DIR/packer.d/` |
| `${HOME}/.config/packer/` | `%APPDATA%\packer.d\` |

-> **Note:** On Unix systems, Packer defaults to using the XDG base directory specification.
When the environment variable `PACKER_CONFIG_DIR` is unset or empty a default equal to `$HOME/.config/packer` should be used.
In all other cases, where there is an existing older style `.packer.d` directory (e.g `$HOME/.packer.d/`) or PACKER_CONFIG_DIR is not empty
the older configuration directory will be used.

Examples:

- On a Unix system, if the `$PACKER_CONFIG_DIR` env var is set to
- On a Unix system, if the `$PACKER_CONFIG_DIR` environment variable is set to
`/home/packer`, the config directory will be: `/home/packer/.packer.d/` and
other values will not be checked.
- On a Unix system, if the `HOME` env var is `/home/azr` or the `USER` env var
is `azr`, then the config directory will default to `/home/azr/.packer.d/`.
- On a Windows system, if the `PACKER_CONFIG_DIR` env var is set to `C:/`,the
- On a Windows system, if the `PACKER_CONFIG_DIR` environment variable is set to `C:/`,the
config directory will be: `C:/packer.d/` and other values will not be checked.

## Packer's plugin directory

@include "plugins/plugin-location.mdx"
<a id="packer-s-config-file"></a>
## Packer's config file (deprecated)

The format of the configuration file is basic JSON.
Packer can optionally read a JSON file for the end user to set core settings.
The config file of Packer will be looked up on the following paths:

## Packer's cache directory
| Unix | Windows |
| -------------------------------- | --------------------------------- |
| `${PACKER_CONFIG}` | `%PACKER_CONFIG%` |
| `${HOME}/.packerconfig` | `%APPDATA%\packer.config\` |

Packer uses a cache directory to download large files and for logistics around
large file download. By default, Packer caches things in the current directory,
under: `./packer_cache/`. This can be changed by setting the `PACKER_CACHE_DIR`
env var. It is recommended to share the same Packer cache dir across your
builds if you have multiple builds doing similar things to avoid downloading the
same ISO twice for example.
The format of the configuration file is basic JSON.

## Packer config file configuration Reference
### Packer config file configuration Reference

Below is the list of all available configuration parameters for the core
configuration file. None of these are required, since all have defaults.
Expand All @@ -94,13 +67,20 @@ configuration file. None of these are required, since all have defaults.
and the [`packer init`](/packer/docs/commands/init) command to install plugins; if
you are using both, the `required_plugin` config will take precedence.

### HCP Packer Configuration
## Packer's plugin directory

@include "plugins/plugin-location.mdx"

You can configure both legacy JSON and HCL2 Packer templates to publish image metadata to an active HCP Packer registry. The HCP Packer registry helps you track information about machine images, clearly designate which images are appropriate for test and production environments, and query the right images to use in both Packer and Terraform configurations.
## Packer's cache directory

For complete configuration details and examples, refer to [Packer Template Configuration](/hcp/docs/packer/store-image-metadata/template-configuration) in the HCP Packer documentation.
Packer uses a cache directory to download large files and for logistics around
large file download. By default, Packer caches things in the current directory,
under: `./packer_cache/`. This can be changed by setting the `PACKER_CACHE_DIR`
env var. It is recommended to share the same Packer cache directory across your
builds if you have multiple builds doing similar things to avoid downloading the
same ISO twice for example.

## Full list of Environment Variables usable for Packer
## Environment Variables usable for Packer

Packer uses a variety of environmental variables. A listing and description of
each can be found below:
Expand Down
114 changes: 58 additions & 56 deletions website/content/docs/plugins/install-plugins.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ post-processor components that ship with the Packer binary. Packer automatically

This page explains how to install custom external plugins. Refer to [External Plugins](/packer/plugins) for a list of available plugins and their documentation.

## Plugin Loading Order

@include "plugins/plugin-location.mdx"

## Installation Guides

Choose the tab that corresponds to the type of plugin you want to install. If you are not sure, check the plugin's name.
Expand All @@ -22,10 +26,10 @@ Choose the tab that corresponds to the type of plugin you want to install. If yo
<Tabs>
<Tab heading="Packer init (recommended from Packer v1.7.0)">

~> **Note**: Only _multi-component plugin binaries_ -- that is plugins named
-> **Note:** Only _multi-component plugin binaries_ -- plugins named
packer-plugin-\*, like the `packer-plugin-amazon` -- are expected to work with
Packer init. The legacy `builder`, `post-processor` and `provisioner` plugin
types will keep on being detected but Packer cannot install them automatically.
types will continue to be detected but Packer cannot install them automatically.
If a plugin you use has not been upgraded to use the multi-component plugin
architecture, contact your maintainer to request an upgrade.

Expand Down Expand Up @@ -64,9 +68,8 @@ packer {

Each plugin has two identifiers:

- A `source` address, which is only necessary when requiring a plugin outside the HashiCorp domain.
- A unique **local name**, which is used everywhere else in a Packer
configuration.
- A `source` address, which is necessary when requiring a plugin not bundled with the Packer binary.
- A unique **local name**, which is used everywhere else in a Packer configuration.

## Local Names

Expand Down Expand Up @@ -135,7 +138,8 @@ follows:
For example, the fictional `myawesomecloud` plugin could belong to the
`hashicorp` namespace on `github.com`, so its `source` could be
`github.com/hashicorp/myawesomecloud`,
Note: the actual _repository_ that myawesomecloud comes from must always have

-> Note: the actual _repository_ that myawesomecloud comes from must always have
the name format `github.com/hashicorp/packer-plugin-myawesomecloud`, but the
`required_plugins` block omits the redundant `packer-plugin-` repository prefix
for brevity.
Expand All @@ -144,9 +148,34 @@ The source address with all three components given explicitly is called the
plugin's _fully-qualified address_. You will see fully-qualified address in
various outputs, like error messages.

## Plugin location
## Plugin Installation Workflow

@include "plugins/plugin-location.mdx"
* [`packer init`](/packer/docs/commands/init) will install plugins in the **last** directory
in the following numbered list.

1. `PACKER_PLUGINS_PATH` if set will be the sole location for installing plugins. All other
plugin directories will be ignored.
1. `PACKER_CONFIG_DIR`\plugins on Windows systems, or `PACKER_CONFIG_DIR`/plugins on all other systems.

* During the initialization of Packer, any plugin required in the
**`required_plugins`** section will be looked up in all entries of the following
list. **First** plugin found takes precedence. Two binaries of the same plugin
with two different version will be considered as two different plugins. Highest
found version matching `required_plugins` will be taken into consideration.

During initialization, on a `darwin_amd64` system, Packer will look-up for the
following files:

* `PACKER_PLUGIN_PATH/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64`
* `PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64`

The first plugin-name/version files found will take precedence.

For plugins located under the `github.com/azr/happycloud/` directory structure an accompanying SHA256SUM file
will be required in order for `packer init` to ensure the plugin being loaded has not been tampered with.
The SHA256SUM file will be automatically generated when a plugin is installed via `packer init` if the plugin
was installed manually into `PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/` then the file
`PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64_SHA256SUM` must be generated manually as well.

## Implicit Github urls

Expand All @@ -171,37 +200,32 @@ will avoid conflicting with other plugins for other tools, like Terraform.
</Tab>
<Tab heading="manually (multi-component plugin)">

-> The [`packer plugins`](/packer/docs/commands/plugins) command allows to install plugins without going through
`init`. For manual installation of plugin binaries, without the `packer plugins` command, please continue reading.
-> The [`packer plugins`](/packer/docs/commands/plugins), available from Packer v1.8.0, command allows
you to install plugins without going through `init`.

The easiest way to manually install a plugin is to name it correctly, then place
it in the proper directory. To name a plugin correctly, make sure the binary is
named `packer-plugin-NAME`. For example, `packer-plugin-amazon` for a "plugin"
binary named "amazon". This binary will make one or more components available to
use. Valid types for plugins are down this page.

Once the plugin is named properly, Packer automatically discovers plugins in
the following directories in the given order. If a conflicting plugin is found
later, it will take precedence over one found earlier.
```shell
packer plugins install github.com/hashicorp/vagrant
```

1. The directory where `packer` is, or the executable directory.
## Plugin Installation Workflow
Plugin installation via `packer plugins install` works similar to that of the `packer init` command, with the following
exceptions no `required_plugins` block required and can be used with both legacy JSON and HCL2 templates.

2. The `$HOME/.packer.d/plugins` directory, if `$HOME` is defined (Unix)
* [`packer plugins install`](/packer/docs/commands/plugins) will install plugins in the **last** directory
in the following numbered list.

3. The `%APPDATA%/packer.d/plugins` if `%APPDATA%` is defined (Windows)
1. `PACKER_PLUGINS_PATH` if set will be the sole location for installing plugins. All other
plugin directories will be ignored.
1. `PACKER_CONFIG_DIR`\plugins on Windows systems, or `PACKER_CONFIG_DIR`/plugins on all other systems.

4. The `%USERPROFILE%/packer.d/plugins` if `%USERPROFILE%` is defined
(Windows)

5. The current working directory.
For manual installation of plugin binaries, without the `packer plugins` command, please continue reading.

6. The directory defined in the env var `PACKER_PLUGIN_PATH`. There can be more
than one directory defined; for example, `~/custom-dir-1:~/custom-dir-2`.
Separate directories in the PATH string using a colon (`:`) on POSIX systems and
a semicolon (`;`) on Windows systems. The above example path would be able to
find a provisioner named `packer-provisioner-foo` in either
`~/custom-dir-1/packer-provisioner-foo` or
`~/custom-dir-2/packer-provisioner-foo`.
The easiest way to manually install a plugin is to name it correctly, then place
it in the proper directory. To name a plugin correctly, make sure the binary is
named `packer-plugin-NAME`. For example, `packer-plugin-amazon` for a "plugin"
binary named "amazon". This binary will make one or more components available to
use. Valid types for plugins are down this page.

The valid types for plugins are:

Expand All @@ -217,6 +241,7 @@ The valid types for plugins are:
- `provisioner` - A provisioner to install software on images created by a
builder.


</Tab>
<Tab heading="manually (single-component plugin)">

Expand All @@ -226,29 +251,6 @@ named `packer-COMPONENT-NAME`. For example, `packer-provisioner-comment` for a "
binary named "comment". This binary will make a single provisioner named `comment` available to
use. Valid types for plugins are down this page.

Once the plugin is named properly, Packer automatically discovers plugins in
the following directories in the given order. If a conflicting plugin is found
later, it will take precedence over one found earlier.

1. The directory where `packer` is, or the executable directory.

2. The `$HOME/.packer.d/plugins` directory, if `$HOME` is defined (Unix)

3. The `%APPDATA%/packer.d/plugins` if `%APPDATA%` is defined (Windows)

4. The `%USERPROFILE%/packer.d/plugins` if `%USERPROFILE%` is defined
(Windows)

5. The current working directory.

6. The directory defined in the env var `PACKER_PLUGIN_PATH`. There can be more
than one directory defined; for example, `~/custom-dir-1:~/custom-dir-2`.
Separate directories in the PATH string using a colon (`:`) on POSIX systems and
a semicolon (`;`) on Windows systems. The above example path would be able to
find a provisioner named `packer-provisioner-foo` in either
`~/custom-dir-1/packer-provisioner-foo` or
`~/custom-dir-2/packer-provisioner-foo`.

The valid types for plugins are:

- `plugin` - A plugin binary that can contain one or more of each Packer component
Expand All @@ -264,4 +266,4 @@ The valid types for plugins are:
builder.

</Tab>
</Tabs>
</Tabs>
71 changes: 19 additions & 52 deletions website/content/partials/plugins/plugin-location.mdx
Original file line number Diff line number Diff line change
@@ -1,58 +1,25 @@
Plugins will usually be located in the
[PACKER_HOME_DIR](/packer/docs/configure#packer-s-home-directory).
Upon the initialization of Packer, any externally installed plugin will be automatically
discovered and loaded.

* [`packer init`](/packer/docs/commands/init) will install plugins in the **last** directory
in the following numbered list.
Packer plugins will usually be located within a plugins sub-directory under Packer's main config directory
[PACKER_CONFIG_DIR](/packer/docs/configure#packer-s-config-directory). If `PACKER_CONFIG_DIR` is
either not set or empty, a default equal to `$HOME/.config/packer/plugins` on UNIX, or `%APPDATA%\packer.d\plugins`
for Windows, will be used.

* During the initialization of Packer, any plugin required in the
**`required_plugins`** section will be looked up in all entries of the following
list. **First** plugin found takes precedence. Two binaries of the same plugin
with two different version will be considered as two different plugins. Highest
found version matching `required_plugins` will be taken into consideration.
Where applicable, some installation processes such as `packer init` may override the plugin loading process.
Refer to the specific installation guides for any plugin loading overrides.

1. The directory where `packer` is, or the executable directory.
1. The current working directory. (`"."`)
1. The `PACKER_HOME_DIR/plugins` directory. `PACKER_HOME_DIR` refers to *[Packer's home
directory](/packer/docs/configure#packer-s-home-directory)*, if it could be found.
1. The director(y/ies) under the `PACKER_PLUGIN_PATH` env var, if `PACKER_PLUGIN_PATH`
is set.
Packer uses the following process for loading the correct plugin:

~> **Note**: There can be more than one directory in the `PACKER_PLUGIN_PATH`
env var, it will be seperated by a semicolon (`;`) on Windows systems and a
colon (`:`) on other systems. The order priority will be kept.
1. All directories under the `PACKER_PLUGIN_PATH` environment variable, if `PACKER_PLUGIN_PATH`
is set. The `PACKER_PLUGIN_PATH` takes precedences over all other plugin directories; no other directories will be checked.
1. The directory where `packer` is installed, or the executable directory.
1. The current working directory, where `packer build` is being invoked. (`"."`)
1. The `PACKER_CONFIG_DIR/plugins` directory. `PACKER_CONFIG_DIR` refers to *[Packer's config
directory](/packer/docs/configure#packer-s-config-directory)*, if it could be found.

Using the following example :
```hcl
required_plugins {
happycloud = {
version = ">= 2.7.0"
source = "github.com/azr/happycloud"
}
}
```
-> **Note:** The `PACKER_PLUGIN_PATH` environment variable can be set to more that one directories;
for example, ~/custom-dir-1:~/custom-dir-2. Separate directories in the PATH string using a colon (:) on UNIX systems
and a semicolon (;) on Windows systems. The above example path would be able to find a single or multi-component plugin
in either `~/custom-dir-1/packer/` or `~/custom-dir-2/`.

The plugin getter will then install the binaries in the following location for a
system with no `PACKER_PLUGIN_PATH` env var set.

* `PACKER_HOME_DIR/plugins/github.com/hashicorp/happycloud/`

During initialization, on a `darwin_amd64` system, Packer will look-up for the
following files:

* `PACKER_EXEC_DIR/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64`
* `./github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64`
* `PACKER_HOME_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64`
* `PACKER_PLUGIN_PATH/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64`
* `./packer-plugin-happycloud`

The first plugin-name/version files found will take precedence.

For plugins located under the `github.com/azr/happycloud/` directory structure an accompanying SHA256SUM file
will be required in order for `packer init` to ensure the plugin being loaded has not been tampered with.
The SHA256SUM file will be automatically generated when a plugin is installed via `packer init` if the plugin
was installed manually into `PACKER_HOME_DIR/plugins/github.com/azr/happycloud/` then the file
`PACKER_HOME_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64_SHA256SUM` must be generated manually as well.

-> Note: `PACKER_HOME_DIR` is not an actual env var and refers to [Packer's home
directory](#packer-s-home-directory). `PACKER_EXEC_DIR` is not an actual env var
and refers to the directory where `packer` is, or the executable directory.

0 comments on commit 275c262

Please sign in to comment.