-
Notifications
You must be signed in to change notification settings - Fork 162
poudriere.8
Bryan Drewery edited this page Dec 5, 2017
·
26 revisions
POUDRIERE(8) | FreeBSD System Manager's Manual | POUDRIERE(8) |
poudriere — bulk
package builder and port tester
The poudriere tool is used to build packages from
the ports tree. It can also be used to test a single port.
poudriere accepts a global option:
A jail will take approximately 3GB of space.
Of course you can use another version of FreeBSD,
regardless of what version you are running. amd64 users can choose i386 arch
like in this example:
This command will fetch and install a minimal jail, small (~400MB) so you can
create a lot of them. It will install the jail under the pool you have chosen,
at poudriere/jailname.
You also need to have at least one ports tree to build packages from it, so let
us take the default configuration by creating a ports tree.
A ports tree will take approximately 4GB of space.
[Specify a list of ports you want to build]
Create a flat text file in which you put the ports you want to see built by
poudriere.
Any line starting with the hash sign will be treated as a comment.
[Launch the bulk build]
Now you can launch the bulk build. You can specify to build for only one
arch/version ; by default it will make the bulk build on all the jails created
by poudriere.
[Find your packages]
Once the bulk build is over, you can meet your shiny new packages here:
with 81i386 as the name of the jail.
This second example show how to use poudriere for a
single port.
Let's take the example of building a single port;
all the tests will be done in myjail.
It starts the jail, then mount the ports tree (nullfs), then mounts the package
dir (pourdriere/data/packages/<jailname>-<tree>-<setname>),
then it mounts the ~/ports-cvs/mybeautifulporttotest (nullfs) it builds all
the dependencies (except runtime ones) and log it to
poudriere/data/logs/testport/jailname/default/mybeautifulporttotest.log).
If packages for the dependencies already exists it will use them
When all the dependencies are built, packages for them are created so that next
time it will be faster.
All the dependency phase is done with PREFIX == LOCALBASE.
After that it will build the port itself with LOCALBASE != PREFIX and log the
build to
poudriere/data/logs/testport/jailname/default/mybeautifulporttotest.log
It will try to: install it, create a package from it, deinstall it, check for
cruft left behind and propose the line to add to pkg-plist if needed.
It is very easy to extend it so that we can easily add other tests if wanted.
For bulk building, you can customize binary packages produced by
poudriere by changing build options port by port,
and you can also specify building directives in a make.conf file.
Before building a package, poudriere can mount a
directory containing option files if available.
poudriere will check for any of these directories
in this order:
If a directory with this name exists, it is null-mounted into the
/var/db/ports/ directory of the jail, thus
allowing to build package with custom OPTIONS.
The options subcommand can be used to easily
configure options in the correct directory.
This directory has the usual layout for options: it contains one directory per
port (the name of the port) containing an 'options' file with lines similar
to:
As a starter, you may want to copy an existing
/var/db/ports/ to
/usr/local/etc/poudriere.d/options.
You can also specify a blacklist which will disallow the lists port origins from
building on the matched jail. Any of the following are allowed and will all be
used in the order shown:
If QEMU is being used then a special qemu blacklist is also loaded.
You can also specify an optional poudriere.conf that is pulled in depending on
the build. Any of the following are allowed and will all be used in the order
shown:
You can also specify a global make.conf which will be used for all the jails.
Any of the following are allowed and will all be used in the order shown:
The
/usr/local/etc/poudriere.d/hooks/plugins/<plugin>/
directory may contain any of the above patterns as well.
You can also specify a global src.conf which will be used for building jails
with the jail -c subcommand. Any of the following
are allowed and will all be used in the order shown:
You can also specify a global src-env.conf which will be used for building jails
with the jail -c subcommand. Any of the following
are allowed and will all be used in the order shown:
Hook scripts may be loaded in any of the following paths:
For specific hook documentation see:
https://github.com/freebsd/poudriere/wiki/hooks
-j, -z and
-p are all used in the name of the jail.
Be careful to respect the names supported by jail(8):
Be also careful to not begin the name of the jail by a number if you are not in
-stable or current:
http://svn.freebsd.org/viewvc/base?view=revision&revision=209820
In case of bugs, feel free to file a report:
https://github.com/freebsd/poudriere/issues
Baptiste Daroussin
⟨bapt@FreeBSD.org⟩
Bryan Drewery ⟨bdrewery@FreeBSD.org⟩
poudriere | command subcommand [options] |
- -e etcdir
- Path to the directory where poudriere will find its configuration data. See FILES and ENVIRONMENT for more information.
- -A
- Force color support, even if not in a TTY.
- -N
- Disable color support.
- -v
- This will show more information during the build. Specify twice to enable debug output.
- bulk
- This command makes a ready-to-export package tree, and
fills it with binary packages built from a given list of ports. During the
build, hit ^T to send
SIGINFO
and get stats and progress back. - image
- This command allows you to create images.
- jail
- This command allows you to manage the jails used by poudriere which are building environments differing by OS version and architecture.
- ports
- This command allows you to manage different portstrees which will be used by poudriere (create, update and delete portstrees).
- testport
- This command, mainly targeted at ports developers, launches a test on a given port (useful before submitting/committing a port).
- options
- This command allows to configure the options for a given port
- logclean
- This command will cleanup old logfiles
- distclean
- This command will cleanup old distfiles
- pkgclean
- This command will cleanup old and unwanted packages
- queue
- This command allows a non-root user to queue poudriere commands. It is currently EXPERIMENTAL. Using it requires starting poudriered via the provided rc script.
- status
- This command shows status of current and previous builds
- version
- Show version of poudriere.
- -a
- Build all ports in the tree.
- -f file
- Absolute path to a file which contains the list of ports to build. Ports must be specified in the form category/port and shell-style comments are allowed. Multiple -f file arguments may be specified at once.
- cat/port cat/port2 ...
- A list of ports can be specified directly.
- -B name
- Specify which buildname to use. By default YYYY-MM-DD_HH:MM:SS will be used. This can be used to resume a previous build and use the same log and URL paths. Resuming a build will not retry built/failed/skipped/ignored packages.
- -c
- Clean all previously built packages and logs.
- -C
- Clean only the packages specified on the command line or in in the file given in -f file. Implies -c for -a.
- -F
- Only fetch from original MASTER_SITES. Skip FreeBSD mirrors.
- -j name
- Run the bulk build on the jail named name.
- -J number[:number]
- This argument specifies how many number jobs will run in parallel for a bulk build. The optional second number is the number of jobs used for the steps before the build, they are more IO bound than CPU bound, so you may want to use a different number. The default pre-build value is 1.25 times the value of the build value.
- -i
- Interactive mode. Enter jail for interactive testing and automatically cleanup when done. A local pkg.conf(5) repository configuration will be installed to LOCALBASE/etc/pkg/repos/local.conf so that pkg(8) can be used with any existing packages built for the jail. The FreeBSD repository will be disabled by default.
- -I
- Advanced Interactive mode. Leaves jail running with ports
installed after test. When done with the jail you will need to manually
shut it down:
As with -i this will install a pkg.conf(5) file for pkg(8) usage.
poudriere jail -k -j JAILNAME.
- -n
- Dry run. Show what would be done, but do not actually build or delete any packages.
- -N
- Do not build package repository or INDEX when build is completed.
- -p tree
- This flag specifies on which ports tree the bulk build will be done.
- -R
- Clean RESTRICTED packages after building.
- -S
- Don't recursively rebuild packages affected by other packages requiring incremental rebuild. This may result in broken packages if the ones they depend on are updated, are not ABI-compatible, and were not properly PORTREVISION bumped.
- -t
- Add some testing to the specified ports. Add -r to recursively test all port dependencies as well. Currently uninstalls the port, and disable parallel jobs for make. When used with -a then -rk are implied.
- -r
- Recursively test all dependencies as well. This flag is automatically set when using -at.
- -k
- When using -t do not consider failures as fatal. Do not skip dependent ports on findings. This flag is automatically set when using -at.
- -T
- Try building BROKEN ports by defining TRYBROKEN for the build.
- -w
- Save WRKDIR on build failure. The WRKDIR will be tarred up into ${POUDRIERE_DATA}/wrkdirs.
- -v
- This will show more information during the build. Specify twice to enable debug output.
- -z set
- This specifies which SET to use for the build. See CUSTOMISATION for examples of how this is used.
- -o outputdir
- This argument specifies directory where the resulting image will be created.
- -j name
- This argument specifies the name of the jail that is used.
- -p tree
- This argument specifies the name of the ports tree that is used.
- -z set
- This specifies which SET to use for the build.
- -s size
- This specifies the maximum size of the image that gets build.
- -n name
- This specifies the name of the resulting image.
- -h hostname
- This specifies the hostname used for the image. Defaults to poudriere-image.
- -t type
- This specifies the type of image to create:
- iso
- An ISO 9660 format image.
- iso+mfs
- An ISO 9660 format image where the root filesystem gets MFS mounted.
- iso+zmfs
- An ISO 9660 format image where the root filesystem is LZ77 compressed and gets MFS mounted.
- usb
- A GPT-layout prepared UFS2 image containing a UEFI boot loader.
- usb+mfs
- A GPT-layout prepared UFS2 image containing a UEFI boot loader where the root filesystem gets MFS mounted.
- usb+zmfs
- A GPT-layout prepared UFS2 image containing a UEFI boot loader where the root filesystem is LZ77 compressed and gets MFS mounted.
- rawdisk
- A raw UFS2, softupdates-enabled, disk image.
- zrawdisk
- A raw ZFS disk image.
- tar
- An XZ-compressed tarball.
- firmware
- A nanobsd style image with a GPT partitions and a UEFI boot loader.
- rawfirmware
- A raw disk image.
- embedded
- Create a u-boot ready embedded image.
- -X excludefile
- This specifies a list of files to exclude from the final image.
- -f packagelist
- This specifies a list of packages to be pre-installed in the final image.
- -c overlaydir
- This specifies an extra directory whose contents will be copied directly into the final image, starting from the root.
- -c
- Creates a jail.
- -d
- Deletes a jail.
- -i
- Show information about a jail. See also status.
- -l
- List all poudriere jails.
- -n
- When combined with -l, only display jail name.
- -s
- Starts a jail.
- -k
- Kills a jail (stops it).
- -r name
- Rename a jail to name.
- -u
- Update a jail.
- -b
- Build the source provided with the -m src=PATH option.
- -J number
- The number of make jobs will run in parallel for buildworld. Defaults to the number of CPUs reported by: sysctl hw.ncpu.
- -q
- Remove the header when -l is the specified mandatory option. Otherwise, it has no effect.
- -j name
- Specifies the name of the jail.
- -v version
- Specifies which version of FreeBSD to use in the jail. If you are using method ftp then the version should in the form of: 9.0-RELEASE. If you are using method of svn then the version should be in the form of git or svn branches: stable/9 or head for CURRENT. Other methods only use the value for display.
- -a architecture
- Specifies which architecture of FreeBSD to use in the jail. (Default: same as host)
- -m method
- Specifies which method to
use to create the jail. (default: http)
Pre-built distribution options:
- allbsd
- Use http://www.allbsd.org.
- ftp http
- Fetch from configured FREEBSD_HOST variable from poudriere.conf.
- ftp-archive
- Fetch from http://ftp-archive.freebsd.org.
- null
- This option can be used to import an existing directory
that already contains an installed system. The path must be specified
with -M
path. It is expected that this
directory be installed to with the following:
/usr/src# make installworld DESTDIR=PATH DB_FROM_SRC=1 /usr/src# make distrib-dirs DESTDIR=PATH DB_FROM_SRC=1 /usr/src# make distribution DESTDIR=PATH DB_FROM_SRC=1
- src=PATH
- Install from the given src directory at PATH. This directory will not be built from. It is expected that it is already built and maps to a corresponding /usr/obj directory.
- tar=PATH
- Install from the tarball at the given PATH. Note that this method requires the tarball contains the /usr/src files as well if you plan to build any port containing modules.
- url=PATH
- Fetch from given PATH. Any
URL supported by fetch(1) can be used.
For example:
url=file:///mirror/10.0
- git git+http git+https git+ssh
- Will use git, the -v flag to set the branch name and the GIT_BASEURL variable in poudriere.conf.
- src=PATH
- With the -b flag, the src tree will be copied into the jail and built.
- svn svn+file svn+http svn+https
- Will use SVN and the SVN_HOST variable in poudriere.conf.
- -f filesystem
- Specifies the filesystem name (${ZPOOL}/jails/filesystem).
- -K kernelname
- Install the jail with a kernel. If the kernelname is an empty string GENERIC will be used. If installing from ftp then the default kernel will be installed what ever the kernelname value is.
- -M mountpoint
- Gives an alternative mountpoint when creating jail.
- -p name
- This specifies which port tree to start/stop the jail with.
- -P patch
- Apply the specified patch to the source tree before building the jail.
- -S srcpath
- Use the specified srcpath as the FreeBSD source tree mounted inside the jail or from the host for -m null.
- -t version
- instead of upgrading to the latest security fix of the jail version, you can jump to the new specified version.
- -z set
- This specifies which SET to start/stop the jail with.
- -x
- Build the native-xtools target using the host's /usr/src tree and copy this into the jail. The use of /usr/src is due to a bug in the native-xtools build which does not allow it to be built from the jail's own source. Used exclusively for cross building a ports set, typically via qemu-user tools.
- -C data
- Clean poudriere data
folders when deleting a jail. Only used for
-d option.
data options are the following:
- all
- cache
- logs
- packages
- wkrdirs
- -D
- When creating the jail from a git checkout, clone it with the full history instead of a --depth=1.
- -c
- Creates a ports tree.
- -d
- Deletes a ports tree.
- -l
- List all available ports trees.
- -u
- Update a ports tree.
- -B branch
- Specifies which branch to checkout when using the svn or git methods. (Default: head/master)
- -F
- When used with -c, only create the needed file systems (for ZFS) and directories, but do not populate them.
- -M mountpoint
- Path to the source of a ports tree.
- -f filesystem
- The name of the filesystem to create for the ports tree. If 'none' then do not create a filesystem. Defaults to poudriere/ports/default.
- -k
- When used with -d, only unregister the ports tree without removing the files.
- -m method
- When used with -c, specify
which method to use to create the ports
tree. Could be portsnap, git, null, svn{,+http,+https,+file,+ssh}. The
default is portsnap.
- null
- This option can be used to import an existing directory that already contains a manually managed ports tree. The path must be specified with -M path. The path will be null-mounted during builds.
- -n
- When combined with -l, only display the name of the ports tree.
- -p name
- Specifies the name of the ports tree to use.
- -q
- When used with -l, remove the header in the list view.
- -v
- Show more verbose output.
TRYBROKEN=yes
is automatically defined in
the environment to test ports marked as
BROKEN
.
- -[o] origin
- Specifies an origin in the ports tree
- -B name
- Specify which buildname to use. By default YYYY-MM-DD_HH:MM:SS will be used. This can be used to resume a previous build and use the same log and URL paths. Resuming a build will not retry built/failed/skipped/ignored packages.
- -c
- Run make config for the given port.
- -i
- Interactive mode. Enter jail for interactive testing and automatically cleanup when done. A local pkg.conf(5) repository configuration will be installed to LOCALBASE/etc/pkg/repos/local.conf so that pkg(8) can be used with any existing packages built for the jail. The FreeBSD repository will be disabled by default.
- -I
- Advanced Interactive mode. Leaves jail running with port
installed after test. When done with the jail you will need to manually
shut it down:
As with -i this will install a pkg.conf(5) file for pkg(8) usage.
poudriere jail -k -j JAILNAME.
- -j name
- Runs only inside the jail named name.
- -J number[:number]
- This argument specifies how many number jobs will run in parallel for building the dependencies. The optional second number is the number of jobs used for the steps before the build, they are more IO bound than CPU bound, so you may want to use a different number. The default pre-build value is 1.25 times the value of the build value.
- -k
- Do not consider failures as fatal. Find all failures.
- -P
- Use custom prefix.
- -N
- Do not build package repository or INDEX when build of dependencies is completed.
- -p tree
- Specifies which ports tree to use.
- -S
- Don't recursively rebuild packages affected by other packages requiring incremental rebuild. This may result in broken packages if the ones they depend on are updated, are not ABI-compatible, and were not properly PORTREVISION bumped.
- -v
- This will show more information during the build. Specify twice to enable debug output.
- -w
- Save WRKDIR on build failure. The WRKDIR will be tarred up into ${POUDRIERE_DATA}/wrkdirs.
- -z set
- This specifies which SET to use for the build. See CUSTOMISATION for examples of how this is used.
- -a
- Remove all logfiles matching the filter.
- days
- How many days old of logfiles to keep matching the filter.
- -N count
- How many logfiles to keep matching the filter per jail/tree/set combination.
- -j name
- Specifies the name of the jail to filter by.
- -n
- Dry run, do not actually delete anything.
- -B name
- Specifies which buildname to match on. May be a glob.
- -p tree
- Specifies which ports tree to use. This can be specified multiple times to consider multiple trees.
- -y
- Assume yes, do not confirm and just delete the files.
- -v
- This will show more information during the build. Specify twice to enable debug output.
- -z set
- This specifies which SET to filter builds by. Use 0 to match on empty SET.
- -a
- Clean all ports in the tree.
- -f file
- Absolute path to a file which contains the list of ports to clean. Ports must be specified in the form category/port and shell-style comments are allowed. Multiple -f file arguments may be specified at once.
- cat/port cat/port2 ...
- A list of ports can be specified directly.
- -J number
- This argument specifies how many number jobs will run in parallel for gathering distfile information.
- -n
- Dry run, do not actually delete anything.
- -p tree
- Specifies which ports tree to use. This can be specified multiple times to consider multiple tress.
- -y
- Assume yes, do not confirm and just delete the files.
- -v
- This will show more information during the build. Specify twice to enable debug output.
- -A
- Remove all packages.
- -a
- Keep all known ports in the tree.
- -f file
- Absolute path to a file which contains the list of ports to keep. Ports must be specified in the form category/port and shell-style comments are allowed. Multiple -f file arguments may be specified at once.
- cat/port cat/port2 ...
- A list of ports can be specified directly.
- -j name
- Jail to use for the packages to inspect.
- -J number
- This argument specifies how many number jobs will run in parallel for gathering package information.
- -n
- Dry run, do not actually delete anything.
- -N
- Do not build package repository or INDEX when done cleaning.
- -p tree
- Specifies which ports tree to use.
- -R
- Also clean restricted packages.
- -y
- Assume yes, do not confirm and just delete the files.
- -v
- This will show more information during the build. Specify twice to enable debug output.
- -z set
- SET to use for the packages to inspect.
- -f file
- Absolute path to a file which contains the list of ports to configure. Ports must be specified in the form category/port and shell-style comments are allowed.
- cat/port cat/port2 ...
- A list of ports can be specified directly.
- -a architecture
- Specifies which architecture of FreeBSD to use to show options. (Default: the one of jail jailname )
- -c
- Use 'config' target, which will always show the dialog for the given ports.
- -C
- Use 'config-conditional' target, which will only bring up the dialog on new options for the given ports. (This is the default)
- -j jailname
- If given, configure the options only for the given jail.
- -p portstree
- Run the configuration inside the given ports tree (by default uses default)
- -n
- Do not be recursive
- -r
- Remove port options instead of configuring them
- -s
- Show port options instead of configuring them
- -z set
- This specifies which SET to use for the build. See CUSTOMISATION for examples of how this is used.
- -a
- Show all builds, not just latest. This implies -f.
- -b
- Show details about what each builder for the matched builds are doing.
- -B name
- Specifies which buildname to match on. This supports shell globbing. Defaults to "latest". Specifying this implies the -f flag.
- -c
- Show a more compact output and do not include some columns.
- -f
- Show finished builds, not just currently running. This is implied by the -a, -B and -r flags.
- -H
- Do not print headers and separate fields by a single tab instead of arbitrary white space.
- -j name
- Specifies the name of the jail to filter by.
- -l
- Show logs instead of URL.
- -p tree
- This flag specifies which ports tree to filter builds by.
- -r
- Show build results. This implies the -f flag.
- -z set
- This specifies which SET to filter builds by. Use 0 to match on empty SET.
- FETCH_BIND_ADDRESS
- The bind address used by fetch(1) for jail. See fetch(3) for more details.
- HTTP_PROXY FTP_PROXY
- The proxy configuration for ports and jail. See fetch(3) for other supported proxy environment variables.
- MAKEOBJDIRPREFIX
- Passes through for jail builds.
- POUDRIERE_ETC
- If specified, the path to poudriere's config directory. Defaults to /usr/local/etc.
- UMASK
- The umask for files created by poudriere. Defaults to 022.
- POUDRIERE_ETC/poudriere.conf
- See self-documented /usr/local/etc/poudriere.conf.sample for example.
- POUDRIERE_ETC/poudriere.d/poudriere.conf
- The configuration can be stored in the poudriere.d directory as well.
- POUDRIERE_ETC/poudriere.d
- This directory contains various configuration files for the different jails.
poudriere jail -c -v 8.2-RELEASE -a amd64 -j
82amd64
poudriere jail -c -v 8.1-RELEASE -a i386 -j
81i386
poudriere ports -c
echo 'sysutils/screen' >
~/pkglist
echo 'editors/vim' >>
~/pkglist
poudriere bulk -f ~/pkglist -j
81i386
/usr/local/poudriere/data/packages/81i386
poudriere testport -o category/port -j
myjail
/usr/local/etc/poudriere.d/<jailname>-<tree>-<setname>-options
/usr/local/etc/poudriere.d/<jailname>-<setname>-options
/usr/local/etc/poudriere.d/<jailname>-<tree>-options
/usr/local/etc/poudriere.d/<tree>-<setname>-options
/usr/local/etc/poudriere.d/<setname>-options
/usr/local/etc/poudriere.d/<tree>-options
/usr/local/etc/poudriere.d/<jailname>-options
/usr/local/etc/poudriere.d/options
WITH_FOO=true
WITHOUT_BAR=true
/usr/local/etc/poudriere.d/blacklist
/usr/local/etc/poudriere.d/<setname>-blacklist
/usr/local/etc/poudriere.d/<tree>-blacklist
/usr/local/etc/poudriere.d/<jailname>-blacklist
/usr/local/etc/poudriere.d/<tree>-<setname>-blacklist
/usr/local/etc/poudriere.d/<jailname>-<tree>-blacklist
/usr/local/etc/poudriere.d/<jailname>-<setname>-blacklist
/usr/local/etc/poudriere.d/<jailname>-<tree>-<setname>-blacklist
/usr/local/etc/poudriere.d/qemu-blacklist
/usr/local/etc/poudriere.d/poudriere.conf
/usr/local/etc/poudriere.d/<setname>-poudriere.conf
/usr/local/etc/poudriere.d/<tree>-poudriere.conf
/usr/local/etc/poudriere.d/<jailname>-poudriere.conf
/usr/local/etc/poudriere.d/<tree>-<setname>-poudriere.conf
/usr/local/etc/poudriere.d/<jailname>-<tree>-poudriere.conf
/usr/local/etc/poudriere.d/<jailname>-<setname>-poudriere.conf
/usr/local/etc/poudriere.d/<jailname>-<tree>-<setname>-poudriere.conf
/usr/local/etc/poudriere.d/make.conf
/usr/local/etc/poudriere.d/<setname>-make.conf
/usr/local/etc/poudriere.d/<tree>-make.conf
/usr/local/etc/poudriere.d/<jailname>-make.conf
/usr/local/etc/poudriere.d/<tree>-<setname>-make.conf
/usr/local/etc/poudriere.d/<jailname>-<tree>-make.conf
/usr/local/etc/poudriere.d/<jailname>-<setname>-make.conf
/usr/local/etc/poudriere.d/<jailname>-<tree>-<setname>-make.conf
/usr/local/etc/poudriere.d/src.conf
/usr/local/etc/poudriere.d/<setname>-src.conf
/usr/local/etc/poudriere.d/<jailname>-src.conf
/usr/local/etc/poudriere.d/src-env.conf
/usr/local/etc/poudriere.d/<setname>-src-env.conf
/usr/local/etc/poudriere.d/<jailname>-src-env.conf
/usr/local/etc/poudriere.d/hooks/<hook>.sh
/usr/local/etc/poudriere.d/hooks/plugins/<plugin>/<hook>.sh
"This is an arbitrary string that identifies a jail (except it may not contain a '.')"
Bryan Drewery ⟨bdrewery@FreeBSD.org⟩
December 4, 2017 | FreeBSD 12.0-CURRENT |