toolchains.rst

Go toolchains

The Go toolchain is at the heart of the Go rules, and is the mechanism used to customize the behavior of the core Go rules.

Contents

• Design
  • The SDK
  • The toolchain
  • The context
• Customizing
  • Normal usage
  • Forcing the Go version
  • Using the installed Go SDK
  • Registering a custom SDK
  • Writing new Go rules
• API
  • go_register_toolchains
  • go_download_sdk
  • go_host_sdk
  • go_local_sdk
  • go_wrap_sdk
  • go_sdk
  • go_toolchain
  • go_context
  • The context object
  • go_rule

---

Design

The Go toolchain consists of three main layers, the SDK and the toolchain and the context.

The SDK

At the bottom is the Go SDK. This is the same thing you would get if you go to the main Go website and download a binary distribution.

In the past, this has always been a repository rule bound to @go_sdk, but we are moving away from this as part of an effort to enable the use of multiple SDKs in remote execution. It's best to access this through the toolchain.

The go_download_sdk, go_host_sdk, go_local_sdk, and go_wrap_sdk rules are responsible for downloading / locating and configuring an SDK by adding just enough of a build file to expose the contents to Bazel.

SDKs are specific to a host platform (e.g., linux_amd64) and a version of Go. They may target all platforms that Go supports. The Go SDK is naturally cross compiling.

SDKs are declared to Bazel with the go_sdk rule, which gathers information about the SDK into the GoSDK provider, which may be consumed by other rules through the context.

By default, when you call go_register_toolchains(), the Go rules will download the most recent official SDK that was available at the last Go rules release. If you need a forked version of Go, want to control the version, or just use the installed SDK, just declare a Go SDK repository before calling go_register_toolchains.

The toolchain

This a wrapper over the SDK that provides extra information about the target platform.

Toolchains are declared with the go_toolchain rule inside an SDK repository. They are registered within the macro that declares the SDK repository. Bazel will select a toolchain based on the target platform (specified with --platforms on the command line).

The toolchain itself should be considered opaque. You should only access its contents through the context.

The context

The context is the type you need if you are writing custom rules that need to be compatible with rules_go. It provides information about the SDK, the toolchain, and the standard library. It also provides a convenient way to declare mode-specific files, and to create actions for compiling, linking, and more.

Customizing

Normal usage

This is an example of normal usage for the other examples to be compared against. This will download and use the latest Go SDK that was available when the version of rules_go you're using was released.

Forcing the Go version

You can select the version of the Go SDK to use by specifying it when you call go_register_toolchains but you must use a value that matches a known toolchain.

Using the installed Go SDK

You can use the Go SDK that's installed on the system where Bazel is running. This may result in faster builds, since there's no need to download an SDK, but builds won't be reproducible across systems with different SDKs installed.

Registering a custom SDK

If you download the SDK through another repository rule, you can configure it with go_wrap_sdk. It must still be named go_sdk, but this is a temporary limitation that will be removed in the future.

Writing new Go rules

If you are writing a new rule that wants to use the Go toolchain, you need to do a couple of things. First, you have to declare that you want to consume the toolchain on the rule declaration. The easiest way to do this is to use the go_rule wrapper, which adds in the toolchain and some hidden attributes that it consumes.

load("@io_bazel_rules_go//go:def.bzl", "go_context", "go_rule")

my_rule = go_rule(
    _my_rule_impl,
    attrs = {
        ...
    },
)

And then in the rule body, call go_context to get the context object. This will give you access to the toolchain and the SDK.

def _my_rule_impl(ctx):
    go = go_context(ctx)

API

go_register_toolchains

Installs the Go toolchains. If go_version is specified, it sets the SDK version to use (for example, "1.10.3"). By default, the latest SDK will be used.

Name	Type	Default value
go_version	string	latest release
This specifies the version of the Go SDK to download. This is only used if no SDK has been declared with the name go_sdk before the call to go_register_toolchains. The default version is the latest version of Go that was released at the time the rules_go release you're using was tagged.

go_download_sdk

This downloads a Go SDK for use in toolchains.

Name	Type	Default value
name	string	mandatory value
A unique name for this SDK. This should almost always be go_sdk if you want the SDK to be used by toolchains.
urls	string_list	official distributions
A list of mirror urls to the binary distribution of a Go SDK. These must contain the {} used to substitute the sdk filename being fetched (using .format. It defaults to the official repository "https://storage.googleapis.com/golang/{}".
strip_prefix	string	"go"
A directory prefix to strip from the extracted files.
sdks	string_list_dict	mandatory value
This consists of a set of mappings from the host platform tuple to a list of filename and sha256 for that file. The filename is combined the urls to produce the final download urls to use.
goos	string	None
The operating system the binaries in the SDK are intended to run on. By default, this is detected automatically, but if you're building on an unusual platform, or if you're using remote execution and the execution platform is different than the host, you may need to specify this explictly.
goarch	string	None
The architecture the binaries in the SDK are intended to run on. By default, this is detected automatically, but if you're building on an unusual platform, or if you're using remote execution and the execution platform is different than the host, you may need to specify this explictly.

Example:

load(
    "@io_bazel_rules_go//go:deps.bzl",
    "go_download_sdk",
    "go_register_toolchains",
    "go_rules_dependencies",
)

go_download_sdk(
    name = "go_sdk",
    sdks = {
        "linux_amd64":   ("go1.8.1.linux-amd64.tar.gz",
            "a579ab19d5237e263254f1eac5352efcf1d70b9dacadb6d6bb12b0911ede8994"),
        "darwin_amd64":      ("go1.8.1.darwin-amd64.tar.gz",
            "25b026fe2f4de7c80b227f69588b06b93787f5b5f134fbf2d652926c08c04bcd"),
    },
)

go_rules_dependencies()

go_register_toolchains()

go_host_sdk

This detects and configures the host Go SDK for use in toolchains.

If the GOROOT environment variable is set, the SDK in that directory is used. Otherwise, go env GOROOT is used.

Name	Type	Default value
name	string	mandatory value
A unique name for this SDK. This should almost always be go_sdk if you want the SDK to be used by toolchains.

go_local_sdk

This prepares a local path to use as the Go SDK in toolchains.

Name	Type	Default value
name	string	mandatory value
A unique name for this SDK. This should almost always be go_sdk if you want the SDK to be used by toolchains.
path	string	""
The local path to a pre-installed Go SDK. The path must contain the go binary, the tools it invokes and the standard library sources.

go_wrap_sdk

This configures an SDK that was downloaded or located with another repository rule.

Name	Type	Default value
name	string	mandatory value
A unique name for this SDK. This should almost always be go_sdk if you want the SDK to be used by toolchains.
root_file	label	mandatory value
A Bazel label referencing a file in the root directory of the SDK. Used to determine the GOROOT for the SDK.

Example:

load(
    "@io_bazel_rules_go//go:deps.bzl",
    "go_register_toolchains",
    "go_rules_dependencies",
    "go_wrap_sdk",
)

go_wrap_sdk(
    name = "go_sdk",
    root_file = "@other_repo//go:README.md",
)

go_rules_dependencies()

go_register_toolchains()

go_sdk

go_sdk collects information about a Go SDK. The SDK must have a normal GOROOT directory structure.

Normally, go_sdk rules are declared in repositories configured with go_download_sdk, go_host_sdk, go_local_sdk, or go_wrap_sdk. You usually won't need to declare this rule explicitly.

Name	Type	Default value
name	string	mandatory value
A unique name for the SDK rule. Typically go_sdk.
goos	string	mandatory value
The host operating system the SDK was built for. See go/platform/list.bzl for valid values.
goarch	string	mandatory value
The host architecture the SDK was built for. See go/platform/list.bzl for valid values.
root_file	label	mandatory value
A file in the SDK root directory. Used to determine GOROOT.
package_list	label	None
A text file containing a list of packages in the standard library that may be imported. If unspecified, this will be generated from srcs.
libs	label_list	[]
Pre-compiled .a files for the standard library, built for the execution platform.
headers	label_list	[]
.h files from pkg/include that may be included in assembly sources.
srcs	label_list	[]
Source files for packages in the standard library.
tools	label_list	[]
Executable files from pkg/tool, built for the execution platform.
go	label	mandatory value
The go binary.

go_toolchain

This adds a toolchain of type "@io_bazel_rules_go//go:toolchain".

Normally, go_toolchain rules are declared and registered in repositories configured with go_download_sdk, go_host_sdk, go_local_sdk, or go_wrap_sdk. You usually won't need to declare these explicitly

Name	Type	Default value
name	string	mandatory value
A unique name for the toolchain.
target	string	mandatory value
Specifies the default target platform for this toolchain. It must be of the form <goos>_<goarch> (for example, linux_amd64). See go/platform/list.bzl for valid values.
host	string	target
Specifies the host platform for this toolchain. It must be of the form <goos>_<goarch> (for example, linux_amd64). See go/platform/list.bzl for valid values.
sdk	label	mandatory value
The SDK this toolchain is based on. The target must provide GoSDK. This is usually a go_sdk rule.
link_flags	string_list	[]
Flags passed to the Go external linker.
cgo_link_flags	string_list	[]
Flags passed to the external linker (if it is used).
constraints	label_list	[]
Additional constraints for the target platform. Bazel will take these into account when selecting the toolchain.

go_context

This collects the information needed to form and return a GoContext from a rule ctx. It uses the attributes and the toolchains. It can only be used in the implementation of a rule that has the go toolchain attached and the go context data as an attribute. To do this declare the rule using the go_rule wrapper.

def _my_rule_impl(ctx):
    go = go_context(ctx)
    ...

my_rule = go_rule(
    _my_rule_impl,
    attrs = {
        ...
    },
)

Name	Type	Default value
ctx	ctx	mandatory value
The Bazel ctx object for the current rule.

The context object

GoContext is never returned by a rule, instead you build one using go_context(ctx) in the top of any custom skylark rule that wants to interact with the go rules. It provides all the information needed to create go actions, and create or interact with the other go providers.

When you get a GoContext from a context it exposes a number of fields and methods.

All methods take the GoContext as the only positional argument. All other arguments must be passed as keyword arguments. This allows us to re-order and deprecate individual parameters over time.

Fields

Name	Type
toolchain	ToolchainInfo
The underlying toolchain. This should be considered an opaque type subject to change.
sdk	GoSDK
The SDK in use. This may be used to access sources, packages, and tools.
mode	Mode
Controls the compilation setup affecting things like enabling profilers and sanitizers. See compilation modes for more information about the allowed values.
root	string
Path of the effective GOROOT. If stdlib is set, this is the same as go.stdlib.root_file.dirname. Otherwise, this is the same as go.sdk.root_file.dirname.
go	File
The main "go" binary used to run go sdk tools.
stdlib	GoStdLib
The standard library and tools to use in this build mode. This may be the pre-compiled standard library that comes with the SDK, or it may be compiled in a different directory for this mode.
actions	ctx.actions
The actions structure from the Bazel context, which has all the methods for building new bazel actions.
exe_extension	string
The suffix to use for all executables in this build mode. Mostly used when generating the output filenames of binary rules.
shared_extension	string
The suffix to use for shared libraries in this build mode. Mostly used when generating output filenames of binary rules.
crosstool	list of File
The files you need to add to the inputs of an action in order to use the cc toolchain.
package_list	File
A file that contains the package list of the standard library.
env	dict of string to string
Environment variables to pass to actions. Includes GOARCH, GOOS, GOROOT, GOROOT_FINAL, CGO_ENABLED, and PATH.
tags	list of string
List of build tags used to filter source files.

Methods

• Action generators
  • archive
  • asm
  • binary
  • compile
  • cover
  • link
  • pack
• Helpers
  • args
  • declare_file
  • library_to_source
  • new_library

archive

This emits actions to compile Go code into an archive. It supports embedding, cgo dependencies, coverage, and assembling and packing .s files.

It returns a GoArchive.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
source	GoSource	mandatory value
The GoSource that should be compiled into an archive.

asm

The asm function adds an action that runs go tool asm on a source file to produce an object, and returns the File of that object.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
source	File	mandatory value
A source code artifact to assemble. This must be a .s file that contains code in the platform neutral go assembly language.
hdrs	File iterable	[]
The list of .h files that may be included by the source.

binary

This emits actions to compile and link Go code into a binary. It supports embedding, cgo dependencies, coverage, and assembling and packing .s files.

It returns a tuple containing GoArchive, the output executable file, and a runfiles object.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
name	string	""
The base name of the generated binaries. Required if executable is not given.
source	GoSource	mandatory value
The GoSource that should be compiled and linked.
test_archives	list GoArchiveData	[]
List of archives for libraries under test. See link.
gc_linkopts	string_list	[]
Go link options.
version_file	File	None
Version file used for link stamping. See link.
info_file	File	None
Info file used for link stamping. See link.
executable	File	None
Optional output file to write. If not set, binary will generate an output file name based on name, the target platform, and the link mode.

compile

The compile function adds an action that compiles a list of source files into a package archive (.a file).

It does not return anything.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
sources	File iterable	mandatory value
An iterable of source code artifacts. These must be pure .go files, no assembly or cgo is allowed.
importpath	string	""
The import path this package represents. This is passed to the -p flag. When the actual import path is different than the source import path (i.e., when importmap is set in a go_library rule), this should be the actual import path.
archives	GoArchive iterable	[]
An iterable of all directly imported libraries. The action will verify that all directly imported libraries were supplied, not allowing transitive dependencies to satisfy imports. It will not check that all supplied libraries were used though.
out_lib	File	mandatory value
The archive file that should be produced.
out_export	File	None
File where extra information about the package may be stored. This is used by nogo to store serialized facts about definitions. In the future, it may be used to store export data (instead of the .a file).
gc_goopts	string_list	[]
Additional flags to pass to the compiler.
testfilter	string	"off"
Controls how files with a _test suffix are filtered. "off" - files with and without a _test suffix are compiled. "only" - only files with a _test suffix are compiled. "exclude" - only files without a _test suffix are compiled.
asmhdr	File	None
If provided, the compiler will write an assembly header to this file.

cover

The cover function adds an action that runs go tool cover on a set of source files to produce copies with cover instrumentation.

Returns a covered GoSource with the required source files process for coverage.

Note that this removes most comments, including cgo comments.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
source	GoSource	mandatory value
The source object to process. Any source files in the object that have been marked as needing coverage will be processed and substiuted in the returned GoSource.

link

The link function adds an action that runs go tool link on a library.

It does not return anything.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
archive	GoArchive	mandatory value
The library to link.
test_archives	GoArchiveData list	[]
List of archives for libraries under test. These are excluded from linking if transitive dependencies of archive have the same package paths. This is useful for linking external test archives that depend internal test archives.
executable	File	mandatory value
The binary to produce.
gc_linkopts	string_list	[]
Basic link options, these may be adjusted by the mode.
version_file	File	None
Version file used for link stamping.
info_file	File	None
Info file used for link stamping.

pack

The pack function adds an action that produces an archive from a base archive and a collection of additional object files.

It does not return anything.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
in_lib	File	mandatory value
The archive that should be copied and appended to. This must always be an archive in the common ar form (like that produced by the go compiler).
out_lib	File	mandatory value
The archive that should be produced. This will always be an archive in the common ar form (like that produced by the go compiler).
objects	File iterable	()
An iterable of object files to be added to the output archive file.
archives	list of File	[]
Additional archives whose objects will be appended to the output. These can be ar files in either common form or either the bsd or sysv variations.

args

This creates a new Args object, using the ctx.actions.args method. The object is pre-populated with standard arguments used by all the go toolchain builders.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.

declare_file

This is the equivalent of ctx.actions.declare_file. It uses the current build mode to make the filename unique between configurations.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
path	string	""
A path for this file, including the basename of the file.
ext	string	""
The extension to use for the file.
name	string	""
A name to use for this file. If path is not present, this becomes a prefix to the path. If this is not set, the current rule name is used in it's place.

library_to_source

This is used to build a GoSource object for a given GoLibrary in the current build mode.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
attr	ctx.attr	mandatory value
The attributes of the rule being processed. In a normal rule implementation this would be ctx.attr.
library	GoLibrary	mandatory value
The GoLibrary that you want to build a GoSource object for in the current build mode.
coverage_instrumented	bool	mandatory value
This controls whether cover is enabled for this specific library in this mode. This should generally be the value of ctx.coverage_instrumented()

new_library

This creates a new GoLibrary. You can add extra fields to the go library by providing extra named parameters to this function, they will be visible to the resolver when it is invoked.

Name	Type	Default value
go	GoContext	mandatory value
This must be the same GoContext object you got this function from.
resolver	function	None
This is the function that gets invoked when converting from a GoLibrary to a GoSource. The function's signature must be def _testmain_library_to_source(go, attr, source, merge) attr is the attributes of the rule being processed source is the dictionary of GoSource fields being generated merge is a helper you can call to merge
importable	bool	mandatory value
This controls whether the GoLibrary is supposed to be importable. This is generally only false for the "main" libraries that are built just before linking.

go_rule

go_rule is a wrapper for the rule function. It's used to declare rules that have access to the Go toolchain and SDK.

Example:

def _my_rule_impl(ctx):
    go = go_context(ctx)
    ...

my_rule = go_rule(
    _my_rule_impl,
    attrs = {
        ...
    },
)

Name	Type	Default value
implementation	function	mandatory value
The implementation function for the rule.
attrs	dict	{}
Attributes the rule accepts. Additional implicit attributes may be added to this list.
toolchains	list	See below
List of toolchains types that Bazel should select when invoking this rule. "@io_bazel_rules_go//go:toolchain" is automatically added to this list, so that does not need to be specified explicitly.