Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variation of #214 (Rule to generate docker images with nix store paths) #351

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions containers/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package(default_visibility = ["//visibility:public"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One problem is the unquoted ~ as discussed above. The linked workaround is the correct fix.

Another issue with bzlmod mode will be that this containers package doesn't fit into any of the new rules_nixpkgs Bazel modules (see #181, in particular #182 and #322). As laid out there rules_nixpkgs has been split into multiple sub-modules such as rules_nixpkgs_core under core/ or rules_nixpkgs_cc under toolchains/cc.

The motivation for this split is to avoid rules_nixpkgs turning into a Bazel module that transitively depends on almost all Bazel modules out there.

For the specific rule added here, it doesn't actually introduce any new Bazel module dependencies. The fact that it generates a Docker image might suggest that it should belong into a dedicated rules_nixpkgs_containers or rules_nixpkgs_docker Bazel module. But, since it only depends on functionality in rules_nixpkgs_core and Nix itself, it can actually be part of rules_nixpkgs_core.

@benradf any thoughts on this? My inclination would be to go with the simpler route of just placing this into core/ intead of introducing all the overhead of a new dedicated rules_nixpkgs_containers module when it's not really necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benradf any thoughts on this? My inclination would be to go with the simpler route of just placing this into core/ intead of introducing all the overhead of a new dedicated rules_nixpkgs_containers module when it's not really necessary.

I agree, there's no compelling reason for a new module right now, so putting it in rules_nixpkgs_core makes most sense. It can always be moved later if additional module dependencies are required.


load("@bazel_skylib//:bzl_library.bzl", "bzl_library")

exports_files([
"docker/stream.sh",
])

bzl_library(
name = "docker",
srcs = [
"docker.bzl",
],
visibility = ["//visibility:public"],
)
104 changes: 104 additions & 0 deletions containers/docker.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
"""
# Docker containerization Bazel Nixpkgs rules

To run bazel artifacts across systems and platforms, nixpkgs_rules exposed a
docker hook. e.g.

```starlark
nixpkgs_docker_image(
name = "nix_deps_image",
srcs = [
"@cc_toolchain_nixpkgs_info////bazel-support",
"@nixpkgs_python_toolchain_python3//bazel-support",
"@nixpkgs_sh_posix_config//bazel-support",
"@rules_haskell_ghc_nixpkgs//bazel-support",
"@nixpkgs_valgrind//bazel-support",
],
bazel = "@nixpkgs_bazel//bazel-support",
repositories = {"nixpkgs": "@nixpkgs"},
)
```

here, nixpkgs rules dependencies are bundled into a docker container for use and
deployment.

"""

def _nixpkgs_docker_image_impl(repository_ctx):
repositories = repository_ctx.attr.repositories

nix_build_bin = repository_ctx.which("nix-build")
repository_ctx.symlink(nix_build_bin, "nix-build")

srcs = repository_ctx.attr.srcs
bazel = repository_ctx.attr.bazel

# HACK! On bazel from nixpkgs, shebangs get mangled in things like
# @bazel_tools//tools/cpp:linux_cc_wrapper.sh.tpl, so that nix store
# paths end up referenced there.
# One needs to ensure those paths are available on the docker image
# as well, we can do that by including bazel
srcs = srcs + [bazel] if bazel else srcs

contents = []
for src in srcs:
path_to_default_nix = repository_ctx.path(src.relative("default.nix"))
package = "bazel-support/%s" % src.workspace_name
repository_ctx.symlink(path_to_default_nix.dirname, package)
contents.append("(import ./%s {})" % package)

repository_ctx.template(
"default.nix",
Label("@io_tweag_rules_nixpkgs//containers:docker/default.nix.tpl"),
substitutions = {
"%{name}": repr(repository_ctx.name),
"%{contents}": "\n ".join(contents),
},
executable = False,
)

args = list(repository_ctx.attr.nixopts)
for repo_label, repo_name in repositories.items():
absolute_repo = repository_ctx.path(repo_label).dirname

# Excessive quoting due to nix limitations for ~ in file path
# (see NixOS/nix#7742).
args.extend([
'"-I"',
"\"%s=\\\"%s\\\"\"" % (repo_name, absolute_repo),
])

repository_ctx.template(
"BUILD",
Label("@io_tweag_rules_nixpkgs//containers:docker/BUILD.bazel.tpl"),
substitutions = {
"%{args_comma_sep}": ",\n ".join(args),
"%{args_space_sep}": " ".join(args),
},
executable = False,
)

_nixpkgs_docker_image = repository_rule(
implementation = _nixpkgs_docker_image_impl,
attrs = {
"nixopts": attr.string_list(),
"repositories": attr.label_keyed_string_dict(),
"srcs": attr.label_list(
doc = 'List of nixpkgs_package to include in the image. E.g. ["@hello//nixpkg"]',
),
"bazel": attr.label(
doc = """If using bazel from nixpkgs, this a nixpackage_package
based on exactly the same bazel derivation. This is to ensure any paths
for mangled '/usr/env bash' introduced by nix exist in the store.
Example: '<nixpkgs>.bazel_4'.
""",
),
},
)

def nixpkgs_docker_image(name, **kwargs):
repositories = kwargs.get("repositories")
if repositories:
inversed_repositories = {value: key for key, value in repositories.items()}
kwargs["repositories"] = inversed_repositories
_nixpkgs_docker_image(name = name, **kwargs)
29 changes: 29 additions & 0 deletions containers/docker/BUILD.bazel.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
sh_binary(
name = "stream",
srcs = ["@io_tweag_rules_nixpkgs//containers:docker/stream.sh"],
data = [
":default.nix",
":nix-build",
],
env = {"NIX_BUILD": "$(location :nix-build)"},
args = [
'"./$(location :default.nix)"',
%{args_comma_sep},
],
)

genrule(
name = "image",
srcs = [
":default.nix",
],
outs = ["image.tgz"],
exec_tools = [":nix-build"],
cmd = """
$(location :nix-build) %{args_space_sep} \
--arg stream false \
--out-link $@ \
"./$(location :default.nix)"
""",
local = True,
)
46 changes: 46 additions & 0 deletions containers/docker/default.nix.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{ stream ? true, tag ? null, nixpkgs ? import <nixpkgs> {} }:
let
dockerImage = if stream
then nixpkgs.dockerTools.streamLayeredImage
else nixpkgs.dockerTools.buildLayeredImage;

name = %{name};

contents = [
%{contents}
];

manifest = nixpkgs.writeTextFile
{ name = "${name}-MANIFEST";
text = nixpkgs.lib.strings.concatMapStrings (pkg: "${pkg}\n") contents;
destination = "/MANIFEST";
};

usr_bin_env = nixpkgs.runCommand "usr-bin-env" {} ''
mkdir -p "$out/usr/bin/"
ln -s "${nixpkgs.coreutils}/bin/env" "$out/usr/bin/"
'';
in
dockerImage {
inherit name tag;

contents = [
# Contents get copied to the top-level of the image, so we jus putt
# a short manifest file here and get all the store paths as dependencies
manifest

# Ensure "/usr/bin/env bash" works correctly
nixpkgs.bash
nixpkgs.coreutils
usr_bin_env

# avoid "commitBuffer: invalid argument (invalid character)" running tests
nixpkgs.glibcLocales
];

extraCommands = "mkdir -m 0777 tmp";

config = {
Cmd = [ "${nixpkgs.bashInteractive}/bin/bash" ];
};
}
7 changes: 7 additions & 0 deletions containers/docker/stream.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
set -euo pipefail

run() {
${NIX_BUILD} --arg stream true "$@"
}

$(run "$@")
13 changes: 13 additions & 0 deletions core/default.nix.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{ %{args_with_defaults} }:
let
value_or_function = %{def};
value =
if builtins.isFunction value_or_function then
let
formalArgs = builtins.functionArgs value_or_function;
actualArgs = builtins.intersectAttrs formalArgs { inherit %{args}; };
in
value_or_function actualArgs
else value_or_function;
in
value%{maybe_attr}
76 changes: 70 additions & 6 deletions core/nixpkgs.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -377,9 +377,35 @@ def nixpkgs_local_repository(
**kwargs
)

def _nixopts_args(nixopts):
result = {}
arg_opt, arg_name = None, None
for opt in nixopts:
if opt == "--arg" or opt == "--argstr":
arg_opt, arg_name = opt, None
elif arg_opt:
if arg_name == None:
arg_name = opt
else:
arg_val = opt if arg_opt == "--arg" else "''%s''" % opt
result[arg_name] = arg_val
arg_opt, arg_name = None, None
return result

def _sanitize_attribute_path(attribute_path):
# Wrap attribute path in strings for special characters if it exisits
maybe_attr = ""
if attribute_path:
attrs = []
for attr in attribute_path.split("."):
attrs.append("\"%s\"" % attr.split("~")[-1])
maybe_attr = "." + ".".join(attrs)
return maybe_attr

def _nixpkgs_package_impl(repository_ctx):
repository = repository_ctx.attr.repository
repositories = repository_ctx.attr.repositories
attribute_path = repository_ctx.attr.attribute_path

expr_args = []

Expand Down Expand Up @@ -453,26 +479,64 @@ def _nixpkgs_package_impl(repository_ctx):
else:
# No user supplied build file, we may create the default one.
create_build_file_if_needed = True

# Workaround to bazelbuild/bazel#4533
repository_ctx.path("BUILD")

if repository_ctx.attr.nix_file and repository_ctx.attr.nix_file_content:
fail("Specify one of 'nix_file' or 'nix_file_content', but not both.")
elif repository_ctx.attr.nix_file:

# Create a default.nix and BUILD file in bazel-support for external
# reference.
maybe_attr = _sanitize_attribute_path(attribute_path)
if repository_ctx.attr.nix_file:
nix_file = cp(repository_ctx, repository_ctx.attr.nix_file)
expr_args.append(repository_ctx.path(nix_file))
default_nix_substs = {
"%{def}": "import /${\"%s\"}" % repository_ctx.path(nix_file),
"%{maybe_attr}": maybe_attr,
}
elif repository_ctx.attr.nix_file_content:
expr_args.extend(["-E", repository_ctx.attr.nix_file_content])
default_nix_substs = {
"%{def}": repository_ctx.attr.nix_file_content,
"%{maybe_attr}": maybe_attr,
}
else:
expr_args.extend(["-E", "import <nixpkgs> { config = {}; overlays = []; }"])
default_nix_substs = {
"%{def}": "import <nixpkgs> { config = {}; overlays = []; }",
"%{maybe_attr}": maybe_attr if maybe_attr else _sanitize_attribute_path(repository_ctx.attr.name),
}

nix_file_deps = {}
for dep_lbl, dep_str in repository_ctx.attr.nix_file_deps.items():
nix_file_deps[dep_str] = cp(repository_ctx, dep_lbl)

nixopts = [
expand_location(
repository_ctx = repository_ctx,
string = opt,
labels = nix_file_deps,
attr = "nixopts",
)
for opt in repository_ctx.attr.nixopts
]
nixopts_args = _nixopts_args(nixopts)
default_nix_substs["%{args_with_defaults}"] = ", ".join([
"%s ? %s" % kv
for kv in nixopts_args.items()
])
default_nix_substs["%{args}"] = " ".join(nixopts_args.keys())

repository_ctx.template(
"bazel-support/default.nix",
Label("@rules_nixpkgs_core//:default.nix.tpl"),
substitutions = default_nix_substs,
executable = False,
)

repository_ctx.file("bazel-support/BUILD", 'exports_files(["nix-out-link"])\nfilegroup(name="bazel-support", srcs=glob(["nix-out-link*/**/*"], exclude=["BUILD"]))')

expr_args.extend([
"-A",
repository_ctx.attr.attribute_path if repository_ctx.attr.nix_file or repository_ctx.attr.nix_file_content else repository_ctx.attr.attribute_path or repository_ctx.attr.unmangled_name,
repository_ctx.path("bazel-support/default.nix"),
# Creating an out link prevents nix from garbage collecting the store path.
# nixpkgs uses `nix-support/` for such house-keeping files, so we mirror them
# and use `bazel-support/`, under the assumption that no nix package has
Expand Down