How to properly use dockerfile_image?

[ghost]

I found myself in this problem by the following set of circumstances:

• I need to build a Docker image for a sidecar container
• the whole service along with the sidecar runs in a Kubernetes cluster, where I am not an admin
• I can run without issues anything that calls docker build, but not docker run
• container_run_and_commit_layer runs docker run :(

On top of that, this is all in rules_docker 0.15.0. I can't update right now, but if it takes that, I'll update.

For a while, we had this sidecar based on Debian and we had a bunch of tools that would install Debian packages into a layer and spit the finished product as a layer, where we can then stitch up the base image and layers with container_image.

I can no longer rely on Debian because the network landscape and policy changed - I can only re-create it with RHEL. No big issue, I whipped up a mock Dockerfile for that in a few minutes, and the end result works fine. However, it won't work in my CICD pipeline, which is contained entirely in the same Kubernetes cluster in another namespace dedicated to CICD operations. We have means of using docker there, but with limited powers as stated before.

My initial rewrite of the sidecar looked like this in BUILD.bazel:

package(default_visibility = ["//visibility:public"])
load(
# @io_bazel_rules_docker is provided by the WORKSPACE through http_archive
    "@io_bazel_rules_docker//container:container.bzl",
    "container_image",
    "container_load"
)
load(
    "@io_bazel_rules_docker//docker/util:run.bzl",
    "container_run_and_commit_layer"
)

container_run_and_commit_layer(
    name = "sidecar_on_rhel",
# rhel8 is provided by the WORKSPACE through container_pull
    image = "@rhel8//image",
    commands = [
# here's all the magic
    ]
)

container_image(
    name = "sidecar_image",
    base = "@rhel8//image",
    layers = [
        ":sidecar_on_rhel"
    ],
    entrypoint = ["/cool/stuff"]
)

So while that worked on my work laptop, it can't in CICD. I have searched the source code of rules_docker and found that my solution is hidden in contrib/dockerfile_build.bzl's dockerfile_image. So now, I have rewritten this thing as such:

package(default_visibility = ["//visibility:public"])
load(
    "@io_bazel_rules_docker//container:container.bzl",
    "container_image",
    "container_load"
)
load(
    "@io_bazel_rules_docker//contrib:dockerfile_build.bzl",
    "dockerfile_image"
)

dockerfile_image(
    name = "sidecar_dockerfile_image",
    dockerfile = ":Dockerfile"
)

container_load(
    name = "sidecar_loaded",
    file = ":sidecar_dockerfile_image//image:dockerfile_image.tar"
)

container_image(
    name = "sidecar_image",
    base = ":sidecar_loaded//image",
    entrypoint = ["/cool/stuff"]
)

With this, I'm getting the following problems:

java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:target' (requested by nodes '//mycoolservice:sidecar_image, excludedSubdirs=[], filteringPolicy=com.google.devtools.build.lib.pkgcache.FilteringPolicies$FilterManual@xxxxxxxx')

I tried to look for documentation for dockerfile_image and found that rules_docker has no such thing. There's no contrib/README.md. I tried to look for example usage of dockerfile_image in the code and the stuff in tests and testing isn't really helping. In fact, my rewrite is based on the stuff I have found there.
In case you're wondering about the Java exception, it's because BUILD.bazel for this is in fact in the target called mycoolservice. And xxxxxxxx is just random 32bit hex integer.

Can someone please help? The major issue is that I'm suffering from lack of understanding of how dockerfile_image is supposed to be used, which comes from lack of documentation. I'd appreciate if my short-term problem of fixing my BUILD.bazel would be fixed though.

[bweston92]

Did you manage to solve this? I am too finding it hard to find any docs.

[manuelnaranjo]

So the idea behind rules_docker is to not require a docker context and a dockerfile, for the RHEL case you need to use bazeldnf and install the RPMs through it, but take into account that it will not execute any of the post install scripts. And there are some caveats with some packages that can't be installed so easily.

[bweston92]

Thanks for your response. I managed to get it working, on this occasion I needed a few libraries inside the image for my app to work. I don't know how this would be achievable without a custom base image being built somewhere, that somewhere for me was best along side the application code which is why I came to use the dockerfile_image repository rule.

Would it be worth me/anyone expanding the documentation showing the use of the repository_rule?

[manuelnaranjo]

This repo is in kind of maintenance/archive mode, rules_oci is where all the fun is happening right now, adding dependencies to images is not something a lot of folks though, most of the time you build some sort of binary with bazel (even if it's not a binary but something that's entirely self-contained) and that gets added to the image.

At Booking we have an entire setup where we start from scratch and we install the required RPMs with bazeldnf into some of our base images, but it's not open source and I've never taken the time to make a smaller repo to show how it works