Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build: Make distroless image default and publish mimir-alpine #8204

Merged
merged 4 commits into from
May 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/scripts/build-images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ do
make \
BUILD_IN_CONTAINER=false \
PUSH_MULTIARCH_TARGET="type=oci,dest=$OUTPUT/$NAME.oci" \
PUSH_MULTIARCH_TARGET_DISTROLESS="type=oci,dest=$OUTPUT/$NAME\-distroless.oci" \
PUSH_MULTIARCH_TARGET_ALPINE="type=oci,dest=$OUTPUT/$NAME\-alpine.oci" \
PUSH_MULTIARCH_TARGET_CONTINUOUS_TEST="type=oci,dest=$OUTPUT/$NAME\-continuous\-test.oci" \
push-multiarch-$target
done
16 changes: 8 additions & 8 deletions .github/workflows/test-build-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -281,15 +281,15 @@ jobs:
- name: Build Mimir with race-detector
run: |
# When building uptodate_race target, we create two images since
# a Dockerfile.distroless exists. The distroless image is built with
# the `-distroless` suffix.
# a Dockerfile.alpine exists. The alpine image is built with
# the `-alpine` suffix.
# We build both until we have finished migrating to distroless.
# We test the distroless race image in every integration test.
# We test the (legacy) alpine race image when deploying (aka pushing).
make BUILD_IN_CONTAINER=false cmd/mimir/.uptodate_race
export IMAGE_TAG_RACE=$(make image-tag-race)
export MIMIR_DISTROLESS_IMAGE="grafana/mimir-distroless:$IMAGE_TAG_RACE"
export MIMIR_ALPINE_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export MIMIR_DISTROLESS_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export MIMIR_ALPINE_IMAGE="grafana/mimir-alpine:$IMAGE_TAG_RACE"
docker save $MIMIR_DISTROLESS_IMAGE -o ./mimir_race_image_distroless
docker save $MIMIR_ALPINE_IMAGE -o ./mimir_race_image_alpine
- name: Upload archive with race-enabled Mimir
Expand Down Expand Up @@ -355,7 +355,7 @@ jobs:
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
docker load -i ./mimir_race_image_distroless
docker run "grafana/mimir-distroless:$IMAGE_TAG_RACE" --version
docker run "grafana/mimir:$IMAGE_TAG_RACE" --version
- name: Preload Images
# We download docker images used by integration tests so that all images are available
# locally and the download time doesn't account in the test execution time, which is subject
Expand All @@ -364,7 +364,7 @@ jobs:
- name: Integration Tests
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
export MIMIR_IMAGE="grafana/mimir-distroless:$IMAGE_TAG_RACE"
export MIMIR_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export IMAGE_TAG=$(make image-tag)
export MIMIRTOOL_IMAGE="grafana/mimirtool:$IMAGE_TAG"
export MIMIR_CHECKOUT_DIR="/go/src/github.com/grafana/mimir"
Expand Down Expand Up @@ -428,7 +428,7 @@ jobs:
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
docker load -i ./mimir_race_image_alpine
docker run "grafana/mimir:$IMAGE_TAG_RACE" --version
docker run "grafana/mimir-alpine:$IMAGE_TAG_RACE" --version
- name: Preload Images
# We download docker images used by integration tests so that all images are available
# locally and the download time doesn't account in the test execution time, which is subject
Expand All @@ -437,7 +437,7 @@ jobs:
- name: Integration Tests
run: |
export IMAGE_TAG_RACE=$(make image-tag-race)
export MIMIR_IMAGE="grafana/mimir:$IMAGE_TAG_RACE"
export MIMIR_IMAGE="grafana/mimir-alpine:$IMAGE_TAG_RACE"
export IMAGE_TAG=$(make image-tag)
export MIMIRTOOL_IMAGE="grafana/mimirtool:$IMAGE_TAG"
export MIMIR_CHECKOUT_DIR="/go/src/github.com/grafana/mimir"
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### Grafana Mimir

* [CHANGE] Build: `grafana/mimir` docker image is now based on `gcr.io/distroless/static-debian12` image. Alpine-based docker image is still available as `grafana/mimir-alpine`, until Mimir 2.15. #8204
* [CHANGE] Ingester: `/ingester/flush` endpoint is now only allowed to execute only while the ingester is in `Running` state. The 503 status code is returned if the endpoint is called while the ingester is not in `Running` state. #7486
* [CHANGE] Distributor: Include label name in `err-mimir-label-value-too-long` error message: #7740
* [CHANGE] Ingester: enabled 1 out 10 errors log sampling by default. All the discarded samples will still be tracked by the `cortex_discarded_samples_total` metric. The feature can be configured via `-ingester.error-sample-rate` (0 to log all errors). #7807
Expand Down
52 changes: 32 additions & 20 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -103,11 +103,11 @@ SED ?= $(shell which gsed 2>/dev/null || which sed)
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-continuous-test:$(IMAGE_TAG) $(@D)/; \
fi;
if [ -f $(@D)/Dockerfile.distroless ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.distroless \
if [ -f $(@D)/Dockerfile.alpine ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.alpine \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG) $(@D)/; \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG) $(@D)/; \
fi;
@echo
$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) -t $(IMAGE_PREFIX)$(shell basename $(@D)) -t $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG) $(@D)/
Expand All @@ -117,7 +117,7 @@ SED ?= $(shell which gsed 2>/dev/null || which sed)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-continuous-test:$(IMAGE_TAG)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG)
@echo
@echo Please use '"make push-multiarch-build-image"' to build and push build image.
@echo Please use '"make push-multiarch-mimir"' to build and push Mimir image.
Expand All @@ -126,32 +126,38 @@ SED ?= $(shell which gsed 2>/dev/null || which sed)

%/$(UPTODATE_RACE): GOOS=linux
%/$(UPTODATE_RACE): %/Dockerfile
# Build Dockerfile.distroless if it exists, we use distroless/base-nossl-debian12 as base image since race detector needs glibc packages.
if [ -f $(@D)/Dockerfile.distroless ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.distroless \
# Build Dockerfile.alpine if it exists
if [ -f $(@D)/Dockerfile.alpine ]; then \
$(SUDO) docker build -f $(@D)/Dockerfile.alpine \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
--build-arg=BINARY_SUFFIX=_race \
--build-arg=BASEIMG="gcr.io/distroless/base-nossl-debian12" \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG_RACE) $(@D)/; \
--build-arg=EXTRA_PACKAGES="gcompat" \
-t $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG_RACE) $(@D)/; \
fi;
@echo
# We need gcompat -- compatibility layer with glibc, as race-detector currently requires glibc, but Alpine uses musl libc instead.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment should be moved to "alpine" version. For distroless image, we should restore comment about using distroless/base-nossl-debian12 as base image.

$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) --build-arg=USE_BINARY_SUFFIX=true --build-arg=BINARY_SUFFIX=_race --build-arg=EXTRA_PACKAGES="gcompat" -t $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG_RACE) $(@D)/
$(SUDO) docker build \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
--build-arg=BINARY_SUFFIX=_race \
--build-arg=BASEIMG="gcr.io/distroless/base-nossl-debian12" \
-t $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG_RACE) $(@D)/
@echo
@echo Go binaries were built using GOOS=$(GOOS) and GOARCH=$(GOARCH)
@echo
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG_RACE)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-distroless:$(IMAGE_TAG_RACE)
@echo Image name: $(IMAGE_PREFIX)$(shell basename $(@D))-alpine:$(IMAGE_TAG_RACE)
@echo
@touch $@

# This variable controls where result of building of multiarch image should be sent. Default is registry.
# Other options are documented in https://docs.docker.com/engine/reference/commandline/buildx_build/#output.
# CI workflow uses PUSH_MULTIARCH_TARGET="type=oci,dest=file.oci" to store images locally for next steps in the pipeline.
PUSH_MULTIARCH_TARGET ?= type=registry
PUSH_MULTIARCH_TARGET_DISTROLESS ?= type=registry
PUSH_MULTIARCH_TARGET_ALPINE ?= type=registry
PUSH_MULTIARCH_TARGET_CONTINUOUS_TEST ?= type=registry

# This target compiles mimir for linux/amd64 and linux/arm64 and then builds and pushes a multiarch image to the target repository.
Expand All @@ -164,7 +170,13 @@ push-multiarch-%/$(UPTODATE):
$(MAKE) GOOS=linux GOARCH=amd64 BINARY_SUFFIX=_linux_amd64 $(DIR)/$(shell basename $(DIR)); \
$(MAKE) GOOS=linux GOARCH=arm64 BINARY_SUFFIX=_linux_arm64 $(DIR)/$(shell basename $(DIR)); \
fi
$(SUDO) docker buildx build -o $(PUSH_MULTIARCH_TARGET) --platform linux/amd64,linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) --build-arg=USE_BINARY_SUFFIX=true -t $(IMAGE_PREFIX)$(shell basename $(DIR)):$(IMAGE_TAG) $(DIR)/
$(SUDO) docker buildx build \
-o $(PUSH_MULTIARCH_TARGET) \
--platform linux/amd64,linux/arm64 \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
-t $(IMAGE_PREFIX)$(shell basename $(DIR)):$(IMAGE_TAG) $(DIR)/

# Build Dockerfile.continuous-test
if [ -f $(DIR)/Dockerfile.continuous-test ]; then \
Expand All @@ -176,15 +188,15 @@ push-multiarch-%/$(UPTODATE):
--build-arg=USE_BINARY_SUFFIX=true \
-t $(IMAGE_PREFIX)$(shell basename $(DIR))-continuous-test:$(IMAGE_TAG) $(DIR)/; \
fi;
# Build Dockerfile.distroless if it exists
if [ -f $(DIR)/Dockerfile.distroless ]; then \
$(SUDO) docker buildx build -f $(DIR)/Dockerfile.distroless \
-o $(PUSH_MULTIARCH_TARGET_DISTROLESS) \
# Build Dockerfile.alpine if it exists
if [ -f $(DIR)/Dockerfile.alpine ]; then \
$(SUDO) docker buildx build -f $(DIR)/Dockerfile.alpine \
-o $(PUSH_MULTIARCH_TARGET_ALPINE) \
--platform linux/amd64,linux/arm64 \
--build-arg=revision=$(GIT_REVISION) \
--build-arg=goproxyValue=$(GOPROXY_VALUE) \
--build-arg=USE_BINARY_SUFFIX=true \
-t $(IMAGE_PREFIX)$(shell basename $(DIR))-distroless:$(IMAGE_TAG) $(DIR)/; \
-t $(IMAGE_PREFIX)$(shell basename $(DIR))-alpine:$(IMAGE_TAG) $(DIR)/; \
fi;

push-multiarch-mimir: ## Push mimir docker image.
Expand Down Expand Up @@ -740,8 +752,8 @@ integration-tests: ## Run all integration tests.
integration-tests: cmd/mimir/$(UPTODATE)
go test -tags=requires_docker,stringlabels ./integration/...

integration-tests-race: ## Run all integration tests with race-enabled Mimir-distroless docker image.
integration-tests-race: export MIMIR_IMAGE=$(IMAGE_PREFIX)mimir-distroless:$(IMAGE_TAG_RACE)
integration-tests-race: ## Run all integration tests with race-enabled distroless docker image.
integration-tests-race: export MIMIR_IMAGE=$(IMAGE_PREFIX)mimir:$(IMAGE_TAG_RACE)
integration-tests-race: cmd/mimir/$(UPTODATE_RACE)
go test -timeout 30m -tags=requires_docker,stringlabels ./integration/...

Expand Down
10 changes: 4 additions & 6 deletions cmd/mimir/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Provenance-includes-location: https://github.com/cortexproject/cortex/cmd/cortex/Dockerfile
# Provenance-includes-license: Apache-2.0
# Provenance-includes-copyright: The Cortex Authors.
# We use different base images for mimir and mimir race images since race-detector needs glibc packages.
# See difference between distroless static and base-nossl at https://github.com/GoogleContainerTools/distroless/blob/main/base/README.md.

FROM alpine:3.19.1
ARG EXTRA_PACKAGES
RUN apk add --no-cache ca-certificates tzdata $EXTRA_PACKAGES
ARG BASEIMG=gcr.io/distroless/static-debian12
FROM ${BASEIMG}
# Expose TARGETOS and TARGETARCH variables. These are supported by Docker when using BuildKit, but must be "enabled" using ARG.
ARG TARGETOS
ARG TARGETARCH
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# SPDX-License-Identifier: AGPL-3.0-only
# We use different base images for mimir and mimir race images since race-detector needs glibc packages.
# See difference between distroless static and base-nossl at https://github.com/GoogleContainerTools/distroless/blob/main/base/README.md.
# Provenance-includes-location: https://github.com/cortexproject/cortex/cmd/cortex/Dockerfile
# Provenance-includes-license: Apache-2.0
# Provenance-includes-copyright: The Cortex Authors.

ARG BASEIMG=gcr.io/distroless/static-debian12
FROM ${BASEIMG}
FROM alpine:3.19.1
ARG EXTRA_PACKAGES
RUN apk add --no-cache ca-certificates tzdata $EXTRA_PACKAGES
# Expose TARGETOS and TARGETARCH variables. These are supported by Docker when using BuildKit, but must be "enabled" using ARG.
ARG TARGETOS
ARG TARGETARCH
Expand Down
Loading