Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(hadoop): Fix the JMX exporter configuration #962

Merged
merged 2 commits into from
Dec 20, 2024

Conversation

siegfriedweber
Copy link
Member

@siegfriedweber siegfriedweber commented Dec 18, 2024

Description

hadoop: Fix the JMX exporter configuration for metrics suffixed with _total, _info and _created.

The type of the metrics suffixed with _total is changed from GAUGE to COUNTER.
The names of the metrics suffixed with _info and _created are now appended with an additional _.

Fixes stackabletech/hdfs-operator#634

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Tasks

Preview Give feedback
TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

bake --product <product> --image-version <stackable-image-version>
kind load docker-image <image-tagged-with-the-major-version> --name=<name-of-your-test-cluster>

See the output of bake to retrieve the image tag for <image-tagged-with-the-major-version>.

@razvan
Copy link
Member

razvan commented Dec 19, 2024

If we copy these files in the final stage we can reuse the layer cache better. WDYT?

diff --git a/hadoop/Dockerfile b/hadoop/Dockerfile
index 0b68186..acb4e09 100644
--- a/hadoop/Dockerfile
+++ b/hadoop/Dockerfile
@@ -11,10 +11,7 @@ ARG TARGETARCH
 ARG TARGETOS
 ARG STACKABLE_USER_UID
 
-WORKDIR /stackable
-
-COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/jmx /stackable/jmx
-COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/fuse_dfs_wrapper /stackable/fuse_dfs_wrapper
+WORKDIR /stackable/jmx
 
 # The symlink from JMX Exporter 0.16.1 to the versionless link exists because old HDFS Operators (up until and including 23.7) used to hardcode
 # the version of JMX Exporter like this: "-javaagent:/stackable/jmx/jmx_prometheus_javaagent-0.16.1.jar"
@@ -27,6 +24,8 @@ RUN curl "https://repo.stackable.tech/repository/packages/jmx-exporter/jmx_prome
     ln -s "/stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar" /stackable/jmx/jmx_prometheus_javaagent.jar && \
     ln -s /stackable/jmx/jmx_prometheus_javaagent.jar /stackable/jmx/jmx_prometheus_javaagent-0.16.1.jar
 
+WORKDIR /stackable
+
 RUN ARCH="${TARGETARCH/amd64/x64}" && \
     curl "https://repo.stackable.tech/repository/packages/async-profiler/async-profiler-${ASYNC_PROFILER}-${TARGETOS}-${ARCH}.tar.gz"  | tar -xzC . && \
     ln -s "/stackable/async-profiler-${ASYNC_PROFILER}-${TARGETOS}-${ARCH}" /stackable/async-profiler
@@ -141,6 +140,7 @@ COPY --chown=${STACKABLE_USER_UID}:0 --from=hadoop-builder /stackable/jmx /stack
 COPY --chown=${STACKABLE_USER_UID}:0 --from=hadoop-builder /stackable/async-profiler /stackable/async-profiler/
 COPY --chown=${STACKABLE_USER_UID}:0 --from=hdfs-utils-builder /stackable/hadoop-${PRODUCT}/share/hadoop/common/lib/hdfs-utils-${HDFS_UTILS}.jar /stackable/hadoop-${PRODUCT}/share/hadoop/common/lib/hdfs-utils-${HDFS_UTILS}.jar
 COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/fuse_dfs_wrapper /stackable/
+COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/jmx /stackable/jmx
 
 
 # fuse is required for fusermount (called by fuse_dfs)

razvan
razvan previously approved these changes Dec 19, 2024
Copy link
Member

@razvan razvan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. tested with the patch in comments.

@siegfriedweber siegfriedweber added this pull request to the merge queue Dec 20, 2024
Merged via the queue into main with commit 7e34d80 Dec 20, 2024
3 checks passed
@siegfriedweber siegfriedweber deleted the fix_hadoop/metrics branch December 20, 2024 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Metrics with specific suffixes are renamed or not exported at all
2 participants