fix(druid): CVE-2023-34455

[razvan]

Description

See: https://github.com/stackabletech/vulnerabilities/issues/558

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Tasks

Preview Give feedback

1. Changes are OpenShift compatible
2. All added packages (via microdnf or otherwise) have a comment on why they are added
3. Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
4. All packages should have (if available) signatures/hashes verified
5. Add an entry to the CHANGELOG.md file
6. Integration tests ran successfully

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

bake --product <product> --image-version <stackable-image-version>
kind load docker-image <image-tagged-with-the-major-version> --name=<name-of-your-test-cluster>

See the output of bake to retrieve the image tag for <image-tagged-with-the-major-version>.

[NickLarsenNZ]

LGTM

[lfrancke]

I don't think this'll work properly as it will not be reflected properly in the SBOMs.

Usually you'd do this with exclusions in the Maven POM file. Did you try that?

[razvan]

I don't think this'll work properly as it will not be reflected properly in the SBOMs.

LGTM though. The broken jar is not listed in the SBOM file.

$ docker run --rm -ti docker.stackable.tech/stackable/druid:30.0.0-stackable0.0.0-dev bash
stackable@ab6cdeb5d499 /stackable/apache-druid-30.0.0 $ grep snappy-java apache-druid-30.0.0.cdx.json
      "name" : "snappy-java",
      "description" : "snappy-java: A fast compression/decompression library",
      "purl" : "pkg:maven/org.xerial.snappy/snappy-java@1.1.10.4?type=jar",
          "url" : "https://github.com/xerial/snappy-java"
          "url" : "https://github.com/xerial/snappy-java"
      "bom-ref" : "pkg:maven/org.xerial.snappy/snappy-java@1.1.10.4?type=jar"
        "pkg:maven/org.xerial.snappy/snappy-java@1.1.10.4?type=jar",
      "ref" : "pkg:maven/org.xerial.snappy/snappy-java@1.1.10.4?type=jar",
stackable@ab6cdeb5d499 /stackable/apache-druid-30.0.0 $

Also Trivy doesn't report the CVE anymore after building the image.

Usually you'd do this with exclusions in the Maven POM file. Did you try that?

The Druid pom file already contains the newer (1.1.10.4) snappy version.

The vulnerable version is pulled in by the dist profile when executing pull-deps step. This step pulls the hadoop-client (and the vulnerable snappy jar) directly from a remote Maven repository:

<execution>
    <id>pull-deps</id>
    <phase>package</phase>
    <goals>
        <goal>exec</goal>
    </goals>
    <configuration>
        <executable>${project.parent.basedir}/examples/bin/run-java</executable>
        <arguments>
            ...
            <argument>org.apache.druid.cli.Main</argument>
            <argument>tools</argument>
            <argument>pull-deps</argument>
            ...
            <argument>-h</argument>
            <argument>org.apache.hadoop:hadoop-client-api:${hadoop.compile.version</argument>
             <argument>-h</argument>
            <argument>org.apache.hadoop:hadoop-client-runtime:${hadoop.compile.version</argument>
            ...

We would need a patched version of the hadoop-client to be published to our own Maven repository.
Alternatively, the pull-deps tool aparently also supports --no-default-hadoop. This would need a separate investigation.

Anyway, after that step, the following snappy versions are pulled in:

stackable@39a75316efb2 /stackable/apache-druid-30.0.0 $ find /stackable/druid/ -name '*snappy*'
/stackable/druid/extensions/druid-avro-extensions/snappy-java-1.1.10.4.jar
/stackable/druid/extensions/druid-hdfs-storage/snappy-java-1.1.10.4.jar
/stackable/druid/extensions/druid-kafka-extraction-namespace/snappy-java-1.1.10.4.jar
/stackable/druid/extensions/druid-kafka-indexing-service/snappy-java-1.1.10.4.jar
/stackable/druid/extensions/druid-kerberos/snappy-java-1.1.10.4.jar
/stackable/druid/extensions/druid-orc-extensions/snappy-java-1.1.10.4.jar
/stackable/druid/extensions/druid-parquet-extensions/snappy-java-1.1.10.4.jar
/stackable/druid/hadoop-dependencies/hadoop-client-api/3.3.6/snappy-java-1.1.8.2.jar
/stackable/druid/hadoop-dependencies/hadoop-client-runtime/3.3.6/snappy-java-1.1.8.2.jar
/stackable/druid/lib/snappy-java-1.1.10.4.jar

This PR adds another execution step after the pull-deps above that removes the 1.1.8.2 version again.

Related to this, Druid has jar deduplication script they use in their Dockerfile. We should use that to further reduce the final image size.

[dervoeti]

Yeah I think in general @lfrancke is right, but this case is special, because these dependencies are not directly pulled in by Maven. They are not added to the SBOM that Maven generates because Maven does not seem to be aware of them. They are currently added to the final SBOM of the container image through the JAR cataloger of Syft (it scans the filesystem after the image is built and detects the JAR files). So even in the current Druid 30 image, they are not in the apache-druid-30.0.0.cdx.json file (that's just the one generated by Maven, not the final SBOM for the image).
So the approach @razvan is taking is fine I think.

But yes, we could think about pulling in Hadoop 3.4.0 or 3.4.1 instead of 3.3.6 to get rid of this snappy version and some other vulnerable dependencies as well. Hadoop 3.4.1 depends on snappy 1.1.10.4 for example, the same version that Druid depends on.