ODC container fails to connect to "central"

[elafontaine]

Describe the bug
Since the last update, our jobs have been failing to execute the "Central Analyzer" ;

[ERROR] Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:363)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by: java.io.IOException: Could not connect to MavenCentral (504): Gateway Time-out
	at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:232)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:340)
	... 8 common frames omitted

Version of dependency-check used
latest

Log file

[ERROR] Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:363)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by: java.io.IOException: Could not connect to MavenCentral (504): Gateway Time-out
	at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:232)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:340)
	... 8 common frames omitted

To Reproduce
Steps to reproduce the behavior:

1. compile your java jars under target
2. use the container to analyse the local directory (docker run -it -v $(pwd):/data sh)
3. execute this ; /usr/share/dependency-check/bin/dependency-check.sh --failOnCVSS 5 --noupdate --out "target" --scan 'target/**/*.jar' ${SUPPRESSION_FILE_PATH:+--suppression "$SUPPRESSION_FILE_PATH"}
4. Observe error

Expected behavior
No error OR errors because of vulnerabilities found in the jars.

Additional context
This is ran in a pipeline with internet access, so the error makes no sense to me... I will try to confirm the direct connectivity to the default URI ;

                .addOption(newOptionWithArg(ARGUMENT.CENTRAL_URL, "url",
                        "Alternative URL for Maven Central Search. If not set the public Sonatype Maven Central will be used."))

[cortex35]

Same error for us. Since the last time this worked, we haven't made any changes to my knowledge. The error appeared this morning

[elafontaine]

I think this may be an issue on central itself, but I wouldn't be putting my hand in the fire for that. I just did a basic GET on the URI https://search.maven.org/solrsearch/select and it took a good 15 seconds to get an actual response (400) after the TLS was established...
This may explain why my jobs are hanging over an hour.

[elafontaine]

I think I hit the bulleye; https://status.maven.org/

[aikebah]

That's right. Already spotted the status-mail flood in my mailbox.
I subscribed to their status updates at the time of an earlier longer-during instability of their infrastructure (around the year-turn 2022/2023) and traffic in the last few days was extraordinary high (as also is reflected on the status-page where you see it turn reddish for the recent days and mostly green for the distant past).

[finnlander]

I've noticed the instability of the "legacy search" has been going around for ~ 2 weeks or so (i.e. CI jobs keeps failing on timeouts)... is that maven central service something the dependency checker cannot work without? (i.e. it is the "source of truth" same as NVD API for some required information)

According to Maven Centrals FAQ the "legacy search API" is not in EoL (although the information related to future improvement plans seems to be few years old) and it seems to be the only available API to use the search functionality (or at least I couldn't find any alternatives from their documentation) -> there's probably no better working alternatives to use as the data source 🤔.

p.s. I think that OWASP dependency checker is a great tool. Thank you for all the effort. 💚

[aikebah]

@finnlander The importance of central search (CentralAnalyzer) varies depending on the exact usecase. In case of maven/gradle plugin most of the time maven coordinates of dependencies will be known. (though in case of jar/war packaged inside an ear/war dependency you might still need it for a clean match to the maven artifact coordinates).

When using the CLI (as in the case of the docker container) the analyzer is, for Maven Central hosted libraries, often key in being able to reliably link the corresponding maven coordinates (central analyzer uses the hash of the library to discover hash-matching artifacts in Maven Central).

Having properly identified maven G/A/V coordinates for jar artifacts helps to build a proper package-url so that the OSSINDEX can also be consulted for known vulnerabilities.
Also the maven packageurl is by far the most-used handle to suppress false positives, so also known FP suppression will suffer when maven coordinates for a jar cannot be established.

If your CI environment is using a Nexus repository manager as a caching proxy between your CI and Maven Central you should be able to swap out the central analyzer for the Nexus analyzer.

If your projects do not have dependencies on maven-central hosted libraries it can be disabled without any effect, as the hash-searches would not surface any match.

[elafontaine]

@aikebah , Would your suggestion for the Nexus repository hold true for Artifactory as well ?

As we're having many issues right now with it, it would also reduce the load on them...

[aikebah]

@elafontaine Yes, it holds true for Artifactory/ArtifactoryAnalyzer in case your use of Artifactory includes using it as a caching proxy for artifacts that are hosted on Maven Central.

[elafontaine]

@aikebah I was able to get time on this, and I still see the central analyzer being used. What exactly should I be doing ?

1- Disable the central analyzer and enable artifactory (I don't think that's the right option)
2- Enable artifactory analyzer only (leaving the central analyzer in place, but this seems to still go to the upstream repo)
3- Activate the "proxy" option for artifactory as well as enabling artifactory analyzer.

One of the reason I'm pretty sure it's going upstream is because the artifactory repo is taking 6s while the central analyser is still taking 206s on execution... I may be wrong though... not sure how to enable debug logs...

[aikebah]

@elafontaine If your artifactory is also mirroring central then I would expect option 1) be the right one (no need to also search central for a sha1 match)

Based on the current status page of Maven it looks like they did not (yet?) acknowledge the outage that gave you a 504 result.

[elafontaine]

I tried option 1 and this is the output ;

[INFO] Finished Jar Analyzer (6 seconds)
[WARN] pom.xml not found for <myjar>: hamcrest-2.2.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: bcprov-jdk15on-1.64.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: hamcrest-2.2.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: lombok-1.18.24.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: lombok-1.18.24.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: lombok-1.18.24.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: lombok-1.18.24.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: lombok-1.18.24.jar from Artifactory; this could result in undetected CPE/CVEs.
[WARN] pom.xml not found for <myjar>: lombok-1.18.24.jar from Artifactory; this could result in undetected CPE/CVEs.

Could it be that there is another issue with Artifactory analyzer ?

[elafontaine]

I ran the checks with and without the central analyzer and the results are differents. With the central analyzer, I do not get the message above, so I don't know what isn't working as we do mirror on artifactory.

My guess is that we should probably be polling the pom.xml from upstream when building, but it may not be happening (security concerns there). I tested both unauthenticated and authenticated on artifactory as well.

At this point, the result is richer with maven central and the output is not throwing lines (see above). I'll keep on using central, but if you figure out why the difference happen, I would love to know and possibly update our side.