[Question] Large multi-module maven project with private maven repository

[acmcelwee]

Which Renovate are you using? CLI, App, or Pro

App

Which platform are you using? GitHub, GitLab, Bitbucket Azure DevOps

GitHub

Have you checked the logs? Don't forget to include them if relevant

I have, but nothing jumps out as the configuration that I need to tweak to fix this. The most recent build is Job ID 82808594, in the Renovate Bot dashboard (https://renovatebot.com/dashboard).

What is your question?

I'm in the process of onboarding all of our company repos to Renovate, and the process has been really great so far. We have a multi-module Maven repo that we're trying to get the initial Renovate onboard PR finalized. The maven setup is a parent POM that references all of our modules, where sometimes the module referenced is, itself, an aggregating module that has its own sub-modules. None of the modules in the repo are released for consumption by other repos, so the modules are all just a -SNAPSHOT version, instead of doing regular releases with maven.

Each of the modules directly reference in the parent POM have a reference to the parent, something like this:

...
<parent>
    <groupId>world.data</groupId>
    <artifactId>parent</artifactId>
    <version>0.1-SNAPSHOT</version>
</parent>

<groupId>world.data.renovate</groupId>
<artifactId>renovate-parent</artifactId>
<packaging>pom</packaging>

<modules>
    <module>renovate-demo</module>
</modules>
...

Then, for example, the renovate-demo pom might look something like this:

...
<parent>
    <groupId>world.data.renovate</groupId>
    <artifactId>renovate-parent</artifactId>
    <version>0.1-SNAPSHOT</version>
</parent>

<artifactId>renovate-demo</artifactId>
...

I have private registry authentication setup, thanks to the docs that already existed. I have a few remaining issues that I haven't been able to figure out from the documentation and logs, and I believe that they might be related to our use of a SNAPSHOT version for all of the parent POMs that we reference in the various levels:

• Failed to look up dependency world.data:parent
• Failed to look up dependency world.data.acl:acl-parent

The list goes on to list 20 of our parent modules, and it's identifying a failure to look up all of parent modules across all of our modules.

I also have the following three packages that give the same error, but they don't have the same characteristics of our failures on the parent modules:

• Failed to look up dependency com.amazonaws:aws-lambda-java-core
• Failed to look up dependency com.amazonaws:aws-lambda-java-events
• Failed to look up dependency com.amazonaws:aws-lambda-java-log4j

The aws-lambda-java-core and aws-lambda-java-events deps are currently on 1.1.0 and aws-lambda-java-log4j is on 1.0.0.

Any thoughts on what's going on or suggestions for config updates that I might need to make?

Thanks!

[zharinov]

Hi @acmcelwee, as I can see, we're relying on <relativePath> tag during resolution of parents chain in projects with multiple POMs. May you try to add it to the <parent> section to see if something changed?

[acmcelwee]

@zharinov, thanks for weighing in. I updated the repo to add a <relativePath>../pom.xml</relativePath> node to all of the <parent>...</parent> nodes, since ../pom.xml is the correct path for each of the parents. Renovate re-ran the onboarding job, but it's still reporting the same issues for the parent modules.

FWIW, the default value of <relativePath> is simply ../pom.xml, so I wonder if it would be worth adding that default behavior to the maven package extraction.

[acmcelwee]

In all but 1 occurrence out 20, we have named the parent module with a -parent suffix, and they're all rooted with at least a base world.data <groupId>. Would a targeted packageRules config be our best strategy here to just disable the available updates check?

Also, say we merge the onboarding PR, what's the worst case scenario for these 20 parent module packages? Is it simply that the job will log a warning every time it checks, and that's it? Or would it have any more visible side-effects on our repo?

[zharinov]

@acmcelwee sorry you had to edit 20 files with explicit relativePath, I prepared the fix. I'm not sure about packageRules, I think @rarkins can answer you

[rarkins]

@acmcelwee once you finish onboarding, you won’t actually see the lookup warnings except in the logs, which you never need to look at unless you want to. From a GitHub.com perspective, they’ll just be ignored for now

[acmcelwee]

Thanks @zharinov & @rarkins. I'll just move on with the warnings for our parent modules, for now. It looks like all of the warnings for the com.amazonaws:aws-lambda-java-... packages are due to a missing maven-metadata.xml in our private Artifactory cache, so it looks like I can figure that out on my own.

On a related note - once the initial onboarding is done, is there a way to be alerted when renovate encounters new errors/warnings like these? Basically, if we updated our config, and it somehow stopped being able to resolve things that we expect it to resolve, would we even know about it?

[rarkins]

@acmcelwee right now there is no way to learn of new errors/warnings, but we have plans for this.

Our main plan is to introduce the concept of "criticality" of dependencies, and have the ability for Renovate to abort/stop a run if critical dependencies are not found. We do this for any deps on public repositories like npmjs or Docker Hub currently, but not for private repos. The challenge holding us back is that many users - potentially like yourself here - are OK to let certain private dependencies "fail" because everything else is looking up fine. So this new "criticality" option will instead let you opt-in to the setting for all dependencies so that you can choose for Renovate to stop and alert (e.g. via an Issue or other dashboard) that there are lookup failures.

[acmcelwee]

Thanks @zharinov + @rarkins. I think we'll just let the bot log the silent warnings for now on our parent poms. @rarkins, is that criticality concept what has been discussed in renovatebot/renovate#2263? I subscribed to that issue, but if there's a better place to track that work, please let me know, so I can watch the issue. Thanks again!

[rarkins]

@acmcelwee that's a good issue to follow, we may split the task up into additional issues/PRs but they should be linked from there.

[rarkins]

@zharinov can you make sure we also have an issue in the main Renovate repo tracking the underlying problem here, so that we can hopefully solve it one day?