Versions Maven Plugin not showing latest plugins available with ignores. #959
Comments
|
Can't reproduce. Using version 2.15.0 with a minimal pom.xml with the failsafe plugin and the rule set you provided: <build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>2.15.0</version>
<executions>
<execution>
<id>run</id>
<phase>validate</phase>
<goals>
<goal>display-plugin-updates</goal>
</goals>
<configuration>
<ruleSet>
<ignoreVersions>
<ignoreVersion><type>regex</type><version>.*-SNAPSHOT</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-alpha(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-beta(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-M(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-RC(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-pfd(?:-?\d+)?</version></ignoreVersion>
</ignoreVersions>
</ruleSet>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
</plugin>
</plugins>
</build>and further: So, feel free to provide a pom.xml or a debug log or anything that would help reproduce this. |
Sure. Use this |
|
Otherwise, you're free to contribute. |
|
Without any rules, it will show upgrades to 3.1.0 etc. will give |
|
You also mentioned: <executions>
<execution>
<id>run</id>
<phase>validate</phase>
<goals>
<goal>display-plugin-updates</goal>
</goals>Just a note: this may or may not make a difference, but please note that in my report I am not tying check to any lifecycle. I am invoking it from the command line. (This is essential for my use case.) You should be able to simply take the POM I linked to and run the Bash/PowerShell command I mentioned. Then add the exact configuration I provided to the POM with no further changes. |
Yes. 👍 The whole point of of this ticket is that by adding ignore rules for e.g. milestones, it will also, incorrectly and erroneously, ignore valid, non-ignored versions. |
|
Well, the phase is there so that I can then invoke it using validate. This does not influence anything. Please read the output of the command line invocation above. And with a set of ignored versions: mvn org.codehaus.mojo:versions-maven-plugin:2.15.0:display-plugin-updates -Dmaven.version.ignore=1\.0\.1,.+-M. |
|
Oh, @ajarmoniuk , could it be that I looked at the wrong section of my output? (It was several screens-ful.) I saw: But now I see that this is saying that no plugins require a newer version of Maven, not that no plugins have updates. It may be that I didn't see the list scrolled up above it. I'm late for a meeting right now, but give me two or three hours and I'll double-check this. I appreciate your quick reply. It's possible that this is my mistake, looking at the wrong output. I'll reply back after my meeting. |
|
@ajarmoniuk I was hoping that I had just made a mistake and looked at the wrong section of the output. Unfortunately I'm still seeing what I initially reported in this ticket, and I cannot reproduce the output you have. Here is the exact POM I am testing against: Please note that I am using Apache Maven 3.9.1 ( Here is the exact command I'm running, using Git for Windows Bash (or with PowerShell): mvn versions:display-plugin-updatesHere is my output: Here is the only plugin it says needs updating: But something stranger is going on. You'll note that in your output, it shows that the latest version for In addition I see that your output includes things like It's as if my machine is stuck connected to an old version of Maven Central. It's unclear whether this is a plugin bug; it seems to be related to what information my machine is getting from Maven Central. Can you help me with any hints of what might be happening? |
|
@ajarmoniuk , where does the Version Maven Plugin get its information from, and how does it cache the results? Does it rely on some version in Maven's cache? Does it keep its own cache? Does it always query the information each time, bypassing any cache? |
|
I tried running this with Maven 3.9.2. Maven tells me there are validation problems with It also tells me:
I'll check to see if there is a separate issue for that already. I hope I don't get a warning each time I use the plugin after upgrading to Maven 3.9.2. But for this current issue, I wonder if using this end-of-life In any case, Maven 3.9.2 gives me the same outdated output. |
|
@ajarmoniuk your output had As I mentioned above, your output includes But the more ominous line is this one:
Why does it think that the local metadata is up-to-date? As mentioned there is no Oh, my … what has screwed up the timestamps in my local Maven repository? Could it have been a plugin? Or Eclipse m2e? (I'm using Eclipse EE 2023-03.) There are several confusing things for
OK, now here is something interesting. On 2022-09-08 is when I released v0.8.10 of this POM to Maven Central. As part of doing that:
Since the checksum of |
|
Hi, could you please share the whole obfuscated debug log somewhere? The feature has been quite thoroughly tested but there are always loose ends. So maybe we're onto something. I'm suspecting that there's something fishy about the regular expressions. Not that yours are wrong but the shell you're using might not passing them correctly to Maven. Re your questions - the plugin does not do things like dependency resolution etc., or retrieving dependencies from a remote repository or from the local repository ("Maven cache") on itself. It relies on Maven or its components to do that. Dependency resolution is done using https://maven.apache.org/resolver/, retrieving or writing to cache is core Maven.
Try using -U https://stackoverflow.com/questions/26224842/how-exactly-mvn-u-works And once again, perhaps it would be faster to just read the code and get to know how the plugin works under the hood. There you could create an integration test testing the very case you're experiencing and you could debug it. Also, faster, because of time difference at the least. |
They aren't my regular expressions exactly; I modified them from an answer on Stack Overflow. I intended to simplify them a bit once this was working. But in any case, do you think the regular expressions are even relevant anymore? It's not seeing that
Right, that seems to be like the workaround I'll need to use in order to get this to work. But it worries me that these timestamps were wrong to begin with, and I would really like to figure out what caused them before I simply force all them to be updated. |
|
Regarding Maven warnings in 3.9.2 -- yeah that's a kinda passive-aggressive way of Maven devs' of getting users to chase plugin maintainers to fix their dependencies. The one in versions-maven-plugin has already been done and will be fixed 2.16.0. Plenty of other plugins will still suffer from this and it will cause people relying on warnings from their CI/CD some hassle. |
Good to know, thanks. So Maven Resolver is what should be updating those I note that Maven recently updated Resolver in MNG-7753 and MNG-7769. These supposedly fixed MRESOLVER-352 and MRESOLVER-357. I don't see anything in the tickets regarding corrupted I just had to go on and just cross my fingers it won't happen again without knowing what caused it, because at any time there could be new versions and I wouldn't know it if whatever did this did it again. |
But you said the issue is not occurring if you skip those ignored versions along with their regexes entirely. Or did I get that wrong? |
I'm not sure what you mean when you say "skip those ignored versions". If I check for updates on the latest Adding the regular expression configuration only removes the Did I explain it better that time? Does it make sense? |
Oh one more thing. The plugin will use the version of Maven (and its libraries) that you're using to run it with. So, to bump them, you need to use a never version of Maven. Also, if you clone this repository, you will be able to add your test case as an integration test. E.g. somewhere in there you can then run it using maven invoker, e.g.
or rely on the Github Actions flows to do it for you -- there you can add multiple platforms (Windows, MacOS, Linux) to test it on. |
Right. I'm using Maven 3.9.1 right now. At this point I'm just trying to find out what caused the bad timestamps—that's why I was looking at the Maven changelog to see if any Maven Resolver bugs had been fixed that said anything about bad timestamps. |
|
Maybe there indeed might be something wrong with Maven itself. Another thing you could do is remove the directory from the local Maven repo and see if that helps. It usually does for me. |
Well, yes, I have no doubt that simply blowing away But my my desire right now is that I'd like to know that caused this in the first place, rather that just "force-fixing" it and thinking, "oh, it was a fluke, it will never happen again". |
|
Does it get resolved properly if you use https://maven.apache.org/plugins/maven-dependency-plugin/get-mojo.html |
You mean and specify an actual version? (I glanced at the docs and it looks like that's required.) I'll have to try it tomorrow. It's best not to work when my mind is tired after a long, busy day. Thanks for the input. I'll investigate more after some rest. |
|
Good morning. I opened Maven lastUpdated timestamps in maven-metadata.xml have been set to the future to try to find out how/why this happened. |
|
@ajarmoniuk they pointed out on Stack Overflow that those |
|
Frankly, I'm not so sure why this might occur. However, this is not this plugin but rather Maven (and, possibly, Resolver). Might be caused by a corrupted local repository. How can it get corrupted is not clear to me, "a network glitch" in the Baeldung article hints to some problems with cleanup while releasing resources faced with network errors. https://www.baeldung.com/maven-force-update contains some hints on how to deal with this Does this only happen with plugins or also with dependencies (with dependencies that would mean that a dependency with the given version, which does exist in the central repo, can't be downloaded)? Another idea is to look for/create a new issue on https://issues.apache.org/jira |
|
I've opened a new question on Stack Overflow to try to get to the bottom of this: Maven Artifact Resolver not seeing latest plugins on Maven Central on my machine. |
|
Garret, I didn't mean you had to delete the whole ~/.m2/repository, but just the subfolder containing the dependency that caused problems within it. This way maven will redownload the dependency with the correct version this time. As to why this happens, it's best to ask on either https://issues.apache.org/jira or on Maven users' mailing lists -- users@maven.apache.org. That last one might feel like the old USENET times, but I think it should be the best resource. |
|
Good morning, @ajarmoniuk .
Right, but how do I know at any time which subfolder contains a dependency that is "caus[ing] problems within it"? I only knew in this case because you were nice enough to respond to the ticket and post your test results for this project. But I have many other projects with many other dependencies. How am I to know that these few are the only ones with problems? If Maven is "stuck" at some point in the past on my machine, I likely have dozens of dependencies for which Versions Maven Plugin would should out-of-date information for, but I have no way of knowing which ones they are (because of the nature of this problem). You see what I mean?
I'll do that as well—thanks for the tip. |
I'm saying I did exactly that above. Nevertheless Maven did not go contact Maven Central, and it did not regenerate the properties file. |
|
Now you made me to get out of bed... 😄 |
Sorry!!! 😬 I was going to wait until tomorrow, but when you clarified that If you want to wait until tomorrow I completely understand. But if you keep replying I'll keep plodding forward in the testing. 😂 |
|
My best "hunch" guess at the moment based upon this behavior (and without knowing anything about the code other than what you've told me) is that either A) something in the code is accidentally bypassing the |
|
Am trying to reproduce, but on laptop (not on desktop in office, where I had your setup), give me a five minutes or so... |
|
Damn me, I think I got it |
The suspense here is unbearable … haha |
Just to confirm: you reproduced this? |
|
yes, with local empty repo (first run) 2.4.1 reported as latest (as expected), with altered local repo (2nd run) no update reported. At first sight, this may be related to XML file timestamp.... After reproducing, I have this: Despite 2.4.1 version present, it goes unnoticed, as XML file (content copied from #959 (comment)) does not contains it. |
|
Seems nope: changed TS to "2 days ago", no change. |
|
So wow: this bug is a nasty one: it seems in this case, Maven metadata is handled "like any other artifact in release repository" -- with update policy "never"! This "never update" is true for release artifacts, but NOT for release metadata, that may contain changes (like new versions)... debugged so far, but now am now done. We have a reproducer, and we have where to go from here. In short, problem lies here: https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.10/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/DefaultUpdateCheckManager.java#L274 The |
|
Created https://issues.apache.org/jira/browse/MRESOLVER-363 |
|
So in Maven 3.9.1 this is happening to everyone who already has metadata? Some particular update to Maven Resolver basically left current metadata "locked" at its present state? Am I understanding that correctly? Is the reason that you could not reproduce this earlier the fact that you had no metadata at all, and it was updating everything from scratch? But the moment anyone has metadata, it gets "locked" going forward, is that right? |
|
yes, with empty repo all is nicely downloaded. also, many user (and even myself) in "daily routine" use this plugin with |
|
So |
|
Yes, mvn -h says So is kinda badly worded. Without it this IF 1st block is executed, with it 2nd block: And my guess is that this "never" we see in case of metadata. Need to look more, as to me this "smells" like artifact policy (that is for release rightfully "never") is by mistake applied to metadata. But now am really off. |
|
Thank you for all your work on this, @cstamas ! It's so good to know we're getting somewhere. I had put some work on hold to help get this resolved, as I didn't want to force everything to update until we knew the source of the problem. @ajarmoniuk , thanks for your input and steering me in the right direction, and letting us hijack this ticket for a bit.
I'll go continue my testing of the new Versions Maven Plugin regex ignore version feature, and if that is working, I'll note that and close this ticket. Further discussion on this bug can happen at Thanks again, and I hope you have a good, restful night @cstamas . |
|
Awesome work! Congrats on getting to the bottom of it! |
No, as on issue: reaolver did not found value for lastUpdated. This as we saw, may happen due several factors:
So is not locked for "all, who already have the metadata". If lastUpdated found, it will work. The conditions above must be met. Again, local repo state may be altered by many apps, not only maven resolver, so it is def not "insanely hard" to get into situation like this, but still, it is not "locked down to all who downloaded it once" either. |
|
Just to confirm: the latest understanding is that pull request #957 will fix this problem, because it will override the default policy of local repository metadata to say that the metadata should be refreshed at least daily (including if no last-update information is found by Maven Artifact Resolver)? (The follow-up question of course is whether there is an idea of when that change might be included in a new release?) |
I have plan a next release in this week 😄 |
Wonderful! But can you confirm that we expect #957 to fix this bug? I just wanted to make sure I understood the current status. |
|
Please look - #965 (comment) So I assume the #957 - will fix problem with missing or wrong property in resolver-status.properties |
|
I saw that v2.16.0 was released yesterday, so here goes the test … 🥁 First here is Now here is Oh, wow! Apparently I was behind in a lot of plugins and never new it!! But it looks like #957 fixed the problem with the stale update from Maven Central. In particular we see #Last modified on: Thu Sep 08 15:34:35 PDT 2022
#Thu Sep 08 15:34:35 PDT 2022
central.maven-metadata-central.xml.lastUpdated=1662676475074Here is what #NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Wed Jun 07 13:39:37 PDT 2023
central.maven-metadata-central.xml.lastUpdated=1662676475074
maven-metadata-central.xml.lastUpdated=1686170377857Basically there is a new During this run of OK, things seem to be working great so far. Now for the grand finale: adding ignores so that I don't see things like the milestone releases. You'll remember this was the original point of all this back in #258 and #684. <configuration>
<ruleSet>
<ignoreVersions>
<ignoreVersion><type>regex</type><version>.*-SNAPSHOT</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-alpha(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-beta(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-M(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-RC(?:-?\d+)?</version></ignoreVersion>
<ignoreVersion><type>regex</type><version>.*-pfd(?:-?\d+)?</version></ignoreVersion>
</ignoreVersions>
</ruleSet>
</configuration>And running You'll notice that OK this is really, great news, especially since this has been a journey of over five years (starting with #258). Thank you everyone involved! I'm finally going to be able to close my own ticket, |
…dates. See [#959](mojohaus/versions#959). See [MRESOLVER-363](https://issues.apache.org/jira/browse/MRESOLVER-363).
On Windows 10 with Maven 3.9.1 I upgraded to
org.codehaus.mojo:versions-maven-plugin:2.15.0.I wanted to see what plugins were out of date, so I ran:
It shows this:
So I added ignores for milestone releases as discussed in #684 and originally in #258:
Now when I run
mvn versions:display-plugin-updatesit gives me:This is wrong on several counts.
org.apache.maven.plugins:maven-failsafe-pluginfor example has v3.0.0 and 3.1.0 available—two versions released after the current version.And the bug apparently has nothing to do with ignoring the
*-M*versions, because it even skipped Spring Boot Maven Plugin, which is currently at v3.1.0.This would appear critically broken, in that it does not show new versions, which is the essential function of this goal.
Or did I get the regular expressions wrong somehow? (I think they can be simplified a bit; I was starting from those in a Stack Overflow answer. But in any case I don't see how these regular expressions would be hiding versions in the form
3.0.5.)The text was updated successfully, but these errors were encountered: