m2e can't handle authenticated parallel artifacts transfers

[claudioweiler]

Original report with more discussions: https://bugs.eclipse.org/bugs/show_bug.cgi?id=562847

Also: takari/aether-connector-okhttp#29 (this repository has been archived)

This is a problem with takari aether connector breaking change in commit takari/aether-connector-okhttp@32ce3f8#diff-aebbc2275687e86e034ea328845cca1c5b3539ca4f5025680ae1295973f5a188.

By default Maven handles artifacts transfer (downloads, deploys) with 5 threads. When the repository is authenticated takari connector fails to handle the authentication negotiation with multiple threads:

1. m2e makes a requisition;
2. it receives a 401 from the repository;
   1. on the first thread it reply the repository with authentication credentials;
   2. on others threads it just send the error to the user.

Workaround

1. Create a Maven Run Configuration, and add JRE VM argument -Daether.connector.basic.threads=1.
   Obs: Problem with workaround is that downloads is 1 threaded also.
2. Use external Maven runtime.

Fix suggestions on m2e side

The issue is on takari aether connector. Unfortunately, there is no active maintainers and the repository is now archived.

On m2e there is some suggestions that I think will handle the problem:

• stop using takari connector;
• downgrade takari connector (this is an issue on last 0.17.8 version, version 0.17.6 do not have the issue);
• force workaround without need of users action.

[mwarren2]

Just a bump message...
Any chance this can be looked at any time soon?

It took me quite a while to find this issue, and the workaround has worked so far, so thanks. But I wish for other people that it could be dealt with, it's all a bit messy isn't it?
However I certainly appreciate that you've got a lot on your hands, so thanks anyway for all your work.

[HannesWell]

Just a bump message... Any chance this can be looked at any time soon?

Unfortunately it looks like no one of the active committers (including myself) is working on this at the moment.

It took me quite a while to find this issue, and the workaround has worked so far, so thanks. But I wish for other people that it could be dealt with, it's all a bit messy isn't it? However I certainly appreciate that you've got a lot on your hands, so thanks anyway for all your work.

From the linked issue I'm not sure if this is really an m2e issue, but I'm not familiar with that specific topic.
If you are sure this is a general issue that has to be fixed in m2e we would welcome a quality PR to fix it very much and help you in that regard.
If this is actually an up-stream issue that would only be worked-around in m2e (please note, that we cannot include fixes/workaround for particular connectors) I want to encourage you to provide such fix in the upstream project.

[claudioweiler]

Right, the issue is on aether connector. Unfortunately, there is no active maintainers.

On m2e there is only 3 approaches that I can see:

• downgrade aether connector;
• stop using aether connector;
• force workaround without need of users action.

[laeubi]

the issue is on aether connector

Can you reference the issue there?

there is no active maintainers

You probably could propose a patch then?

[claudioweiler]

Can you reference the issue there?

Sure. It's already on first post above.

You probably could propose a patch then?

Despite I know what caused the problem (specific commit), I'm not aware of the real problem. So it's not a simple rollback case, it will be needed to understand how aether works and how credentials are passed on new multithreading management added on referred commit.

Even if I understand all this and submit a PR, there is a chance that no one approve and merge this PR!

[laeubi]

Sure. It's already on first post above.

Upps, didn't see that :-)

Even if I understand all this and submit a PR, there is a chance that no one approve and merge this PR!

If yo don'T try we never know :-) if this is important enough for people (I have never used that) we can try to adopt it at m2e (but need someone who likes to take care) or fork the project.

So most interesting at the momment would be:

1. Is this directly used in m2e or is this a maven plugin extension (the repo does not contain a readme)
2. is anyone willing to step up in helping to revive/maintain the code if it is not important for m2e directly

[claudioweiler]

Is this directly used in m2e or is this a maven plugin extension

Um... Maybe someone on m2e-core could answer better, but it is directly used by m2e or indirectly used through Eclipse, for sure it's not used by maven (if you run maven on console, or do not use embedded one in eclipse, it works).

[man0ca]

I have a similar issue with the eclipse plugin. I can not upload artifacts bigger than a few MB(500kB works fine, 8.4Mb does not) to my repo. The message is: Failed to execute goal �[32morg.apache.maven.plugins:maven-deploy-plugin:2.7:deploy�[m �[1m(default-deploy)�[m on project �[36mfacerecognition-module�[m: �[1;31mFailed to deploy artifacts: Could not transfer artifact xxxx:jar:0.0.1 from/to 2fmaven (http://xxx/releases): Access denied to http://xxx.jar. Error code 401, Unauthorized�[m -> �[1m[Help 1]�[m

[claudioweiler]

Repository https://github.com/takari/aether-connector-okhttp has been archived. No chances of corrections on takari side.

[laeubi]

Then obviously that project is dead and you should look for alternatives...

It is completely unclear to me what problem should be solved here as it all refers to some commercial product not working under certain circumstances and some workaround when passing a system property but then something else undesired seem to happen... also it seems this is not really something the usual committer/contributors encounter and thus handled with very low priority, so probably you like to provide a PR to fix that together with a test-case to verify it works?

Beside that, sponsoring is a good way to get specific issues fixed, so if this is crucial to your business and you likes to speed up the development of m2e in general a sponsoring would allow me to assign more time-slots to bug fixing or contact me for an individual contract for example fixing a specific issue (not limited to m2e scope).

[claudioweiler]

Wow @laeubi , sorry!

Why that?

It is completely unclear to me what problem should be solved here

I think that the problem is well described, but I will be happy to help on better understand any point that is misunderstood!

it all refers to some commercial product not working under certain circumstances and some workaround when passing a system property but then something else undesired seem to happen...

This issue can be handled on m2e side. Edited first post.

also it seems this is not really something the usual committer/contributors encounter and thus handled with very low priority

I understand it, it's ok to me, and I am not making any demands. Just updating de issue.

so probably you like to provide a PR to fix that together with a test-case to verify it works?

As the only possible way is to change m2e, I can try to make a PR that just downgrade takari connector version. I'm lame on plugin development, but I think that this can be done, but the side of test-case got me.

[laeubi]

As said, if this is important for m2e (I have no clue where/why this takari connector is used) we can try to get clearance to "adopt" the code into m2e itself. If you think it can work without it ... even better :-)

So a standalone test-case would actually be great, it seems the workaround is simply like restoring old behavior (only one thread) so for me that the most unclear part, why the workaround seems not valid.

Another option seems to ask jfrog supporting parallel deploys ... the can even simply halt one deploy if currently one is running, so throwing 401 seems not suitable here...

[claudioweiler]

Right, I can see your misunderstand @laeubi !

Another option seems to ask jfrog supporting parallel deploys

This is why you mention "commercial product". This is not an issue on JFrog, or Nexus (other issue linked to this one by @chadj2).

Problem is on takari connector, and this problem hit others tools around the internet.

JFrog and Nexus handles well parallel deploys or downloads. If you run Maven outside m2e (external maven, or directly on shell) it will use threads (by default: 5) and will work.

This issue is primarily detected on deploys because they commonly are authenticated, but authenticated downloads will fail too, its' not so common to protect artifacts downloads to be authenticated.

This issue was created referencing JFrog because of first notice. But I can change issue title for something better as "m2e can't handle authenticated parallel artifacts transfers". What you think?

so throwing 401 seems not suitable here...

This is the correct behavior, a 401 requesting authentication. The takari connector handles this on 1 thread, but fail to do this on threaded requests. I don't have even a clue why!

why the workaround seems not valid

Workaround is valid. But users eventually got this error and don't know why this is happening, so handling workaround directly on m2e could avoid this new users that don't even know why this is happening.

[laeubi]

Yes if it is about authentication (and not only jfrog) then the title is a bit confusing. Maybe you can even adjust the summary to explain why it works with one thread but not with multiple ones?

[claudioweiler]

Ok @laeubi, done!

[HannesWell]

With #1513 the takari aether connector has been replaced by the standard maven resolver. This will probably fix this issue.
Please help and test if that's really the case in the latest snapshots from:
https://download.eclipse.org/technology/m2e/snapshots/2.5.0/