Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bnd resolver "hangs" #3201

Closed
maggu2810 opened this issue May 23, 2019 · 15 comments
Closed

bnd resolver "hangs" #3201

maggu2810 opened this issue May 23, 2019 · 15 comments
Assignees
@rotty3000
Labels
abeyance need of contributor [requires is:closed]

Comments

@maggu2810
Copy link
Contributor

While working on the migration of an existing code base to Bnd and adding integration tests, I went into a problem.
The resolution of an integration tests "never ends".

I tried to create a simple demonstration project that shows you the error.
I don't like to say that the application in that project and the bundle selection itself makes any sense, but this should not be the point.

If I try to resolve that application (using the Maven plugin) it ends after some minutes with an error: [ERROR] GC overhead limit exceeded -> [Help 1].

You can find the project and the readme here: https://github.com/maggu2810/bnd-resolvebug

Used bnd version: 4.2.0
@maggu2810
Copy link
Contributor Author

"Bug" seems still to be present on 4.3.0-SNAPSHOT

@ViToni
Copy link

ViToni commented May 24, 2019

You might want to try different memory settings. At least in the IDE the resolver might "hang" which was "cured" by giving more memory (in my case going from 1024MB to 2048MB).

@maggu2810
Copy link
Contributor Author

I give it a new try using the most recent Bnd resolver Maven plugin snapshot.

Machine: i7-4712HQ

$ mvn -version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T20:41:47+02:00)
Maven home: /home/maggu2810/bin/pkgs/java/maven/current
Java version: 1.8.0_202, vendor: Oracle Corporation, runtime: /home/maggu2810/data/shared/bin/pkgs/java/jdk/8/OracleJDK/jdk1.8.0_202/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.0.7-gentoo", arch: "amd64", family: "unix"

$ uname -a
Linux m3800 5.0.7-gentoo #1 SMP Thu Apr 11 15:48:53 CEST 2019 x86_64 Intel(R) Core(TM) i7-4712HQ CPU @ 2.30GHz GenuineIntel GNU/Linux

The load of every CPU is something between 96 and 100%.

I allowed to use max. 4G of heap (-Xmx4G).

The command runs over 10 minutes now.
Will keep you up to date (I used time to get the information).

@maggu2810
Copy link
Contributor Author

So, it failed again because of Java heap space.

Command that has been executed:

$ time (MAVEN_OPTS="-Xmx4G" mvn -pl resolvebug-app -am bnd-indexer:index bnd-indexer:index@test-index bnd-resolver:resolve package)

So, you will see also the time (and as written above it fully load all my CPUs that time):

...
[INFO] 
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ resolveapp-dummy ---
[INFO] Building jar: /home/maggu2810/data/shared/workspace/projects/de.maggu2810/bnd/bnd-resolvebug/resolveapp-dummy/target/resolveapp-dummy-1.0-SNAPSHOT.jar
[INFO] 
[INFO] -------< de.maggu2810.playground.bnd.resolvebug:resolvebug-app >--------
[INFO] Building resolvebug-app 1.0-SNAPSHOT                               [3/3]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- bnd-indexer-maven-plugin:4.3.0-SNAPSHOT:index (default-cli) @ resolvebug-app ---
[INFO] 
[INFO] --- bnd-indexer-maven-plugin:4.3.0-SNAPSHOT:index (test-index) @ resolvebug-app ---
[INFO] 
[INFO] --- bnd-resolver-maven-plugin:4.3.0-SNAPSHOT:resolve (default-cli) @ resolvebug-app ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for resolvebug 1.0-SNAPSHOT:
[INFO] 
[INFO] resolvebug ......................................... SUCCESS [  0.483 s]
[INFO] resolveapp-dummy ................................... SUCCESS [  1.665 s]
[INFO] resolvebug-app ..................................... FAILURE [27:00 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  27:03 min
[INFO] Finished at: 2019-05-24T16:49:28+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Java heap space -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

real    27m3.986s
user    206m27.432s
sys     0m31.020s

@rotty3000
Copy link
Contributor

rotty3000 commented May 24, 2019 via email

@maggu2810
Copy link
Contributor Author

As written above:

I tried to create a simple demonstration project that shows you the error.

You can find the project and the readme here: https://github.com/maggu2810/bnd-resolvebug

Didn't the project work for you?

@rotty3000
Copy link
Contributor

rotty3000 commented May 24, 2019 via email

@IOOOTAlan IOOOTAlan mentioned this issue May 30, 2019
Closed
@bjhargrave bjhargrave added this to the 4.3 milestone May 31, 2019
@maggu2810
Copy link
Contributor Author

@rotty3000 Have you had already time to download the (minimal) demo project to reproduce the problem?

@rotty3000
Copy link
Contributor

sorry, I have not yet had a chance.

@maggu2810
Copy link
Contributor Author

I would like to test this against the most recent snapshot of the Felix Resolver.
I checked out the Apache Felix repository and build the resolver artifact myself.
After that I rebuild the bnd-resolver-maven-plugin (adding a dependency to the felix resolver snapshot at the beginning to prefer the most recent version). After some testing I realized that that artifact is not used as the Bnd Resolver bundles the Felix resolver in its bundle.
So, I (perhaps you already realized I know how to work with Maven but not really how to use Gradle and the Bndtools / Bnd Workspace correctly) edit the central.mvn and update the version of the felix resolver to the snapshot one (that is already present in my local Maven repo).
As I did not know how to build only the related stuff and if it gets really injected in my local Maven repo, I rebuild the whole Bnd workspace (./gradlew).
After that I build the bnd-resolver-maven-plugin.
To be honest, I am still not sure if now the correct Bnd Resolver bundle is used...

Can someone give me the instructions how I can update third party dependencies (e.g. version numbers) and so build all that the bnd-resolver-maven-plugin needs to get that changes reflected?
As the non Maven plugins are build by Gradle and not all bundles are part of one Maven reactor, I cannot use .e.g mvn clean install -am -pl maven/bnd-resolver-maven-plugin.
But I am pretty sure there is a simple way to handle it.

@bjhargrave
Copy link
Member

If you build the Bnd workspace with ./gradlew :build :maven:deploy (You can see this command in the travis and the azure pipeline yaml), this will build the snapshot and deploy to your ~/.m2 repo. Then you can use the 4.3.0-SNAPSHOT bnd maven plugins in your test build.

@maggu2810
Copy link
Contributor Author

maggu2810 commented Jul 15, 2019 •
edited
Loading

As I don't know if you track the resolver changes used by the resolver plugin( there has been apache/felix@eaf038b#diff-f8609e719e58d9fd43255f51f3b48e8e), I gave it another try if perhaps changes of the dependent project solves this bug.

Bnd is still not able to resolve the demo project (by providing up to 4G of heap) in a "reasonable" time.
It does not yet run into an out of heap space error but the resolution already runs over 50 minutes.
All four cores are using 50% of CPU for the job.

What I have done:

  • I cloned the "felix" repo and built the current snapshot of the resolver
  • I changed in the file cnf/ext/central.mvn the line org.apache.felix:org.apache.felix.resolver:2.0.0 to org.apache.felix:org.apache.felix.resolver:2.1.0-SNAPSHOT
  • I built the Bnd workspace using ./gradlew :build :maven:deploy -x test
  • Start the resolver against the "resolvebug-app" using 4G of heap (and trace the time)
time (MAVEN_OPTS="-Xmx4G" mvn -pl resolvebug-app bnd-resolver:resolve)

# cancelled manually as I need to restart the machine

^C
real    65m26.297s
user    98m22.419s
sys     13m52.140s

@bjhargrave bjhargrave modified the milestones: 4.3, 4.4 Jul 19, 2019
@maggu2810 maggu2810 mentioned this issue Jul 21, 2019
Closed
@bjhargrave bjhargrave modified the milestones: 5.0, 5.1 Dec 13, 2019
@bjhargrave bjhargrave modified the milestones: 5.1, 5.2 Apr 17, 2020
@juergen-albert juergen-albert mentioned this issue Jun 10, 2020
Closed
@kriegfrj
Copy link
Contributor

I just had an interesting situation where I had a launch file that would resolve in a reasonable time on one machine and not on another, even when I synchronised the two workspaces through GitHub. Then when I did a clean build on the one that was working it started hanging too. Unfortunately while I tried to do a controlled test I'm not 100% sure if I succeeded and it is proving difficult to repeat. The resolve was for an Eclipse instance for an integration test for bndtools.core.

@rotty3000 rotty3000 added the abeyance need of contributor [requires is:closed] label Sep 9, 2020
@rotty3000 rotty3000 removed this from the 5.2 milestone Sep 9, 2020
@kriegfrj
Copy link
Contributor

kriegfrj commented Jan 6, 2021

Btw, i think the bndtools.core resolve issue is (among other reasons) because biz.aQute.junit and org.junit are both available. Blacklisting biz.aQute.junit helped me to get bndtools.core to resolve.

@maggu2810
Copy link
Contributor Author

I updated the demonstration I linked above to use bnd 5.2.0 and the error is not present anymore.
Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rotty3000 rotty3000

Labels
abeyance need of contributor [requires is:closed]
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@rotty3000 @maggu2810 @bjhargrave @kriegfrj @ViToni