Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reworked some BSP module inspection tasks #1803

Merged
merged 9 commits into from
Apr 7, 2022

Conversation

lefou
Copy link
Member

@lefou lefou commented Mar 28, 2022

This should improve (fix) performance and OOM issues with large multi module projects where modules depend heavily on each other.

bspCompileClassesPath and bspCompileClasspath are now Targets instead of Tasks. This makes them cacheable and also easier to override in derived modules. I also changed the logic in some transitiveXXX targets from a recursive call to all dependencies to just calling the non-recursive target on a transitive modules list. This should mitigate OOM issue as reported here: #1792 (comment)

Fix #1792

Supersedes #1794

This should improve (fix) performance and OOM issues with large multi module projects where modules depend heavily on each other.
@lefou lefou changed the title Reworked some BSP module inspections tasks Reworked some BSP module inspection tasks Mar 28, 2022
@lefou
Copy link
Member Author

lefou commented Mar 28, 2022

Turns out, Mill's compileModuleDeps handling was a bit sloppy. The calclulation of the transitive module dependencies also included transitive compile-only dependencies, which should have never end up in. My change fixed that. As a result, I needed to correct our build.sc and add some dependencies, to make the self-bootstrap tests succeed.

@lefou lefou marked this pull request as ready for review April 1, 2022 13:18
I have the feeling that our `mill.api.internal` annotation is ignored.
I added some ignore rules manually to get CI green, but I'm not sure if
it is actually safe to change the signature of these`bspXxx` methods.
It should, as they are only used internally, but my detailed knowledge of
trait encoding into classes/objects is a bit rusty.
@lolgab
Copy link
Member

lolgab commented Apr 4, 2022

@lefou There is a bug in mill-mima that ignores the mimaExcludeAnnotations. I reproduced it in a test but the implementation seems exactly the same as Sbt MiMa so I couldn't understand why it doesn't work.
I'm now trying to call MiMa from an insulated classloader to see if the problem is related to the classpath.
In the meantime, the only workaround is to manually add excludes as you did.

Opened an issue here: lolgab/mill-mima#41

@lefou lefou merged commit 7553313 into com-lihaoyi:main Apr 7, 2022
@lefou lefou deleted the bsp-performance branch April 7, 2022 15:22
@lefou lefou added this to the after 0.10.2 milestone Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect & non-performant implementation for JavaModule.bspCompileClasspath
2 participants