Add the first iteration of the Zinc API 1.0 #304
Conversation
Remove unused `GlobalCompat` for compatibility with 2.8.1 and extract the run subclassing out of the main logic of the `run` method.
The callback implementation should be independent of the compiler interface and the source code should reflect so. Breaking change in comparison with the previous API: `Compiler` is now called `ZincCompiler` to make the naming more clear and better distinguish between `CachedCompiler` and its underlying `ZincCompiler`. Docs have been added and simplifications to methods have been provided and double checked with previous Scala versions. For instance, `dropRun` has been confirmed to only exist in 2.10.x series and removed in 2.11.x. Therefore, its implementation has been moved to the compatibility traits for 2.10.x series. We then remove a reflective call for versions above 2.10.6.
We need `CompileAnalysis` to start creating the Java API with good docs. Contraband is not necessary for the analysis since we only serialize/deserialize `MAnalysis`, its subclass.
* Make `ReadStamps` a Java interface exposed in the public API. * Create a `Stamp` interface that Java users can use without any knowledge of the internal requirement. * Add docs to the pertinent Java interface. * Change `Stamp` companion object by `Stamper` to make it Java-friendly. I also think the name is clearer about the contents of the object. * Rename `internalSouce` to `source` for consistency with the Scala API. It's made clear that source should always be an internal source in the docs. This makes sense anyway, otherwise it wouldn't make sense to ask for the stamp of a source file that is not compiled in the current analysis.
`Exists` has never been used, neither internally in Zinc nor in sbt and pants (I double checked on both). The concept of `exists` does not make sense, since there is no value in asserting that a stamp "exists" only in asserting that a stamp is **not** available for a given file. The latter is represented by the new `Stamp` API since last modified time and hash are both optional. If both are empty, a stamp does not exist.
`CompileAnalysis` should be an interface instead of an abstract class.
Users of `CompileAnalysis` will have access to the stamps of the files that have been processed by Zinc.
`ReadSourceInfos` is a read-only interface for getting source info (namely, compiler errors) associated with a given file. This is very useful in lots of cases. Especially to process and improve error messages in a lightweight way, though ScalaClippy already exists for that purpose. In any case, having access to the compiler errors outside of is of ultimate importance for tools that want to preprocess compilation results independently of sbt.
`OutputSetting` is like `OutputGroup` by just returns strings instead of files. I see no reason why `OutputGroup` should not be used instead, therefore simplifying the Zinc API and allowing users to have access to files and not mere strings. If they want strings, they always can do `getAbsolutePath`. N.B. This is good for machine independence.
* Remove dangerous use of output groups as fields in compilation. In the case of `SingleOutput`, the source directory was invented and incorrect (the root). Users of the API processing this file could create chaos in people's computers. The new design forces consumers of the `Compilation` API to make up their minds and identify whether the compilation was used with a single output or several outputs. In order to do that, we return `Output` instead of an array of `OutputGroup`. * Rename `outputDirectory` and `sourceDirectory` to be Java friendly. * Augment `Output` interface with Java-friendly methods. Scala users can use pattern matching instead.
* Make methods in `Output` subclasses public * Changes to the API. Java idiomatic renames: * sourceDirectory -> getSourceDirectory * outputDirectory -> getOutputDirectory
* `Compilation` has been ported to Java code and documented. * `ReadCompilations` is a read-only API of `Compilations`. * Add a TODO to check the ordering of the array of compilations. * Expose `ReadCompilations` to the `CompileAnalysis`.
* Add interface for the provider. * Rename `IvyComponentCompiler` to `ZincComponentCompiler`. * Split `ZincComponentCompiler` and `ZincComponentManager`. * Define `IfMissing` and `InvalidComponent` into independent files. * Rename variables and internal API to be clearer.
No need to have it in the internal Scala side.
Previous name was incorrect, we actually return the compiled bridge.
* Remove unused code in `BridgeProviderSpecification` and test the real code in the default provider. * Don't use temporary directory to download the Scala jars, retrive them in a directory specified by the user. * Use `java.net.ClassLoader` instead of `ScalaClassLoader`. * Use the `ScalaInstance` interface, not the implementation. Remove any reference to the implementation.
This commit makes sure that we always check that we compile against the latest two versions in Scala 2.10, Scala 2.11 and Scala 2.12. This commit also removes the null loggers and uses normal loggers to check what the output of the bridge provider is. This output is important and the rationale to do this is that it should be visible for us in order to detect cases where there's a regression or the output is modified in some way. Output is short anyway.
* Adds a method to fetch the default implementation of the compiler provider. * Adds a method to get a default `IvyConfiguration` from a small number of parameters. The public API is implemented in Java to ensure binary compatibility no matter what happens to the internal Scala implementations. It also makes it easier for Java users to use the API and avoid importing MODULE$'s and Scala generated class files.'
As suggested by Eugene.
|
Feedback has been addressed @eed3si9n. |
|
Tests are green. |
|
Thanks for all the work on this. |
|
With regard to additional private/internal APIs that are currently in use, can browse through https://github.com/pantsbuild/pants/tree/master/src/scala/org/pantsbuild/zinc (as of zinc |
|
@stuhood I've been browsing through that for a while, but thanks for the heads up. Are you in a hurry to do this? I'll be tackling the second iteration of the API soon before the release candidate and the API freeze, so maybe you want to wait until that's out? |
|
The change I made to sbt is sbt/sbt#3228 |
|
@jvican : It would probably make sense for me to continue to attempt this, just in order to find any additional issues that you might not have seen. As an example, yesterday I noticed that it's not currently possible to extend |
Great, I'll make this (or something similar, I have to look into the details) happen to unblock that implementation. |
|
@jvican: A few more details here:
I'm going to try porting to usage of |
|
Alright, I ported to Also, it doesn't look like it's possible to use a "default" Reporter (as it is with |
|
@stuhood: Yes, it seems that the entanglement between To overcome this issue, some of Zinc's test suites create a With regards to making the reporter |
|
For the record, if you want to move forward and need a |
As explained in the issue and discussions [here](sbt#304 (comment)).
Yea, that makes sense. I'm still thinking of zinc as a CLI tool, but in the context of a library, being explicit about output location is important... ditto logging. |
|
Hey @stuhood, I've been fixing all the issues you've reported regarding logging. I think I have come up with some elegant solutions you may want to have a look at. This is the most important change: scalacenter@a9a9c2f. All the changes are in this branch: 1.0...scalacenter:fix-loggers. I can release these changes as part of a temporary In the new interface, I have added a filtered reporter in Zinc and the normal Let me know if this public API works for you. |
|
Thanks @jvican . The API looks great (with one exception mentioned below): thanks for incorporating file/msg based filtering. I should be able to sanity check incremental compile before the end of Monday. There is a bit of inconsistency between |
|
@stuhood Happy that you mention that, I opened a PR in sbt/util to address that issue three days ago. The idea is to remember any trace of the old Java-friendly classes for bincompat ( |
|
The new API looks good from a logging/usability perspective (other than #317). I have things working well enough to have reproduced pantsbuild/pants#4477, so I will begin debugging that tomorrow. |
|
@jvican : If you have a PR for the |
|
Shipping it tomorrow morning. It's ready but kept it local because I'm changing more things in the API. |
As explained in the issue and discussions [here](sbt#304 (comment)).
As explained in the issue and discussions [here](sbt#304 (comment)).
This is just the first iteration of the Zinc API.
This iteration's goal is to make safe functionality of Zinc public to allow Scala and Java parties (Gradle, Pants, Maven, et cetera) use Zinc with binary compatibility guarantees. This task needs another iteration given:
The last iteration will happen before RC1 (end of June - beginning of July).
The new public API makes it easier to:
sbt.internal.inc.Aside from these, this API also addresses some issues that have arisen in the past:
MaybebyOptional#302I find that in general, these commits make Zinc easier to use and improve the consistency of the whole API. The most remarkable change is the
CompileAnalysisAPI, which now provides useful information for build tools like file stamps, compilations and source infos.Note that all the utilities made public only expose functionality that is a product of the incremental compiler. No internal details of the incremental analysis are exposed to the users.
There are some Java files that have been moved out of the compiler interface because they required static method implementations in the projects on which the implementations of certain classes were provided.
In the future, I feel that these are the things that should be worked on to finish off the Zinc API:
3.1. Empty
Analysis.3.2.
DefinesClass.3.3.
AnalysisStore(up to discussion for next iteration).3.4.
JavaToolet al, necessary to createCompilers.3.5.
PerClasspathLookupEntry.I have more ideas, but those will be discussed when time's up.
The breaking changes are documented in every commit. I suggest that reviewers go through the commits one by one, though I don't expect them to do so (given the big diff size). As this is just the first iteration of the redesign and targets a milestone, I am not writing any migration notes for now. I'll wait until everything is finished in our first RC to write the migration notes so that other build tools (other than sbt) can start using it.
As a reminder, the public API is
xsbti. Scala implementations live insbt.internal.inc.This is a good time to gather feedback. So that's very much welcome.