Set up sbt-release-early and change build semantics

.drone.yml

@@ -12,11 +12,25 @@ pipeline:
       event: [push, pull_request, tag, deployment]
 
   tests:
-    image: scalacenter/scala:1.0
+    image: scalacenter/scala:1.3
     commands:
       - git fetch --tags && git log | head -n 20
       - ./bin/run-ci.sh ${CI_SCALA_VERSION}
 
+  publish:
+    image: scalacenter/scala:1.3
+    volumes:
+      - /scalacenter:/keys
+    when:
+      event: [ push, tag, deployment ]
+      status: success
+    commands:
+      # To know what we're releasing
+      - git log | head -n 20
+      - /usr/local/bin/setup_keys.sh
+      - /usr/local/bin/store_credentials.sh
+      - sbt releaseEarly
+
   # Save folders in distributed cache
   sftp_cache_rebuild:
     image: plugins/sftp-cache

.drone.yml.sig

@@ -1 +1 @@
-eyJhbGciOiJIUzI1NiJ9.cGlwZWxpbmU6CiAgIyBGZXRjaCBmb2xkZXJzIGZyb20gZGlzdHJpYnV0ZWQgY2FjaGUKICBzZnRwX2NhY2hlX3Jlc3RvcmU6CiAgICBpbWFnZTogcGx1Z2lucy9zZnRwLWNhY2hlCiAgICByZXN0b3JlOiB0cnVlCiAgICBtb3VudDoKICAgICAgLSAvZHJvbmUvLml2eTIKICAgICAgLSAvZHJvbmUvLmNvdXJzaWVyLWNhY2hlCiAgICAgIC0gL2Ryb25lLy5zYnQKICAgICAgLSAvZHJvbmUvLmdpdAogICAgd2hlbjoKICAgICAgZXZlbnQ6IFtwdXNoLCBwdWxsX3JlcXVlc3QsIHRhZywgZGVwbG95bWVudF0KCiAgdGVzdHM6CiAgICBpbWFnZTogc2NhbGFjZW50ZXIvc2NhbGE6MS4wCiAgICBjb21tYW5kczoKICAgICAgLSBnaXQgZmV0Y2ggLS10YWdzICYmIGdpdCBsb2cgfCBoZWFkIC1uIDIwCiAgICAgIC0gLi9iaW4vcnVuLWNpLnNoICR7Q0lfU0NBTEFfVkVSU0lPTn0KCiAgIyBTYXZlIGZvbGRlcnMgaW4gZGlzdHJpYnV0ZWQgY2FjaGUKICBzZnRwX2NhY2hlX3JlYnVpbGQ6CiAgICBpbWFnZTogcGx1Z2lucy9zZnRwLWNhY2hlCiAgICByZWJ1aWxkOiB0cnVlCiAgICBtb3VudDoKICAgICAgLSAvZHJvbmUvLml2eTIKICAgICAgLSAvZHJvbmUvLmNvdXJzaWVyLWNhY2hlCiAgICAgIC0gL2Ryb25lLy5zYnQKICAgICAgLSAvZHJvbmUvLmdpdAogICAgd2hlbjoKICAgICAgZXZlbnQ6IFtwdXNoLCBwdWxsX3JlcXVlc3QsIHRhZywgZGVwbG95bWVudF0KCm1hdHJpeDoKICBDSV9TQ0FMQV9WRVJTSU9OOgogICAgLSAyLjExLjExCiAgICAtIDIuMTIuMwo.usuc6GVWQgjD1XYQW9Fd8U8N7Qj1b6WfmGdtIn44Q-8
+eyJhbGciOiJIUzI1NiJ9.cGlwZWxpbmU6CiAgIyBGZXRjaCBmb2xkZXJzIGZyb20gZGlzdHJpYnV0ZWQgY2FjaGUKICBzZnRwX2NhY2hlX3Jlc3RvcmU6CiAgICBpbWFnZTogcGx1Z2lucy9zZnRwLWNhY2hlCiAgICByZXN0b3JlOiB0cnVlCiAgICBtb3VudDoKICAgICAgLSAvZHJvbmUvLml2eTIKICAgICAgLSAvZHJvbmUvLmNvdXJzaWVyLWNhY2hlCiAgICAgIC0gL2Ryb25lLy5zYnQKICAgICAgLSAvZHJvbmUvLmdpdAogICAgd2hlbjoKICAgICAgZXZlbnQ6IFtwdXNoLCBwdWxsX3JlcXVlc3QsIHRhZywgZGVwbG95bWVudF0KCiAgdGVzdHM6CiAgICBpbWFnZTogc2NhbGFjZW50ZXIvc2NhbGE6MS4zCiAgICBjb21tYW5kczoKICAgICAgLSBnaXQgZmV0Y2ggLS10YWdzICYmIGdpdCBsb2cgfCBoZWFkIC1uIDIwCiAgICAgIC0gLi9iaW4vcnVuLWNpLnNoICR7Q0lfU0NBTEFfVkVSU0lPTn0KCiAgcHVibGlzaDoKICAgIGltYWdlOiBzY2FsYWNlbnRlci9zY2FsYToxLjMKICAgIHZvbHVtZXM6CiAgICAgIC0gL3NjYWxhY2VudGVyOi9rZXlzCiAgICB3aGVuOgogICAgICBldmVudDogWyBwdXNoLCB0YWcsIGRlcGxveW1lbnQgXQogICAgICBzdGF0dXM6IHN1Y2Nlc3MKICAgIGNvbW1hbmRzOgogICAgICAjIFRvIGtub3cgd2hhdCB3ZSdyZSByZWxlYXNpbmcKICAgICAgLSBnaXQgbG9nIHwgaGVhZCAtbiAyMAogICAgICAtIC91c3IvbG9jYWwvYmluL3NldHVwX2tleXMuc2gKICAgICAgLSAvdXNyL2xvY2FsL2Jpbi9zdG9yZV9jcmVkZW50aWFscy5zaAogICAgICAtIHNidCByZWxlYXNlRWFybHkKCiAgIyBTYXZlIGZvbGRlcnMgaW4gZGlzdHJpYnV0ZWQgY2FjaGUKICBzZnRwX2NhY2hlX3JlYnVpbGQ6CiAgICBpbWFnZTogcGx1Z2lucy9zZnRwLWNhY2hlCiAgICByZWJ1aWxkOiB0cnVlCiAgICBtb3VudDoKICAgICAgLSAvZHJvbmUvLml2eTIKICAgICAgLSAvZHJvbmUvLmNvdXJzaWVyLWNhY2hlCiAgICAgIC0gL2Ryb25lLy5zYnQKICAgICAgLSAvZHJvbmUvLmdpdAogICAgd2hlbjoKICAgICAgZXZlbnQ6IFtwdXNoLCBwdWxsX3JlcXVlc3QsIHRhZywgZGVwbG95bWVudF0KCm1hdHJpeDoKICBDSV9TQ0FMQV9WRVJTSU9OOgogICAgLSAyLjExLjExCiAgICAtIDIuMTIuMwo.hqDOPvurmUpIxCK_VDzn_v7mH6DPqZBSR6EqO-8mJm8

.gitignore

@@ -1,2 +1,3 @@
 target/
 zinc/src/test/resources/bin
+.ivy2/

.jvmopts

@@ -0,0 +1,4 @@
+-Xms1024m
+-Xmx4086m
+-XX:ReservedCodeCacheSize=512m
+-XX:MaxInlineLevel=20

CONTRIBUTING.md

@@ -31,8 +31,7 @@ Once you understand the basics of incremental compilation, start having a look
 at open tickets you can help with. All issues are labelled and will give you an
 idea about its difficulty and scope.
 
-Hacking on Zinc should not seem like a difficult task. Zinc does not implement
-a compiler, it defines the logic to analyse dependencies based on the compiler
+Hacking on Zinc should not seem like a difficult task. Zinc does not implement a compiler, it defines the logic to analyse dependencies based on the compiler
 API and creates all the required infrastructure around it to let build tools
 use it.
 
@@ -55,6 +54,69 @@ bridge, you don't need to run these benchmarks.
 If you need any help, the Zinc team hangs out in [sbt/zinc-contrib][].
 Feel free to ask any question.
 
+### Getting familiar with the build
+
+#### Project structure
+
+As of now, the current project structure is not as simple as it can be and we
+believe that it can be simpler in the future. However, as of now, no work is
+happening in this area because it's deemed to have low impact on the overall
+quality of the project.
+
+How is the Zinc build structured? Let's see it.
+
+|Project name| Project description|
+|------------|--------------------|
+|zincRoot| The root of the project. Aggregates all projects except the benchmarks.|
+|zinc|The user-facing Zinc incremental compiler.|
+|zincTesting|The project that defines testing facilities.|
+|zincCompile|A thin wrapper that provides doc capabilities.|
+|zincPersist|The project that persists incremental compiler's data into a binary file.|
+|zincCore|The project that defines relations, analysis, stamps, and essential core utils.| 
+|zincBenchmarks|The project that defines the benchmarks.|
+|zincIvyIntegration|The project that defines the ivy utilities to fetch compiler bridges.|
+|zincCompileCore|The project that interfaces with the compiler API and provides compilation capabilities.| 
+|zincApiInfo|The project that defines name hashes and provides way to interpret api changes.|
+|zincClassfile|The project that parses class files to provide Java incremental compilation.|
+|zincClasspath|The project that provides basic utilities to load libraries with classloaders and represents Scala instances.|
+|zincScripted|The project that defines the scripted logic to run Zinc's integration test suite.|
+|compilerInterface|The public binary interface used to connect the bridges with the Zinc modules. It is written in Java and uses Contraband.|
+|compilerBridge|The module that defines the compiler plugin phases that provide incrementality for all Scala versions.|
+
+If you want to visualize the relationships between the projects, have a look at
+the following diagram:
+
+![Diagram of projects](docs/project-structure.png)
+
+Note: ignore `jar2`, `jar1` and `classesDep1`. 
+
+#### Build-specific commands/keys
+
+The sbt build defines several keys that help contributors run and test Zinc.
+Zinc's build requires the compiler bridges to be published before tests are run
+(compiler bridges are compiler-specific Scala sources that need to be fetched
+to perform incremental compilation).
+
+The following keys assume that you're at the root project `zincRoot` (true unless you do `project somethingElse`).
+
+|Key|Use|
+|---|---|
+|+compilerBridge/cachedPublishLocal|Cross-publishes all the compiler bridges if there have been changes. Required by `scripted`.|
+|cachedPublishLocal|Compiles and publishes all the modules if there have been changes. Executed by `scripted` internally.|
+|test|Run the unit tests (if defined).|
+|scripted|Run all the integration tests that check that incremental compilation happens with real-world examples. You can also pass the scripted test you want to run as an argument.|
+|publishBridgesAndTest|Publish bridges and test the whole incremental compiler. Inefficient for a fast, local developer workflow, ideal for CI.|
+
+##### Workflow?
+
+The most likely scenario is that you don't change code in the bridges. In that case, if you want to
+unit test you run `test` on whichever modules you want (typically in the `zincRoot` for all of them).
+If you want to run integration tests, you run `scripted` directly.
+
+If you change code in the bridges and you want to test it, you must always run
+`+compilerBridge/cachedPublishLocal` for scripted to pick up your changes for all the scala versions.
+Then, you proceed with `scripted`.
+
 ### Benchmarking Zinc
 
 To run JMH benchmarks, run the sbt task `runBenchmarks`. By default,

bin/run-ci.sh

@@ -3,15 +3,27 @@ set -eu
 set -o nounset
 SCALA_VERSION="$1"
 
+# Exemplifies how the build should be used by user
+# The following script makes sure to:
+#  1. Checking binary compatibility
+#  2. Checking formatting and headers 
+#  3. Testing the compiler bridge for all versions
+#  4. Setting up the scala version specified by the argument
+#  5. Running tests for all the projects and then scripted
 
 sbt -Dfile.encoding=UTF-8 \
   -J-XX:ReservedCodeCacheSize=256M \
   -J-Xmx3046M -J-Xms3046M -J-server \
   +mimaReportBinaryIssues \
   scalafmt::test \
   test:scalafmt::test \
+  headerCheck \
+  test:headerCheck \
+  ++2.10.6 \
+  compilerInterface/compile \
   zincRoot/test:compile \
-  crossTestBridges \
-  "publishBridgesAndSet $SCALA_VERSION" \
+  +compilerBridge/cachedPublishLocal \
+  +compilerBridge/test \
+  "++$SCALA_VERSION" \
   zincRoot/test \
   zincRoot/scripted