Set up sbt-release-early and change build semantics #429
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jvican
changed the title
jvican
commented
Oct 17, 2017
| @@ -1,9 +1,7 @@ | |||
| addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.2.27") | |||
| addSbtPlugin("org.foundweekends" % "sbt-bintray" % "0.5.1") | |||
| addSbtPlugin("com.jsuereth" % "sbt-pgp" % "1.1.0-M1") | |||
| addSbtPlugin("com.lucidchart" % "sbt-scalafmt" % "1.12") | |||
There was a problem hiding this comment.
This is already in sbt-houserules.
jvican
force-pushed
the
improvements-sbt-build-change-semantics
branch
2 times, most recently
from
7995d3f
to
50bd155
Compare
|
This pull request adds a cached publish local that surprisingly allows us to simplify the build by removing ad-hoc commands and workarounds ( |
jvican
force-pushed
the
improvements-sbt-build-change-semantics
branch
from
0066e21
to
62e95cc
Compare
jvican
added a commit
to scalacenter/zinc
that referenced
this pull request
Oct 22, 2017
These are the improvements that I've added to scripted:
1. Scripted is now parallel and does batch execution.
2. Scripted logs to both a file and the console (if `bufferLog == true`).
All the logs can be inspected locally by going to a directory in
`/tmp`. This directory is shown to the user at the end of the
execution.
3. Scripted UI has been improved.
3.1. Colors are used for `+` and `x`.
3.1. It shows the command that actually failed, not `Command failed {line 1}`.
3.2. It trims the stack traces of the wrapping exceptions
(corresponding to the scripted infrastructure). Only the stack traces
of the causing exceptions are shown (which are the ones we're
interested in and are usually assertion errors).
I think these improvements enhance the current dev workflow
considerably. I invite you to give them a try.
This change combined with sbt#429, gives a really fast execution of
scripted. Testing just one test is under 7 seconds in my machine (note
that in those 7 seconds we have to fetch the bridge, etc).
jvican
added a commit
to scalacenter/zinc
that referenced
this pull request
Oct 22, 2017
These are the improvements that I've added to scripted:
1. Scripted is now parallel and does batch execution.
2. Scripted logs to both a file and the console (if `bufferLog == true`).
All the logs can be inspected locally by going to a directory in
`/tmp`. This directory is shown to the user at the end of the
execution.
3. Scripted UI has been improved.
3.1. Colors are used for `+` and `x`.
3.1. It shows the command that actually failed, not `Command failed {line 1}`.
3.2. It trims the stack traces of the wrapping exceptions
(corresponding to the scripted infrastructure). Only the stack traces
of the causing exceptions are shown (which are the ones we're
interested in and are usually assertion errors).
I think these improvements enhance the current dev workflow
considerably. I invite you to give them a try.
This change combined with sbt#429, gives a really fast execution of
scripted. Testing just one test is under 7 seconds in my machine (note
that in those 7 seconds we have to fetch the bridge, etc).
jvican
added a commit
to scalacenter/zinc
that referenced
this pull request
Oct 22, 2017
These are the improvements that I've added to scripted:
1. Scripted is now parallel and does batch execution.
2. Scripted logs to both a file and the console (if `bufferLog == true`).
All the logs can be inspected locally by going to a directory in
`/tmp`. This directory is shown to the user at the end of the
execution.
3. Scripted UI has been improved.
3.1. Colors are used for `+` and `x`.
3.1. It shows the command that actually failed, not `Command failed {line 1}`.
3.2. It trims the stack traces of the wrapping exceptions
(corresponding to the scripted infrastructure). Only the stack traces
of the causing exceptions are shown (which are the ones we're
interested in and are usually assertion errors).
I think these improvements enhance the current dev workflow
considerably. I invite you to give them a try.
This change combined with sbt#429, gives a really fast execution of
scripted. Testing just one test is under 7 seconds in my machine (note
that in those 7 seconds we have to fetch the bridge, etc).
jvican
force-pushed
the
improvements-sbt-build-change-semantics
branch
10 times, most recently
from
a269ead
to
c23f534
Compare
This is a squashed commit. This is a general list of the things that are done. - Port common settings and commands to BuildPlugin - Remove repeated resolvers - Move `relaxNon212` to the build plugin - Move build and alternative publish settings. This commit consolidates the build settings and the settings to set up the alternative publishing process into the `BuildPlugin`. - Move `noPublish` to the `BuildPlugin` - Move the rest of the build logic to the plugin
`sbt-scalafmt` is removed because it's already brought in by `sbt-houserules`.
We only set it up for Bintray for now. In the future we can release to Sonatype, but we need either sbt org's credentials or the sbt org to give us Sonatype rights to publish to "org.scala-sbt" or change org to "ch.epfl.scala".
This build requires too much cross-compiling and cross-publishing, and hence at the moment sbt loads all the compiler classes from all the different versions everything starts to slow down and be close to OOMs and SOs. The following commit adds a jvmopts file that sets up the java flags to be used by sbt when starting up on this repository. These jvmopts file include: - 1024m for the memory at startup time. - 4086m as the maximum allocated memory for sbt. - 512m for the code cache size (otherwise code cache becomes full and JDK disables C2 and fallbacks on interpreter). - Using a bigger inline level considerably helps with compile times and speeds up the whole build (tested in several places).
These changes to the build allow a better workflow with scripted. `cachedPublishLocal` is a special task that avoids publishing if an artifact has already been published and the publication time is greater than the changes to the source directories. In order to find this out, `cachedPublishLocal` resolves the artifact intransitively and in a project-specific directory. This directory is also the used to resolve everything from scripted. This resolution is fast because it assumes that no other task is able to publish that jar to that concrete resolver. As a result, this assumption allows us to remove the global dependency lock. Running `cachedPublishLocal` in the root directory takes from 0 to 1 second. When a new change is written into a project's source file or a resource file, `cachedPublishLocal` publishes only the project that received the changes. This is possible because the optimizer is not allowed and jars are independent from each other. In this setup, `scripted` uses `cachedPublishLocal`. `scripted` **needs** the compiler bridges for all the Scala versions, so if you change the bridge or this is the first time you use this setup, you must run `+compilerBridge/cachedPublishLocal`.
The following code is not necessary anymore. - `zincPublishLocal` et al: Nobody is using the zinc-scripted-alternative because scripted does not use sbt, it uses its own backend. - `scriptedUnpublished` is not necessary anymore since scripted now relies on `cachedPublishLocal`.
jvican
force-pushed
the
improvements-sbt-build-change-semantics
branch
3 times, most recently
from
48a214c
to
d4bf307
Compare
Without this change, we get some linking errors (`scala.Product$` not found) because `+` is not capable of switching the compiler interface to the new scala version, and linking to it from other projects with newer scala versions breaks it. Forcing the scala 2.10 version as the only version of the interface causes also problems when testing the bridge on 2.11 and 2.12 (which are the default versions of the build). Therefore, we remove it from the build and we force that we validate that the interface can be compiled with 2.10 (and hence Java < 8).
jvican
force-pushed
the
improvements-sbt-build-change-semantics
branch
from
d4bf307
to
faa0cd9
Compare
These commands are not necessary anymore since they can be expressed by operations at the task level that are more intuitive than the previous commands. This helps simplifying the whole build setup a lot.
In sbt 1.x, it seems that `publishArtifact` is scoped in the project and not the build. The reason why I noticed this is because these settings were not being applied before this change.
Eugene and Dale think this way is better, because they are not used to the previous style that has a saner way of using scopes in Scala. They have proposed to make the changes in an independent pull request, but I don't have time for it.
jvican
force-pushed
the
improvements-sbt-build-change-semantics
branch
from
faa0cd9
to
af3fa21
Compare
Closed
eed3si9n
requested changes
Nov 21, 2017
build.sbt
Outdated
| // but given that Scala 2.10 compiler cannot parse Java 8 source, it's probably good to keep this. | ||
| crossScalaVersions := Seq(scala210), | ||
| scalaVersion := scala210, | ||
| crossScalaVersions := bridgeScalaVersions, |
There was a problem hiding this comment.
During the release, wouldn't this result to double publishing?
There was a problem hiding this comment.
Good question, I think it would, I'll try it tomorrow and fix it if that's the case.
build.sbt
Outdated
| @@ -202,8 +187,6 @@ lazy val compilerBridge: Project = (project in internalPath / "compiler-bridge") | |||
| .value, | |||
| ) | |||
|
|
|||
| // defines operations on the API of a source, including determining whether it has changed and converting it to a string | |||
There was a problem hiding this comment.
Why are we removing these comments?
There was a problem hiding this comment.
Because it's documented in the contributing guide, better.
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request aims at:
Builds upon #428 and needs to be merged afterwards. When rebase, the actual diff size will be small.