Skip to content

Commit

Permalink
Tweak readme, fix SBT executable assembly prefix
Browse files Browse the repository at this point in the history
  • Loading branch information
lihaoyi committed Nov 27, 2017
1 parent eb0f0a5 commit c6d95ab
Show file tree
Hide file tree
Showing 4 changed files with 165 additions and 10 deletions.
2 changes: 1 addition & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ val sharedSettings = Seq(
assemblyOption in assembly := (assemblyOption in assembly).value.copy(
prependShellScript = Some(
// G1 Garbage Collector is awesome https://github.com/lihaoyi/Ammonite/issues/216
Seq("#!/usr/bin/env sh", """exec java -jar -Xmx500m -XX:+UseG1GC $JAVA_OPTS "$0" "$@"""")
Seq("#!/usr/bin/env sh", """exec java -cp "$0" mill.Main "$@" """)
)
),
assembly in Test := {
Expand Down
5 changes: 4 additions & 1 deletion build.sc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ trait MillModule extends ScalaModule{ outer =>
def scalaVersion = "2.12.4"
override def sources = basePath/'src/'main/'scala
object test extends this.Tests{
override def projectDeps =
if (this == Core.test) Seq(Core)
else Seq(outer, Core.test)
def basePath = outer.basePath
override def ivyDeps = Seq(Dep("com.lihaoyi", "utest", "0.6.0"))
override def sources = basePath/'src/'test/'scala
Expand Down Expand Up @@ -51,5 +54,5 @@ object ScalaPlugin extends MillModule {
def basePath = pwd / 'scalaplugin
override def prependShellScript =
"#!/usr/bin/env sh\n" +
"exec java -cp \"$0\" mill.scalaplugin.Main \"$@\""
"""exec java -cp "$0" mill.Main "$@" """
}
151 changes: 151 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,157 @@ This is similar to SBT's `:=`/`.value` macros, or `scala-async`'s
their code in a "direct" style and have it "automatically" lifted into a graph
of `Task`s.

## How Mill aims for Simple

Why should you expect that the Mill build tool can achieve simple, easy &
flexible, where other build tools in the past have failed?

Build tools inherently encompass a huge number of different concepts:

- What "Tasks" depends on what?
- How do I define my own tasks?
- What needs to run in what order to do what I want?
- What can be parallelized and what can't?
- How do tasks pass data to each other? What data do they pass?
- What tasks are cached? Where?
- How are tasks run from the command line?
- How do you deal with the repetition inherent a build? (e.g. compile, run &
test tasks for every "module")
- What is a "Module"? How do they relate to "Tasks"?
- How do you configure a Module to do something different?
- How are cross-builds (across different configurations) handled?

These are a lot of questions to answer, and we haven't even started talking
about the actually compiling/running any code yet! If each such facet of a build
was modelled separately, it's easy to have an explosion of different concepts
that would make a build tool hard to understand.

Before you continue, take a moment to think: how would you answer to each of
those questions using an existing build tool you are familiar with? Different
tools like [SBT](http://www.scala-sbt.org/),
[Fake](https://fake.build/legacy-index.html), [Gradle](https://gradle.org/) or
[Grunt](https://gruntjs.com/) have very different answers.

Mill aims to provide the answer to these questions using as few, as familiar
core concepts as possible. The entire Mill build is oriented around a few
concepts:

- The Object Hierarchy
- The Call Graph
- Instantiating Traits & Classes

These concepts are already familiar to anyone experienced in Scala (or any other
programming language...), but are enough to answer all of the complicated
build-related questions listed above.

## The Object Hierarchy

The module hierarchy is the graph of objects, starting from the root of the
`build.sc` file, that extend `mill.Module`. At the leaves of the hierarchy are
the `Target`s you can run.

A `Target`'s position in the module hierarchy tells you many things. For
example, a `Target` at position `Core.test.compile` would:

- Cache output metadata at `out/Core/test/compile.mill.json`

- Output files to the folder `out/Core/test/compile/`

- Be runnable from the command-line via `mill run Core.test.compile`

- Be referenced programmatically (from other `Target`s) via `Core.test.compile`

From the position of any `Target` within the object hierarchy, you immediately
know how to run it, find it's output files, find any caches, or refer to it from
other `Target`s. You know up-front where the `Target`'s data "lives" on disk, and
are sure that it will never clash with any other `Target`'s data.

## The Call Graph

The Scala call graph of "which target references which other target" is core to
how Mill operates. This graph is reified via the `T{...}` macro to make it
available to the Mill execution engine at runtime. The call graph tells you:

- Which `Target`s depend on which other `Target`s

- For a given `Target` to be built, what other `Target`s need to be run and in
what order

- Which `Target`s can be evaluated in parallel

- What source files need to be watched when using `--watch` on a given target (by
tracing the call graph up to the `Source`s)

- What a given `Target` makes available for other `Target`s to depend on (via
it's return value)

- Defining your own task that depends on others is as simple as `def foo =
T{...}`

The call graph within your Scala code is essentially a data-flow graph: by
defining a snippet of code:

```scala
val b = ...
val c = ...
val d = ...
val a = f(b, c, d)
```

you are telling everyone that the value `a` depends on the values of `b` `c` and
`d`, processed by `f`. A build tool needs exactly the same data structure:
knowing what `Target` depends on what other `Target`s, and what processing it
does on its inputs!

With Mill, you can take the Scala call graph, wrap everything in the `T{...}`
macro, and get a `Target`-dependency graph that matches exactly the call-graph
you already had:

```scala
val b = T{ ... }
val c = T{ ... }
val d = T{ ... }
val a = T{ f(b(), c(), d()) }
```

Thus, if you are familiar with how data flows through a normal Scala program,
you already know how data flows through a Mill build! The Mill build evaluation
may be incremental, it may cache things, it may read and write from disk, but
the fundamental syntax, and the data-flow that syntax represents, is unchanged
from your normal Scala code.

## Instantiating Traits & Classes

Classes and traits are a common way of re-using common data structures in Scala:
if you have a bunch of fields which are related and you want to make multiple
copies of those fields, you put them in a class/trait and instantiate it over
and over.

In Mill, inheriting from traits is the primary way for re-using common parts of
a build:

- Scala "project"s with multiple related `Target`s within them, are just a
`Trait` you instantiate

- Replacing the default `Target`s within a project, making them do new
things or depend on new `Target`s, is simply `override`-ing them during
inheritence.

- Modifying the default `Target`s within a project, making use of the old value
to compute the new value, is simply `override`ing them and using `super.foo()`

- Required configuration parameters within a `project` are `abstract` members.

- Cross-builds are modelled as instantiating a (possibly anonymous) class
multiple times, each instance with it's own distinct set of `Target`s

In normal Scala, you bundle up common fields & functionality into a `class` you
can instantiate over and over, and you can override the things you want to
customize. Similarly, in Mill, you bundle up common parts of a build into
`trait`s you can instantiate over and over, and you can override the things you
want to customize. "Subprojects", "cross-builds", and many other concepts are
reduced to simply instantiating a `trait` over and over, with tweaks.

## Prior Work

### SBT
Expand Down
17 changes: 9 additions & 8 deletions scalaplugin/src/test/scala/mill/scalaplugin/AcyclicTests.scala
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,9 @@ object AcyclicTests extends TestSuite{
val packageScala = workspacePath/'src/'main/'scala/'acyclic/"package.scala"

'acyclic - {
val scalaVersion = "2.12.4"
// We can compile
val Right((pathRef, evalCount)) = eval(AcyclicBuild.acyclic("2.12.4").compile)
val Right((pathRef, evalCount)) = eval(AcyclicBuild.acyclic(scalaVersion).compile)
val outputPath = pathRef.path
val outputFiles = ls.rec(outputPath)
assert(
Expand All @@ -73,33 +74,33 @@ object AcyclicTests extends TestSuite{
)

// Compilation is cached
val Right((_, evalCount2)) = eval(AcyclicBuild.acyclic("2.12.4").compile)
val Right((_, evalCount2)) = eval(AcyclicBuild.acyclic(scalaVersion).compile)
assert(evalCount2 == 0)

write.append(packageScala, "\n")

// Caches are invalidated if code is changed
val Right((_, evalCount3)) = eval(AcyclicBuild.acyclic("2.12.4").compile)
val Right((_, evalCount3)) = eval(AcyclicBuild.acyclic(scalaVersion).compile)
assert(evalCount3 > 0)

// Compilation can fail on broken code, and work when fixed
write.append(packageScala, "\n}}")
val Left(Result.Exception(ex)) = eval(AcyclicBuild.acyclic("2.12.4").compile)
val Left(Result.Exception(ex)) = eval(AcyclicBuild.acyclic(scalaVersion).compile)
assert(ex.isInstanceOf[sbt.internal.inc.CompileFailed])

write.write(packageScala, read(packageScala).dropRight(3))
val Right(_) = eval(AcyclicBuild.acyclic("2.12.4").compile)
val Right(_) = eval(AcyclicBuild.acyclic(scalaVersion).compile)

// Tests compile & run
val Right(_) = eval(AcyclicBuild.acyclic("2.12.4").test.forkTest())
val Right(_) = eval(AcyclicBuild.acyclic(scalaVersion).test.forkTest())

// Tests can be broken
write.append(packageScala, "\n}}")
val Left(_) = eval(AcyclicBuild.acyclic("2.12.4").test.forkTest())
val Left(_) = eval(AcyclicBuild.acyclic(scalaVersion).test.forkTest())

// Tests can be fixed
write.write(packageScala, read(packageScala).dropRight(3))
val Right(_) = eval(AcyclicBuild.acyclic("2.12.4").test.forkTest())
val Right(_) = eval(AcyclicBuild.acyclic(scalaVersion).test.forkTest())
}

}
Expand Down

0 comments on commit c6d95ab

Please sign in to comment.