Skip to content

Commit

Permalink
Track added products for cache invalidation
Browse files Browse the repository at this point in the history
I discovered that in sbt, the scripted tests/internal-tracking test
would fail if run with -Dsbt.resident.limit=$NUM where $NUM > 0. This
was because the test introduced a dependent project b, that depended on
a, but was set to not track a. If the b project was compiled before the
a project, the compilation would fail, but a cache entry would be added
to the compiler cache with a key that included a's target directory in
its classpath. If a was then independently compiled, the expectation
would be that b would also compile, but this actually failed. The reason
was that when zinc looked for the cached compiler, it found a hit and
did not create a new compiler instance. This was a problem because the
previous instance was unaware of the newly generated classes from the a
project. To get b to compile, it is necessary to invalidate the compiler
cache entry.

To invalidate the entry, we must detect that there are changes to the
project's dependencies. Previously, zinc did not even look at new
classes in a dependencies target directory. It did look at
removedProducts, but did not actually consider the removed products when
constructing the DependencyChanges instance in Incremental.scala. To
resolve this, I just append the removed and new products to the binary
dependency changes. After this change,
`scripted project/internal-tracking` passes even when a global compiler
cache is used.
  • Loading branch information
eatkins committed Aug 9, 2018
1 parent eddf911 commit 4761a63
Showing 1 changed file with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import xsbti.compile.{
import xsbti.compile.analysis.{ ReadStamps, Stamp => XStamp }

import scala.annotation.tailrec
import scala.collection.JavaConverters._

private[inc] abstract class IncrementalCommon(val log: sbt.util.Logger, options: IncOptions) {

Expand Down Expand Up @@ -230,12 +231,15 @@ private[inc] abstract class IncrementalCommon(val log: sbt.util.Logger, options:
.filter(p => !equivS.equiv(previous.product(p), current.product(p)))
.toSet
}
val allProducts =
lookup.analyses.flatMap(_.readStamps().getAllProductStamps.keySet.asScala).toSet
val newProducts = allProducts -- previous.allProducts

val binaryDepChanges = lookup.changedBinaries(previousAnalysis).getOrElse {
previous.allBinaries
.filter(externalBinaryModified(lookup, previous, current, previousRelations))
.toSet
}
} ++ newProducts ++ removedProducts

val incrementalExtApiChanges = changedIncremental(previousAPIs.allExternals,
previousAPIs.externalAPI,
Expand Down

0 comments on commit 4761a63

Please sign in to comment.