Commits on Nov 10, 2017

1. Make classpath hashing more lightweight …

   And make it parallel!
   
   This patch adds a cache that relies on filesystem metadata to cache
   hashes for jars that have the same last modified time across different
   compiler iterations. This is important because until now there was a
   significant overhead when running `compile` on multi-module builds that
   have gigantic classpaths. In this scenario, the previous algorithm
   computed hashes for all jars transitively across all these projects.
   
   This patch is conservative; there are several things that are wrong with
   the status quo of classpath hashing. The most important one is the fact
   that Zinc has been doing `hashCode` on a SHA-1 checksum, which doesn't
   make sense. The second one is that we don't need a SHA-1 checksum for
   the kind of checks we want to do. sbt#371
   explains why. The third limitation with this check is that file hashes
   are implemented internally as `int`s, which is not enough to represent
   the richness of the checksum. My previous PR also tackles this problem,
   which will be solved in the long term.
   
   Therefore, this pull request only tackles these two things:
   
   * Caching of classpath entry hashes.
   * Parallelize this IO-bound task.
   
   Results, on my local machine:
   
   - No parallel hashing of the first 500 jars in my ivy cache: 1330ms.
   - Parallel hashing of the first 500 jars in my ivy cache: 770ms.
   - Second parallel hashing of the first 500 jars in my ivy cache: 1ms.
   
   Fixes sbt#433.

   

   jvican committed Nov 10, 2017
   Configuration menu

   • View commit details
   • Copy full SHA for a90a0e9
   • Browse repository at this point

   Copy the full SHA

   a90a0e9 View commit details

   Browse the repository at this point in the history