Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Checksum failure on Github CI while fetching dependencies #1910

Closed
lefou opened this issue Jul 1, 2022 · 4 comments · Fixed by #2058
Closed

Checksum failure on Github CI while fetching dependencies #1910

lefou opened this issue Jul 1, 2022 · 4 comments · Fixed by #2058
Milestone
0.10.8

Comments

@lefou
Copy link
Member

lefou commented Jul 1, 2022 •
edited
Loading

Sometimes, Mill can't fetch some dependencies. I'm not able to reproduce locally, but it can be seen occasionally on Github CI, e.g. in this run: https://github.com/com-lihaoyi/mill/runs/6181553905?check_suite_focus=true

The error message looks like this:

1 targets failed
scalalib.scalaDocClasspath java.nio.file.NoSuchFileException: /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.9.0/.jna-5.9.0.jar__sha1.computed1274112506791367117.tmp -> /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.9.0/.jna-5.9.0.jar__sha1.computed

An retry typically fixes the issue.

Although the issue can be seen now and then on Github CI, the dependency on which it actually fails is pretty random.

Expand for a more complete log... 
2022-04-26T18:42:35.2215843Z + ./mill -i -j 0 installLocal --binFile target/mill-1
2022-04-26T18:42:36.3942103Z Preparing Java 11.0.15 runtime; this may take a minute or two ...
2022-04-26T18:42:44.4974024Z Compiling /home/runner/work/mill/mill/ci/shared.sc
2022-04-26T18:42:49.3204603Z Compiling /home/runner/work/mill/mill/ci/upload.sc
2022-04-26T18:42:50.5143058Z Compiling /home/runner/work/mill/mill/build.sc
2022-04-26T18:43:08.5515122Z [#1] [2/2629] bsp.scalaVersion 
...
2022-04-26T18:43:12.0530854Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath 
2022-04-26T18:43:12.0538027Z �[1A�[2K�[9999D[#1] [1134/2629] scalalib.scalaDocClasspath 
2022-04-26T18:43:12.0625080Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [1/1] artifacts (~0/0 bytes)
2022-04-26T18:43:12.0654554Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~0/0 bytes)
2022-04-26T18:43:12.0693936Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~0/1824 bytes)
2022-04-26T18:43:12.0706104Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~1824/1824 bytes)
2022-04-26T18:43:12.0715463Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~1824/1824 bytes)
2022-04-26T18:43:12.0745739Z �[1A�[2K�[9999D[#1] [1134/2629] scalalib.scalaDocClasspath | Downloading [1/1] artifacts (~0/0 bytes)
2022-04-26T18:43:12.0767630Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~1824/4162 bytes)
2022-04-26T18:43:12.0768522Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~4162/4162 bytes)
2022-04-26T18:43:12.0796091Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [2/2] artifacts (~4162/4162 bytes)
2022-04-26T18:43:12.0851666Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [3/3] artifacts (~4162/4162 bytes)
2022-04-26T18:43:12.0882948Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~4162/4162 bytes)
2022-04-26T18:43:12.0954881Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~4162/21849 bytes)
2022-04-26T18:43:12.0955693Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~20061/21849 bytes)
2022-04-26T18:43:12.0956923Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~21849/21849 bytes)
2022-04-26T18:43:12.0957609Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~21849/21849 bytes)
2022-04-26T18:43:12.0977494Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~21849/23878 bytes)
2022-04-26T18:43:12.0978316Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~23878/23878 bytes)
2022-04-26T18:43:12.0984342Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [4/4] artifacts (~23878/23878 bytes)
2022-04-26T18:43:12.1031584Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [5/5] artifacts (~23878/23878 bytes)
2022-04-26T18:43:12.1036630Z �[1A�[2K�[9999D[#1] [1134/2629] scalalib.scalaDocClasspath | Downloading [1/1] artifacts (~2338/2338 bytes)
2022-04-26T18:43:12.1037570Z �[1A�[2K�[9999D[#1] [1134/2629] scalalib.scalaDocClasspath | Downloading [1/1] artifacts (~2338/2338 bytes)
2022-04-26T18:43:12.1116542Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [5/5] artifacts (~23878/47684 bytes)
2022-04-26T18:43:12.1117382Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [5/5] artifacts (~39776/47684 bytes)
2022-04-26T18:43:12.1121076Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [5/5] artifacts (~47684/47684 bytes)
2022-04-26T18:43:12.1121856Z �[1A�[2K�[9999D[#0] [1133/2629] bsp.scalaDocClasspath | Downloading [5/5] artifacts (~47684/47684 bytes)
2022-04-26T18:43:12.8447142Z 1 targets failed
2022-04-26T18:43:12.8454229Z scalalib.scalaDocClasspath java.nio.file.NoSuchFileException: /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.9.0/.jna-5.9.0.jar__sha1.computed1274112506791367117.tmp -> /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.9.0/.jna-5.9.0.jar__sha1.computed
2022-04-26T18:43:12.8790044Z ##[error]Process completed with exit code 1.

This is probably an issue in courier, but there is no further analysis yet.
Linking @ollyw here, as we talked about that issue in
@lefou lefou mentioned this issue Jul 1, 2022
Closed
@lefou
Copy link
Member Author

lefou commented Jul 1, 2022

Here are some probably related coursier issues:

@lefou
Copy link
Member Author

lefou commented Jul 4, 2022

Latest event:

contrib.playlib.worker[2.6].scalaDocClasspath java.nio.file.NoSuchFileException: /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.15/.scala-reflect-2.12.15.jar__sha1.computed2468770752871636220.tmp -> /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.15/.scala-reflect-2.12.15.jar__sha1.computed

https://github.com/com-lihaoyi/mill/runs/7177350316?check_suite_focus=true

@ajrnz
Copy link
Collaborator

ajrnz commented Jul 30, 2022

I have just come across a similar problem. I'd see stuff like:

  com.google.protobuf:protobuf-parent:3.7.0 
    checksum not found: /builds/development/scala-common/.cache/coursier/v1/http/artifactory.trackabus.int/artifactory/trackabus/com/google/protobuf/protobuf-parent/3.7.0/protobuf-parent-3.7.0.pom
serverlib[2.13.8].scalaCompilerClasspath Failed to load source dependencies
  concurrent download: http://artifactory.trackabus.int/artifactory/trackabus/org/scala-lang/scala-compiler/2.13.8/scala-compiler-2.13.8.jar
serverlib[2.13.8].test.scalaCompilerClasspath Failed to load source dependencies
  concurrent download: http://artifactory.trackabus.int/artifactory/trackabus/org/scala-lang/scala-compiler/2.13.8/scala-compiler-2.13.8.jar
serverlib[3.1.1].scalaCompilerClasspath 
Resolution failed for 1 modules:

It was resolved when I removed the -j 0 flag from mill. So I'm guessing that two targets are running in parallel which need to download dependencies and they both try to use coursier at the same time and this is causing an issue? (I was starting with a clean cache). Maybe calls to download artifacts need to be serialized?

@lefou
Copy link
Member Author

lefou commented Aug 2, 2022

@ajrnz Those concurrency issues are already fixed (via workarounds) in Mill 0.10.4+

This issue is only about those checksum issues, which don't seem to be caused by parallel builds, as they occur on GitHub CI even in builds with --jobs 1.

@lefou lefou mentioned this issue Oct 6, 2022
Merged
lefou added a commit that referenced this issue Oct 6, 2022
@lefou
Retry coursier artifact loading when "checksum not found" (#2058)
f96162e 
We reuse the same retry logic as we do for retrying concurrent download
issues.

Fix #1910

Pull request: #2058
@lefou lefou added this to the 0.10.8 milestone Oct 6, 2022
@lefou lefou mentioned this issue Nov 9, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.10.8
Development

Successfully merging a pull request may close this issue.

Retry coursier artifact loading when "checksum not found" lefou/mill
2 participants
@lefou @ajrnz