Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use FileFunction.cached #213

Merged
merged 3 commits into from
Jun 12, 2023

Conversation

regiskuckaertz
Copy link
Contributor

hello 🤠 I have been investigating an issue in our pipeline that happens between our build stage where we generate sources and compile, and the test stage where we run tests. We are getting IO errors during the test stage that look like this:

sbt:hello-world> test
[info] compiling 12 Scala sources to /Users/regiskuckaertz/code/guard1/hw2/target/scala-2.13/classes ...
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/AkkaHttpImplicits.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/AkkaHttpImplicits.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/Client.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/Client.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/Implicits.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/Implicits.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/ApiResponse.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/ApiResponse.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Category.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Category.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Order.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Order.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Pet.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Pet.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Tag.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/Tag.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/User.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/User.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/package.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/definitions/package.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] IO error while decoding /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/support/Presence.scala with UTF-8: /Users/regiskuckaertz/code/guard1/hello-world/target/scala-2.13/src_managed/main/com/regiskuckaertz/support/Presence.scala (No such file or directory)
[error] Please try specifying another one using the -encoding option
[error] 11 errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 2 s, completed 10 Jun 2023, 18:46:11%                                                                                                                                                     

I noticed the paths listed in the errors were the absolute paths of the sources generated during the compile stage. Of course they don't exist anymore since the build stage runs in a separate runner with a different base directory. The only place I could think of where absolute paths would be persisted on disk was in the SBT caching (the streams task). It seems that something shady is happening with Tracked.inputChanged, as after replacing with FileFunction.cached I'm unable to reproduce the issue. To reproduce here is what I did:

  • create a new sbt project under directory hello1, with an openapi spec, and a test using the generated code
  • sbt clean compile
  • exit the project
  • rename hello1 to hello2
  • sbt test

That would fail consistently, but not anymore.

I also took the liberty of wrapping the task with a try/finally to make sure the classloader is restored in case an exception is thrown.

Copy link
Member

@blast-hardcheese blast-hardcheese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much simpler, very much appreciated!

@blast-hardcheese
Copy link
Member

I also took the liberty of wrapping the task with a try/finally to make sure the classloader is restored in case an exception is thrown.

I would have expected (hoped?) that that'd have been restored at the end of the Task (🙈)

either way, great to have

@blast-hardcheese blast-hardcheese merged commit 26db8e2 into guardrail-dev:master Jun 12, 2023
@blast-hardcheese blast-hardcheese added the bug Something isn't working label Jun 12, 2023
@blast-hardcheese
Copy link
Member

https://github.com/guardrail-dev/sbt-guardrail/releases/tag/v0.75.2 has been cut!

@regiskuckaertz
Copy link
Contributor Author

@blast-hardcheese Thanks heaps 🙏 have a great day

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants