From 1ab93d9c4427075024ffdd897a5d2f10c2ade763 Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Fri, 15 Jul 2022 10:53:35 +0200 Subject: [PATCH 1/9] Scala CLI SIP --- content/scala-cli.md | 148 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 content/scala-cli.md diff --git a/content/scala-cli.md b/content/scala-cli.md new file mode 100644 index 00000000..1f30cd50 --- /dev/null +++ b/content/scala-cli.md @@ -0,0 +1,148 @@ +--- +layout: sip +permalink: /sips/:title.html +stage: implementation +status: waiting-for-implementation +title: SIP-NN Scala CLI as default Scala command +--- + +**By: Krzysztof Romanowski and Scala CLI team** + +## History + +| Date | Version | +|---------------|--------------------| +| July 15th 2022 | Initial Draft | + +## Summary + +We propose to replace current scripts that are installed as "Scala": `scala`, `scalac` and `scaladoc` with Scala CLI - a batteries included tool to interact with Scala. Scala CLI brings all the features that the commands above provide and expand them with incremental compilation, dependency management, packaging and much more. + + +## Motivation + +The current default Scala scripts: `scala`, `scalac`, `scaladoc` are quite limited. Beside that, the `scala` command that starts the Scala REPL is hardly used by non-power users. + +The current scripts are lacking basic features such as support for resolving dependencies, incremental compilation or support for outputs other than JVM. This forces any user that wants to do anything more than just basic things to learn and use SBT, Mill or an other build tool and that adds to the complexity of learning Scala. + +We observe that the current state of tooling in Scala is limiting creativity, with quite a high cost to create e.g. an application or a script with some dependencies that target Node.js. Many Scala developers are not choosing Scala for their personal projects, scripts, or small applications and we believe that the complexity of setting up a build tool is one of the reasons. + +With this proposal our main goal is to turn Scala into a language with "batteries included" that will also respect the community-first aspect of our ecosystem. + +## Proposed solution + +We propose to gradually replace the current `scala`, `scalac` and `scaladoc` commands by single `scala` command that under the hood will be `scala-cli`. We could also add wrapper scripts for `scalac` and `scaladoc` that will mimic the functionality that will use `scala-cli` under the hood. + +The complete set of `scala-cli` features can be found in [its documentation](https://scala-cli.virtuslab.org/docs/overview). + +Scala CLI brings many features like testing, packaging, exporting to sbt / Mill or upcoming support for publishing micro-libraries. Initially, we may want to limit the set of features available in the `scala` command by default. Scala CLI is a relatively new project and we should battle-proof some of its features before we commit to support them as part of the offical `scala` command. + +Scala CLI offers [multiple native ways to be installed](https://scala-cli.virtuslab.org/install#advanced-installation) so most users should find a suitable method. We would like these packages to become the default `scala` package in most repositories, often replacing existing `scala` packages. + +### High-level overview + +Let us show a few examples where adopting Scala CLI as `scala` command would be a significant improvement ofer current scripts. For this, we have assumed a minial set of features. Each additionnal Scala CLI feature included, such as `package`, would add more and more use cases. + +**Using REPL with a 3rd-party dependency** + +Currently, to start a Scala REPL with a dependency on the class path, users need to resolve this dependency with all its transitive dependencies (coursier can help here) and pass those to the `scala` command using the `--cp` option. Alternatively, one can create an sbt project including a single dependency and use the `sbt console` task. Ammonite gives a better experience with its magic imports. + +With Scala CLI, starting a REPL with a given dependency is as simple as running: + +```scala-cli repl --dep com.lihaoyi::os-lib:0.7.8``` + +Compared to Ammonite, default Scala REPLs provided by Scala 2 and 3 - that Scala CLI uses by default - are somewhat limited. However, Scala CLI also offers to start Ammonite instead of the default Scala REPL, by passing `--ammonite` (or `--amm`) option to `scala-cli repl`. + +Additionally, `scala-cli repl` can also put code from given files / directories / snippets on the class path by just providing their locations as arguments. Running ```scala-cli repl foo.scala baz``` will compile code from `foo.scala` and the `baz` directory, and put their classes on the REPL class path (including their dependencies, scalac options etc. defined within those files). + +Compilation (and running scaladoc as well) benefit in a similar way from the ability to manage dependencies. + +** Providing reproductions of bugs ** + +Currently, when reporting a bug in the compiler (or any other Scala-related) repository, users need to provide depencencies, compiler options etc. in comments, create a repository containing a projet with a Mill / sbt configuration to reproduce. In general, testing the reporoduction or working on further minization is not straightworwad. + +"Using directives", provided by Scala CLI give the ablity to include the whole configuration in single file, for example: + +```scala +//> using platform "native" +//> using "com.lihaoyi::os-lib:0.7.8" +//> using options "-Xfatal-warnings" + +def foo = println("") +``` + +The snippet above when run with Scala CLI without any configuration provided will use Scala Native, resolve and include `os-lib` and provide `-Xfatal-warnings` to the compiler. Even things such as the runtime JVM version can be configured with using directives. + +Moreover, Scala CLI provides the ability to run GitHub gists (including multi-file ones), and more. + +** Watch mode ** + +When working on a piece of code, it is often useful to have it compiled/run everytime the file is changed, and build tools offer a watch mode for that. This is how most people are using watch mode through a build tool. Scala CLI offers a watch mode for most of its commands (by using `--watch` / `-w` flags). + + +### Specification + + In order to be able to expand the functionality of Scala CLI and yet use the same core to power the `scala` command, we propose to include both `scala` and `scala-cli` commands in the installation package. Scala CLI already has a feature to limit accessible sub-commands based the binary name (all sub-commands in `scala-cli`, and a curated list in `scala`). Later, we can include more and more features from `scala-cli` into `scala`. + +The sub-commands necessary to include in the `scala` to match the functionalities of current commands: + - `compile` + - `run` + - `repl` + - `doc` + +On top of that, we think that the following user-facing sub-commands should also be included: + - `clean` - to rebuild project from start without any cached steps + - `setup-ide` - to control the setup of BSP for IDE integration + - `doctor` - to analyze if everything is installed properly + +We also suggest to include additional sub-commands by default: + - `fmt` - to format the code using scalafmt + - `test` - to run tests included in the code + - `package` - to package the code into various package formats: JAR, "fat" be, executable be or even native application or docker image + - `shebang` - a command useful for scripting, designed to be included in the "shebang" section of scripts + - `export` - transform current project to sbt / Mill project using the same settings as provided. Useful to evolve prototypes into bigger projects + +Each of these commands expands what the current scripts offer and can be discussed separately. We can even open a dedicated SIP for each of them. + +Beyond that, `scala-cli` offers multiple sub-commands needed to manage itself (e.g. `update`) or its components (e.g. the Bloop server). In most cases, +these are not user-facing, but still handy. We can elaborate in more detail on what those commands are and why we need them, if necessary. + +Scala CLI can also be configured with ["using directives"](https://scala-cli.virtuslab.org/docs/guides/using-directives) - a comment-based configuration syntax that should be placed at the top of Scala files. This allows for self-containing examples within one file since most of the configuration can be provided either from the command line or via using directives (command line has precedence). This is a game changer for use cases like scripting, reproduction, or within the academic scope. + +We have described the motivation, syntax and implementation basis in the [dedicated pre-SIP](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700). Currently, we recommend to write using directives as comments, so making them part of the language specification is not necessary at this stage. Moreover, the new `scala` command could ignore using directives in the initial version, however we strongly suggest to include comment-based using directives from the start. + +### Compatibility + +Adopting Scala CLI as the new `scala` command, as is, will change some of the behaviour of today's scripts. Some examples: + +- `scala repl` needs to be run to start a REPL instead of just `scala` +- with `scala compile` and `scala doc`, some of the more obscure (or brand new) compile options need to be prefixed with `-O` +- Scala CLI recognizes tests based on the extension used (`*.test.scala`) so running `scala compile a.scala a.test.scala` will only compile `a.scala` +- Scala CLI has its own versioning scheme, that is not related to the Scala compiler. Default version used may dynamically change when new Scala version is released. +- By default, Scala CLI manages its own dependencies (e.g. scalac, zinc, Bloop) and resolves them lazily. This means that the first run of Scala CLI resolves quite some dependencies. Moreover, Scala CLI periodically checks for updates, new defaults accessing online resources (but it is not required to work, so Scala CLI can work in offline environment once setup) +- Scala CLI can also be configured via using directives. Command line options have precedence over using directives, however using directives override defaults. Compiling a file starting with `//> using scala 2.13.8`, without providing a Scala version on the command line, will result in using `2.13.8` rather than the default Scala version. We consider this a feature. However, technically, this is a breaking change. + +### Other concerns + +Scala CLI brings [using directives](https://scala-cli.virtuslab.org/docs/guides/using-directives) and [conventions to mark the test files](https://scala-cli.virtuslab.org/docs/commands/test#test-sources). We are not sure if both can be accepted as a part of this SIP or we should have seperate SIPs for both (we have opened a [pre-SIP for using directives](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700/15)) + +Scala CLI is an ambitious project and may seem hard to maintain in the long-run. // TODO + + +### Open questions + +The main open question for this proposal is wich commands/features should be included by default in the `scala` command. Another aspect is the release cadence: should the new `scala` command follow the current release cadence for Scala CLI (every 2 weeks) or stick to Scala one (every 6 weeks)? + +## Alternatives + +Scala CLI has many alternatives. The most obvious ones are sbt, Mill, or other build tools. However, these are more complicated than Scala CLI, and what is more important they are not designed as command-line first tools. Ammonite, is another alternative, however it covers only part of the Scala CLI features (REPL and scripting), and lacks many of the Scala CLI features (incremental compilation, Scala version selection, support for Scala.js and Scala Native, just to name a few). + +## Related work + +- [Scala CLI website](https://scala-cli.virtuslab.org/) and [road map](https://github.com/VirtusLab/scala-cli/discussions/1101) +- [Pre-SIP](https://contributors.scala-lang.org/t/pre-sip-scala-cli-as-new-scala-command/5628/22) +- [leiningen](https://leiningen.org/) - a similar tool from Closure, but more configuration-oriented + +## FAQ + +This section will probably initially be empty. As discussions on the proposal progress, it is likely that some questions will come repeatedly. They should be listed here, with appropriate answers. From aa05b4dbd47c3d8993e32bfb34fa9a8c2afb5fca Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Fri, 15 Jul 2022 11:52:47 +0200 Subject: [PATCH 2/9] Formatting fixes Co-authored-by: Jamie Thompson --- content/scala-cli.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/scala-cli.md b/content/scala-cli.md index 1f30cd50..f8feae5e 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -49,11 +49,13 @@ Currently, to start a Scala REPL with a dependency on the class path, users need With Scala CLI, starting a REPL with a given dependency is as simple as running: -```scala-cli repl --dep com.lihaoyi::os-lib:0.7.8``` +``` +scala-cli repl --dep com.lihaoyi::os-lib:0.7.8 +``` Compared to Ammonite, default Scala REPLs provided by Scala 2 and 3 - that Scala CLI uses by default - are somewhat limited. However, Scala CLI also offers to start Ammonite instead of the default Scala REPL, by passing `--ammonite` (or `--amm`) option to `scala-cli repl`. -Additionally, `scala-cli repl` can also put code from given files / directories / snippets on the class path by just providing their locations as arguments. Running ```scala-cli repl foo.scala baz``` will compile code from `foo.scala` and the `baz` directory, and put their classes on the REPL class path (including their dependencies, scalac options etc. defined within those files). +Additionally, `scala-cli repl` can also put code from given files / directories / snippets on the class path by just providing their locations as arguments. Running `scala-cli repl foo.scala baz` will compile code from `foo.scala` and the `baz` directory, and put their classes on the REPL class path (including their dependencies, scalac options etc. defined within those files). Compilation (and running scaladoc as well) benefit in a similar way from the ability to manage dependencies. From c5bea7d81ff75eab6796ed5f532e8dce44866de7 Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Tue, 30 Aug 2022 16:20:42 +0200 Subject: [PATCH 3/9] Add lists of supported commands, options and directives --- content/scala-cli.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/content/scala-cli.md b/content/scala-cli.md index f8feae5e..69969089 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -113,6 +113,12 @@ Scala CLI can also be configured with ["using directives"](https://scala-cli.vir We have described the motivation, syntax and implementation basis in the [dedicated pre-SIP](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700). Currently, we recommend to write using directives as comments, so making them part of the language specification is not necessary at this stage. Moreover, the new `scala` command could ignore using directives in the initial version, however we strongly suggest to include comment-based using directives from the start. +#### Commands, options and directives + +We have generated a complete lists of supported [commands](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/commands.md), (CLI options)[https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/cli-options.md] and (directives)[https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/directives.md] that we propose for new `scala` command. Lists contain many options and commands that are marked as `experimental` and/or `internal` and as such, we could change them at any point. + +The lists was generated automatically and starting from next release of Scala CLI such documentation will be included on our website. Using that lists, it will be easy to track and spot any changes in exposed APIs. + ### Compatibility Adopting Scala CLI as the new `scala` command, as is, will change some of the behaviour of today's scripts. Some examples: From f3c5322bed073048a13a488ff8d023ab89d455f4 Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Tue, 30 Aug 2022 16:29:21 +0200 Subject: [PATCH 4/9] Proposal targets only scala commands, keeping current scalac and scaladoc --- content/scala-cli.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/content/scala-cli.md b/content/scala-cli.md index 69969089..e353b36e 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -16,14 +16,16 @@ title: SIP-NN Scala CLI as default Scala command ## Summary -We propose to replace current scripts that are installed as "Scala": `scala`, `scalac` and `scaladoc` with Scala CLI - a batteries included tool to interact with Scala. Scala CLI brings all the features that the commands above provide and expand them with incremental compilation, dependency management, packaging and much more. +We propose to replace current script that is installed as `scala` with Scala CLI - a batteries included tool to interact with Scala. Scala CLI brings all the features that the commands above provide and expand them with incremental compilation, dependency management, packaging and much more. + +Even though Scala CLI could replace `scaladoc` and `scalac` commands as well for now, we do not propose to replace them. ## Motivation -The current default Scala scripts: `scala`, `scalac`, `scaladoc` are quite limited. Beside that, the `scala` command that starts the Scala REPL is hardly used by non-power users. +The current default `scala` script is quite limited since it can only start repl or run pre-compile Scala code. -The current scripts are lacking basic features such as support for resolving dependencies, incremental compilation or support for outputs other than JVM. This forces any user that wants to do anything more than just basic things to learn and use SBT, Mill or an other build tool and that adds to the complexity of learning Scala. +The current script are lacking basic features such as support for resolving dependencies, incremental compilation or support for outputs other than JVM. This forces any user that wants to do anything more than just basic things to learn and use SBT, Mill or an other build tool and that adds to the complexity of learning Scala. We observe that the current state of tooling in Scala is limiting creativity, with quite a high cost to create e.g. an application or a script with some dependencies that target Node.js. Many Scala developers are not choosing Scala for their personal projects, scripts, or small applications and we believe that the complexity of setting up a build tool is one of the reasons. @@ -41,7 +43,7 @@ Scala CLI offers [multiple native ways to be installed](https://scala-cli.virtus ### High-level overview -Let us show a few examples where adopting Scala CLI as `scala` command would be a significant improvement ofer current scripts. For this, we have assumed a minial set of features. Each additionnal Scala CLI feature included, such as `package`, would add more and more use cases. +Let us show a few examples where adopting Scala CLI as `scala` command would be a significant improvement ofer current scripts. For this, we have assumed a minial set of features. Each additional Scala CLI feature included, such as `package`, would add more and more use cases. **Using REPL with a 3rd-party dependency** From d5add3acabb41c3785f64f41cb3f8f45e2f6f02d Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Tue, 30 Aug 2022 17:36:46 +0200 Subject: [PATCH 5/9] Update compatibility - remote section about scalac and scaladoc - remote section about defaulting to repl - fix formatting --- content/scala-cli.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/content/scala-cli.md b/content/scala-cli.md index e353b36e..51765025 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -117,7 +117,7 @@ We have described the motivation, syntax and implementation basis in the [dedica #### Commands, options and directives -We have generated a complete lists of supported [commands](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/commands.md), (CLI options)[https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/cli-options.md] and (directives)[https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/directives.md] that we propose for new `scala` command. Lists contain many options and commands that are marked as `experimental` and/or `internal` and as such, we could change them at any point. +We have generated a complete lists of supported [commands](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/commands.md), [CLI options](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/cli-options.md) and [directives](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/directives.md) that we propose for new `scala` command. Lists contain many options and commands that are marked as `experimental` and/or `internal` and as such, we could change them at any point. The lists was generated automatically and starting from next release of Scala CLI such documentation will be included on our website. Using that lists, it will be easy to track and spot any changes in exposed APIs. @@ -125,10 +125,8 @@ The lists was generated automatically and starting from next release of Scala CL Adopting Scala CLI as the new `scala` command, as is, will change some of the behaviour of today's scripts. Some examples: -- `scala repl` needs to be run to start a REPL instead of just `scala` -- with `scala compile` and `scala doc`, some of the more obscure (or brand new) compile options need to be prefixed with `-O` - Scala CLI recognizes tests based on the extension used (`*.test.scala`) so running `scala compile a.scala a.test.scala` will only compile `a.scala` -- Scala CLI has its own versioning scheme, that is not related to the Scala compiler. Default version used may dynamically change when new Scala version is released. +- Scala CLI has its own versioning scheme, that is not related to the Scala compiler. Default version used may dynamically change when new Scala version is released. Similarly to Scala 3, we intend for Scala CLI to be backward compatible and this should help mitigate this risk. - By default, Scala CLI manages its own dependencies (e.g. scalac, zinc, Bloop) and resolves them lazily. This means that the first run of Scala CLI resolves quite some dependencies. Moreover, Scala CLI periodically checks for updates, new defaults accessing online resources (but it is not required to work, so Scala CLI can work in offline environment once setup) - Scala CLI can also be configured via using directives. Command line options have precedence over using directives, however using directives override defaults. Compiling a file starting with `//> using scala 2.13.8`, without providing a Scala version on the command line, will result in using `2.13.8` rather than the default Scala version. We consider this a feature. However, technically, this is a breaking change. @@ -136,7 +134,7 @@ Adopting Scala CLI as the new `scala` command, as is, will change some of the be Scala CLI brings [using directives](https://scala-cli.virtuslab.org/docs/guides/using-directives) and [conventions to mark the test files](https://scala-cli.virtuslab.org/docs/commands/test#test-sources). We are not sure if both can be accepted as a part of this SIP or we should have seperate SIPs for both (we have opened a [pre-SIP for using directives](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700/15)) -Scala CLI is an ambitious project and may seem hard to maintain in the long-run. // TODO +Scala CLI is an ambitious project and may seem hard to maintain in the long-run. ### Open questions From 572e002997a077a8b1593dcce51e2198627716b3 Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Tue, 13 Sep 2022 12:13:28 +0200 Subject: [PATCH 6/9] Add section why not mill or sbt to repalce scala command --- content/scala-cli.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/content/scala-cli.md b/content/scala-cli.md index 51765025..7e9c3091 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -31,6 +31,17 @@ We observe that the current state of tooling in Scala is limiting creativity, wi With this proposal our main goal is to turn Scala into a language with "batteries included" that will also respect the community-first aspect of our ecosystem. +### Why decided to work on Scala CLI rather then improve existing tools like sbt or Mill? + +Firstly, Scala CLI is in no way an actual replacement for SBT or Mill - nor was it ever meant to be. We do not call it a build tool, even though it does share some similarities with build tools. It doesn't aim at supporting multi-module +projects, nor to be extended via a task system. The main advantages of SBT and Mill: multi-module support and plugin ecosystem in the use cases for Scala CLI and scala command can often be disadvantages as it affects performance: configuration needs to be compiled, plugins resolved etc. + +Mill and SBT uses turing complete configuration for build so the complexity of build scripts in theory is unlimited. Scala CLI is configuration-only and that limits the complexity what put a hard cap how complex Scala CLI builds can be. + + In our opinion, `scala` command should be first and foremost a command line tool. Requirements for a certain project structure or presence configuration files limit SBT and Mill usability certain use cases related to command line. + +One of the main requirements for the new `scala` commands was speed, flexibility and focus on command-line use cases. Initially, we were considering improving SBT or Mill as well as building Scala CLI on top one. We have quickly realized that getting Mill or SBT to reply within Milliseconds (for cases where no hard work like compilation is require) would be pretty much out of reach. Mill and SBT's codebases are too big to compile them to native image using GraalVM, not to mention problems with dynamic loading and reflection. Adding flexibility when in comes to input sources (e.g. support for Gists) and making the tool that can accept most of the configuration using simple command-line parameters would involve writhing a lot of glue code. That is why we decided to build the tool from scratch based on existing components like coursier, bloop or scalafmt. + ## Proposed solution We propose to gradually replace the current `scala`, `scalac` and `scaladoc` commands by single `scala` command that under the hood will be `scala-cli`. We could also add wrapper scripts for `scalac` and `scaladoc` that will mimic the functionality that will use `scala-cli` under the hood. From f815599b4cca6d3493ccbcf0699b654ffe84af68 Mon Sep 17 00:00:00 2001 From: Anatolii Kmetiuk Date: Fri, 16 Sep 2022 10:28:59 +0200 Subject: [PATCH 7/9] Update SIP to be in a good shape for merge --- content/scala-cli.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/scala-cli.md b/content/scala-cli.md index 7e9c3091..ecb04f1e 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -1,9 +1,9 @@ --- layout: sip -permalink: /sips/:title.html +permalink: /sips/scala-cli.html stage: implementation status: waiting-for-implementation -title: SIP-NN Scala CLI as default Scala command +title: SIP-46 - Scala CLI as default Scala command --- **By: Krzysztof Romanowski and Scala CLI team** From b7b0e742de5277dba0825a54c3b235a007b8888f Mon Sep 17 00:00:00 2001 From: Julien Richard-Foy Date: Tue, 27 Sep 2022 11:32:18 +0200 Subject: [PATCH 8/9] Update scala-cli.md --- content/scala-cli.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/scala-cli.md b/content/scala-cli.md index ecb04f1e..64527da6 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -1,6 +1,6 @@ --- layout: sip -permalink: /sips/scala-cli.html +permalink: /sips/:title.html stage: implementation status: waiting-for-implementation title: SIP-46 - Scala CLI as default Scala command From 4a53b9cd851987fedd55e4c0f9ecc95e4fbfed5b Mon Sep 17 00:00:00 2001 From: Krzysztof Romanowski Date: Fri, 21 Oct 2022 11:36:20 +0200 Subject: [PATCH 9/9] Runner specification inlined --- content/scala-cli.md | 612 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 577 insertions(+), 35 deletions(-) diff --git a/content/scala-cli.md b/content/scala-cli.md index 64527da6..2ee61c9c 100644 --- a/content/scala-cli.md +++ b/content/scala-cli.md @@ -38,7 +38,7 @@ projects, nor to be extended via a task system. The main advantages of SBT and M Mill and SBT uses turing complete configuration for build so the complexity of build scripts in theory is unlimited. Scala CLI is configuration-only and that limits the complexity what put a hard cap how complex Scala CLI builds can be. - In our opinion, `scala` command should be first and foremost a command line tool. Requirements for a certain project structure or presence configuration files limit SBT and Mill usability certain use cases related to command line. +`scala` command should be first and foremost a command line tool. Requirements for a certain project structure or presence configuration files limit SBT and Mill usability certain use cases related to command line. One of the main requirements for the new `scala` commands was speed, flexibility and focus on command-line use cases. Initially, we were considering improving SBT or Mill as well as building Scala CLI on top one. We have quickly realized that getting Mill or SBT to reply within Milliseconds (for cases where no hard work like compilation is require) would be pretty much out of reach. Mill and SBT's codebases are too big to compile them to native image using GraalVM, not to mention problems with dynamic loading and reflection. Adding flexibility when in comes to input sources (e.g. support for Gists) and making the tool that can accept most of the configuration using simple command-line parameters would involve writhing a lot of glue code. That is why we decided to build the tool from scratch based on existing components like coursier, bloop or scalafmt. @@ -48,13 +48,13 @@ We propose to gradually replace the current `scala`, `scalac` and `scaladoc` com The complete set of `scala-cli` features can be found in [its documentation](https://scala-cli.virtuslab.org/docs/overview). -Scala CLI brings many features like testing, packaging, exporting to sbt / Mill or upcoming support for publishing micro-libraries. Initially, we may want to limit the set of features available in the `scala` command by default. Scala CLI is a relatively new project and we should battle-proof some of its features before we commit to support them as part of the offical `scala` command. +Scala CLI brings many features like testing, packaging, exporting to sbt / Mill or upcoming support for publishing micro-libraries. Initially, we propose to limit the set of features available in the `scala` command by default. Scala CLI is a relatively new project and we should battle-proof some of its features before we commit to support them as part of the official `scala` command. -Scala CLI offers [multiple native ways to be installed](https://scala-cli.virtuslab.org/install#advanced-installation) so most users should find a suitable method. We would like these packages to become the default `scala` package in most repositories, often replacing existing `scala` packages. +Scala CLI offers [multiple native ways to be installed](https://scala-cli.virtuslab.org/install#advanced-installation) so most users should find a suitable method. We propose that these packages to become the default `scala` package in most repositories, often replacing existing `scala` packages but the fact how new `scala` command would be installed is not intended to be a part of this SIP. ### High-level overview -Let us show a few examples where adopting Scala CLI as `scala` command would be a significant improvement ofer current scripts. For this, we have assumed a minial set of features. Each additional Scala CLI feature included, such as `package`, would add more and more use cases. +Let us show a few examples where adopting Scala CLI as `scala` command would be a significant improvement over current scripts. For this, we have assumed a minimal set of features (described as MUST have and SHOULD have). Each additional Scala CLI feature included in the future, such as `package`, would add more and more use cases. **Using REPL with a 3rd-party dependency** @@ -66,7 +66,7 @@ With Scala CLI, starting a REPL with a given dependency is as simple as running: scala-cli repl --dep com.lihaoyi::os-lib:0.7.8 ``` -Compared to Ammonite, default Scala REPLs provided by Scala 2 and 3 - that Scala CLI uses by default - are somewhat limited. However, Scala CLI also offers to start Ammonite instead of the default Scala REPL, by passing `--ammonite` (or `--amm`) option to `scala-cli repl`. +Compared to Ammonite, default Scala REPLs provided by Scala 2 and 3 - that Scala CLI uses by default - are somewhat limited. However, Scala CLI also offers to start Ammonite instead of the default Scala REPL, by passing `--ammonite` (or `--amm`) option to `scala-cli repl` but we do not propose to include `--ammonite` to the `scala` command not to commit to its maintenance. Additionally, `scala-cli repl` can also put code from given files / directories / snippets on the class path by just providing their locations as arguments. Running `scala-cli repl foo.scala baz` will compile code from `foo.scala` and the `baz` directory, and put their classes on the REPL class path (including their dependencies, scalac options etc. defined within those files). @@ -74,9 +74,9 @@ Compilation (and running scaladoc as well) benefit in a similar way from the abi ** Providing reproductions of bugs ** -Currently, when reporting a bug in the compiler (or any other Scala-related) repository, users need to provide depencencies, compiler options etc. in comments, create a repository containing a projet with a Mill / sbt configuration to reproduce. In general, testing the reporoduction or working on further minization is not straightworwad. +Currently, when reporting a bug in the compiler (or any other Scala-related) repository, users need to provide dependencies, compiler options etc. in comments, create a repository containing a projet with a Mill / sbt configuration to reproduce. In general, testing the reproduction or working on further minimization is not straightforward. -"Using directives", provided by Scala CLI give the ablity to include the whole configuration in single file, for example: +"Using directives", provided by Scala CLI give the ability to include the whole configuration in single file, for example: ```scala //> using platform "native" @@ -92,49 +92,55 @@ Moreover, Scala CLI provides the ability to run GitHub gists (including multi-fi ** Watch mode ** -When working on a piece of code, it is often useful to have it compiled/run everytime the file is changed, and build tools offer a watch mode for that. This is how most people are using watch mode through a build tool. Scala CLI offers a watch mode for most of its commands (by using `--watch` / `-w` flags). +When working on a piece of code, it is often useful to have it compiled/run every time the file is changed, and build tools offer a watch mode for that. This is how most people are using watch mode through a build tool. Scala CLI offers a watch mode for most of its commands (by using `--watch` / `-w` flags). ### Specification - In order to be able to expand the functionality of Scala CLI and yet use the same core to power the `scala` command, we propose to include both `scala` and `scala-cli` commands in the installation package. Scala CLI already has a feature to limit accessible sub-commands based the binary name (all sub-commands in `scala-cli`, and a curated list in `scala`). Later, we can include more and more features from `scala-cli` into `scala`. + In order to be able to expand the functionality of Scala CLI and yet use the same core to power the `scala` command, we propose to include both `scala` and `scala-cli` commands in the installation package. Scala CLI already has a feature to limit accessible sub-commands based the binary name (all sub-commands in `scala-cli`, and a curated list in `scala`). On later date, more features from `scala-cli` could be included into `scala` command by additional SIPs or similar processes. -The sub-commands necessary to include in the `scala` to match the functionalities of current commands: - - `compile` - - `run` - - `repl` - - `doc` +These sub-commands MUST be included in the the specification of the new `scala` command: -On top of that, we think that the following user-facing sub-commands should also be included: - - `clean` - to rebuild project from start without any cached steps - - `setup-ide` - to control the setup of BSP for IDE integration - - `doctor` - to analyze if everything is installed properly + - compile: Compile Scala code + - doc: Generate Scaladoc documentation + - repl: Fire-up a Scala REPL + - run: Compile and run Scala code. + - shebang: Like `run`, but more handy from shebang scripts -We also suggest to include additional sub-commands by default: - - `fmt` - to format the code using scalafmt - - `test` - to run tests included in the code - - `package` - to package the code into various package formats: JAR, "fat" be, executable be or even native application or docker image - - `shebang` - a command useful for scripting, designed to be included in the "shebang" section of scripts - - `export` - transform current project to sbt / Mill project using the same settings as provided. Useful to evolve prototypes into bigger projects +These sub-commands SHOULD be included in the the specification of the new `scala` command: -Each of these commands expands what the current scripts offer and can be discussed separately. We can even open a dedicated SIP for each of them. + - fmt: Format Scala code + - test: Compile and test Scala code + - version: Print `scala-cli` version -Beyond that, `scala-cli` offers multiple sub-commands needed to manage itself (e.g. `update`) or its components (e.g. the Bloop server). In most cases, -these are not user-facing, but still handy. We can elaborate in more detail on what those commands are and why we need them, if necessary. -Scala CLI can also be configured with ["using directives"](https://scala-cli.virtuslab.org/docs/guides/using-directives) - a comment-based configuration syntax that should be placed at the top of Scala files. This allows for self-containing examples within one file since most of the configuration can be provided either from the command line or via using directives (command line has precedence). This is a game changer for use cases like scripting, reproduction, or within the academic scope. +The subcommand that MAY be included in the specification of the new `scala` command. Those sub-commands are specific to implementation of Scala CLI and provide important, user-facing features like integration with IDE or cleaning up incremental compilation state: -We have described the motivation, syntax and implementation basis in the [dedicated pre-SIP](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700). Currently, we recommend to write using directives as comments, so making them part of the language specification is not necessary at this stage. Moreover, the new `scala` command could ignore using directives in the initial version, however we strongly suggest to include comment-based using directives from the start. + - about: Print details about this application + - bsp: Start BSP server + - clean: Clean the workspace + - doctor: Print details about this application + - help: Print help message + - install-completions: Installs completions into your shell + - install-home: Install `scala-cli` in a sub-directory of the home directory + - setup-ide: Generate a BSP file that you can import into your IDE + - uninstall: Uninstall scala-cli - only works when installed by the installation script + - uninstall-completions: Uninstalls completions from your shell + - update: Update scala-cli - only works when installed by the installation script -#### Commands, options and directives +Last section of this proposal is the list of options that each sub-command MUST HAVE and SHOULD HAVE for each sub-commands that MUST or SHOULD be included in the specification of the new `scala` command. The options that are specific to the implementation (MAY have) as well as options for implementation specific sub-commands (MAY have) are included in [full specification](https://romanowski.github.io/scala-cli/docs/reference/scala-command/runner-specification). + -We have generated a complete lists of supported [commands](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/commands.md), [CLI options](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/cli-options.md) and [directives](https://github.com/romanowski/scala-cli/blob/6b7e9614078070ce8ff6cbcb0635b78901228749/website/src/pages/scala-command/directives.md) that we propose for new `scala` command. Lists contain many options and commands that are marked as `experimental` and/or `internal` and as such, we could change them at any point. -The lists was generated automatically and starting from next release of Scala CLI such documentation will be included on our website. Using that lists, it will be easy to track and spot any changes in exposed APIs. +Scala CLI can also be configured with ["using directives"](https://scala-cli.virtuslab.org/docs/guides/using-directives) - a comment-based configuration syntax that should be placed at the top of Scala files. This allows for self-containing examples within one file since most of the configuration can be provided either from the command line or via using directives (command line has precedence). This is a game changer for use cases like scripting, reproduction, or within the academic scope. + +We have described the motivation, syntax and implementation basis in the [dedicated pre-SIP](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700). Currently, we recommend to write using directives as comments, so making them part of the language specification is not necessary at this stage. Moreover, the new `scala` command could ignore using directives in the initial version, however we strongly suggest to include comment-based using directives from the start. + +Last section of this proposal contains a sumamry of Using Directives syntax as well as list of directives that MUST and SHOULD be supported. ### Compatibility -Adopting Scala CLI as the new `scala` command, as is, will change some of the behaviour of today's scripts. Some examples: +Adopting Scala CLI as the new `scala` command, as is, will change some of the behavior of today's scripts. Some examples: - Scala CLI recognizes tests based on the extension used (`*.test.scala`) so running `scala compile a.scala a.test.scala` will only compile `a.scala` - Scala CLI has its own versioning scheme, that is not related to the Scala compiler. Default version used may dynamically change when new Scala version is released. Similarly to Scala 3, we intend for Scala CLI to be backward compatible and this should help mitigate this risk. @@ -143,14 +149,14 @@ Adopting Scala CLI as the new `scala` command, as is, will change some of the be ### Other concerns -Scala CLI brings [using directives](https://scala-cli.virtuslab.org/docs/guides/using-directives) and [conventions to mark the test files](https://scala-cli.virtuslab.org/docs/commands/test#test-sources). We are not sure if both can be accepted as a part of this SIP or we should have seperate SIPs for both (we have opened a [pre-SIP for using directives](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700/15)) +Scala CLI brings [using directives](https://scala-cli.virtuslab.org/docs/guides/using-directives) and [conventions to mark the test files](https://scala-cli.virtuslab.org/docs/commands/test#test-sources). We suggest to accept both accepted as a part of this SIP but we are ready to open dedicated SIPs for both (we have opened a [pre-SIP for using directives](https://contributors.scala-lang.org/t/pre-sip-using-directives/5700/15)) Scala CLI is an ambitious project and may seem hard to maintain in the long-run. ### Open questions -The main open question for this proposal is wich commands/features should be included by default in the `scala` command. Another aspect is the release cadence: should the new `scala` command follow the current release cadence for Scala CLI (every 2 weeks) or stick to Scala one (every 6 weeks)? +The release cadence: should the new `scala` command follow the current release cadence for Scala CLI (every 2 weeks) or stick to Scala one (every 6 weeks)? ## Alternatives @@ -165,3 +171,539 @@ Scala CLI has many alternatives. The most obvious ones are sbt, Mill, or other b ## FAQ This section will probably initially be empty. As discussions on the proposal progress, it is likely that some questions will come repeatedly. They should be listed here, with appropriate answers. + +# Scala Runner Specification + +This section describes proposed Scala Runner specification and was generated from Scala CLI documentation. It contains `MUST` have and `SHOULD` have commands (each with complete list of MUST have and SHOULD have options) followed by a list of using directives. + +## Scalac options + +Scala Runner MUST support following options from Scala Compiler directly: + - `-encoding` + - `-release` + - `-color` + - `-nowarn` + - `-feature` + - `-deprecation` + + + Additionally, all options that start with: +- `-g` +- `-language` +- `-opt` +- `-P` +- `-target` +- `-V` +- `-W` +- `-X` +- `-Y` + +SHOULD be treated as be Scala compiler options and be propagated to Scala Compiler. This applies to all commands that uses compiler directly or indirectly. + +# MUST have commands + +## `compile` command + +Compile Scala code + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--watch`, `-w`: Watch source files for changes +- `--restart`, `--revolver`: Run your application in background and automatically restart if sources have been changed +- `--test`: Compile test scope +--- + +## `doc` command + +Generate Scaladoc documentation + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +- `--output`, `-o`: Set the destination path +- `--force`, `-f`: Overwrite the destination directory, if it exists +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--default-scaladoc-options`, `--default-scaladoc-opts`: Control if scala CLI should use default options for scaladoc, true by default. Use `--default-scaladoc-opts:false` to not include default options. +--- + +## `repl` command + +Aliases: `console` + +Fire-up a Scala REPL + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +- `--java-opt`, `-J`: Set Java options, such as `-Xmx1g` +- `--java-prop`: Set Java properties + +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--watch`, `-w`: Watch source files for changes +- `--restart`, `--revolver`: Run your application in background and automatically restart if sources have been changed + +--- + +## `run` command + +Compile and run Scala code. + +To pass arguments to the application, just add them after `--`, like: + +```sh +scala-cli MyApp.scala -- first-arg second-arg +``` + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +- `--java-opt`, `-J`: Set Java options, such as `-Xmx1g` +- `--java-prop`: Set Java properties +- `--main-class`, `-M`: Specify which main class to run + +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--watch`, `-w`: Watch source files for changes +- `--restart`, `--revolver`: Run your application in background and automatically restart if sources have been changed +- `--main-class-ls`, `--main-class-list`, `--list-main-class`, `--list-main-classes`: List main classes available in the current context +- `--command`: Print the command that would have been run (one argument per line), rather than running it + +--- + +## `shebang` command + +Like `run`, but more handy from shebang scripts + +This command is equivalent to `run`, but it changes the way +`scala-cli` parses its command-line arguments in order to be compatible +with shebang scripts. + +Normally, inputs and scala-cli options can be mixed. Program have to be specified after `--` + +```sh +scala-cli [command] [scala_cli_options | input]... -- [program_arguments]... +``` + +Contrary, for shebang command, only a single input file can be set, all scala-cli options +have to be set before the input file, and program arguments after the input file +```sh +scala-cli shebang [scala_cli_options]... input [program_arguments]... +``` + +Using this, it is possible to conveniently set up Unix shebang scripts. For example: +```sh +#!/usr/bin/env -S scala-cli shebang --scala-version 2.13 +println("Hello, world) +``` + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +- `--java-opt`, `-J`: Set Java options, such as `-Xmx1g` +- `--java-prop`: Set Java properties +- `--main-class`, `-M`: Specify which main class to run + +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--watch`, `-w`: Watch source files for changes +- `--restart`, `--revolver`: Run your application in background and automatically restart if sources have been changed +- `--main-class-ls`, `--main-class-list`, `--list-main-class`, `--list-main-classes`: List main classes available in the current context +- `--command`: Print the command that would have been run (one argument per line), rather than running it + +--- + +# SHOULD have commands + +## `fmt` command + +Aliases: `format`, `scalafmt` + +Format Scala code + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--check`: Check if sources are well formatted + +--- + +## `test` command + +Compile and test Scala code + +### MUST have options + +- `--dependency`, `--dep`: Add dependencies +- `--compiler-plugin`, `-P`, `--plugin`: Add compiler plugin dependencies +- `--scala-version`, `--scala`, `-S`: Set the Scala version +- `--scala-binary-version`, `--scala-binary`, `--scala-bin`, `-B`: Set the Scala binary version +- `--extra-jars`, `--jar`, `--jars`, `--extra-jar`, `--class`, `--extra-class`, `--classes`, `--extra-classes`, `-classpath`, `-cp`, `--classpath`, `--class-path`, `--extra-class-path`: Add extra JARs and compiled classes to the class path +- `--resource-dirs`, `--resource-dir`: Add a resource directory +- `--compilation-output`, `--output-directory`, `-d`, `--destination`, `--compile-output`, `--compile-out`: Copy compilation results to output directory using either relative or absolute path +- `--java-opt`, `-J`: Set Java options, such as `-Xmx1g` +- `--java-prop`: Set Java properties +### SHOULD have options + +- `--js`: Enable Scala.js. To show more options for Scala.js pass `--help-js` +- `--js-version`: The Scala.js version +- `--js-mode`: The Scala.js mode, either `dev` or `release` +- `--js-module-kind`: The Scala.js module kind: commonjs/common, esmodule/es, nomodule/none +- `--js-check-ir`: +- `--js-emit-source-maps`: Emit source maps +- `--js-source-maps-path`: Set the destination path of source maps +- `--js-dom`: Enable jsdom +- `--js-header`: A header that will be added at the top of generated .js files +- `--js-es-version`: The Scala.js ECMA Script version: es5_1, es2015, es2016, es2017, es2018, es2019, es2020, es2021 +- `--native`: Enable Scala Native. To show more options for Scala Native pass `--help-native` +- `--native-version`: Set the Scala Native version +- `--native-mode`: Set Scala Native compilation mode +- `--native-gc`: Set the Scala Native garbage collector +- `--native-linking`: Extra options passed to `clang` verbatim during linking +- `--native-compile`: List of compile options +- `--embed-resources`: Embed resources into the Scala Native binary (can be read with the Java resources API) +- `--repository`, `--repo`, `-r`: Add repositories +- `--debug`: Turn debugging on +- `--debug-port`: Debug port (5005 by default) +- `--debug-mode`: Debug mode (attach by default) +- `--java-home`: Set the Java home directory +- `--jvm`, `-j`: Use a specific JVM, such as `14`, `adopt:11`, or `graalvm:21`, or `system` +- `--javac-plugin`: Javac plugin dependencies or files +- `--javac-option`, `--javac-opt`: Javac options +- `--script-snippet`: Allows to execute a passed string as a Scala script +- `--execute-script`, `--execute-scala-script`, `--execute-sc`, `-e`: A synonym to --script-snippet, which defaults the sub-command to `run` when no sub-command is passed explicitly +- `--scala-snippet`: Allows to execute a passed string as Scala code +- `--extra-compile-only-jars`, `--compile-only-jar`, `--compile-only-jars`, `--extra-compile-only-jar`: Add extra JARs in the compilaion class path. Mainly using to run code in managed environments like Spark not to include certain depenencies on runtime ClassPath. +- `--extra-source-jars`, `--source-jar`, `--source-jars`, `--extra-source-jar`: Add extra source JARs +- `--platform`: Specify platform +- `--semantic-db`: Generate SemanticDBs +- `--watch`, `-w`: Watch source files for changes +- `--restart`, `--revolver`: Run your application in background and automatically restart if sources have been changed +- `--test-framework`: Name of the test framework's runner class to use while running tests +- `--require-tests`: Fail if no test suites were run +--- + +# Using Directives + + +As a part of this SIP we propose to introduce Using Directives, a special comments containing configuration. Withing Scala CLI and by extension `scala` command, the command line arguments takes precedence over using directives. + +Using directives can be place on only top of the file (above imports, package definition etx.) and can be proceed only by plain comments (e.g. to comment out an using directive) + +Comments containing directives needs to start by `>`, for example: + +``` +//> using Scala 3 +/*> using + Scala 3 + option "-Xfatal-warning" +*/ +``` + +We propose following sytax for Using Directives (within special comments described above): + +``` +UsingDirective ::= "using" (Setting | Settings) +Settings ::= "{" Setting { ";" Setting } [";"] "}" +Setting ::= Ident [Values | Settings] +Ident ::= ScalaIdent { “.” ScalaIdent } +Values ::= Value { "," Value} [","] +Value ::= stringLiteral | ["-"] numericLiteral | true | false +``` + +Where: + +- A Settings block is similar to the standard Scala code block. Braces and indentation syntax is allowed. e.g.: + + +``` +someSettings { setting1 value 1; setting2; } + +// or + +someSettings { + setting1 + setting2 +} + +// or + +someSettings: + setting1 + setting2 +``` + +- Ident is the standard Scala identifier or list of identifiers separated by dots: + +``` +foo + +foo.bar + +`foo-bar`.bazz +``` + +- String literals and numeric literals are similar to Scala. +- No value after the identifier is treated as true value: `using scalaSettings.fatalWarnings` +- Specifying a setting with the same path more than once and specifying the same setting with a list of values are equivalent +- A path created using dot-separated idents is semantically equivalent to a path created by nesting idents: +`foo.bar` is equivalent to `foo { bar }` + + +The list of proposed directives split into MUST have and SHOULD have groups: + +## MUST have directives: + + - `option`, `options`: Add Scala compiler options + - `plugin`, `plugins`: Adds compiler plugins + - `lib`, `libs`: Add dependencies + - `javaOpt`, `javaOptions`, `java-opt`, `java-options`: Add Java options which will be passed when running an application. + - `javaProp`: Add Java properties + - `main-class`, `mainClass`: Specify default main class + - `scala`: Set the default Scala version +## SHOULD have directives: + + - `jar`, `jars`: Manually add JAR(s) to the class path + - `file`, `files`: Manually add sources to the Scala CLI project + - `java-home`, `javaHome`: Sets Java home used to run your application or tests + - `javacOpt`, `javacOptions`, `javac-opt`, `javac-options`: Add Javac options which will be passed when compiling sources. + - `platform`, `platforms`: Set the default platform to Scala.js or Scala Native + - `repository`, `repositories`: Add a repository for dependency resolution + - `resourceDir`, `resourceDirs`: Manually add a resource directory to the class path + - `native-gc`, `native-mode`, `native-version`, `native-compile`, `native-linking`, `native-clang`, `native-clang-pp`, `native-no-embed`, `nativeGc`, `nativeMode`, `nativeVersion`, `nativeCompile`, `nativeLinking`, `nativeClang`, `nativeClangPP`, `nativeEmbedResources`: Add Scala Native options + - `jsVersion`, `jsMode`, `jsModuleKind`, `jsCheckIr`, `jsEmitSourceMaps`, `jsSmallModuleForPackage`, `jsDom`, `jsHeader`, `jsAllowBigIntsForLongs`, `jsAvoidClasses`, `jsAvoidLetsAndConsts`, `jsModuleSplitStyleStr`, `jsEsVersionStr`: Add Scala.js options + - `test-framework`, `testFramework`: Set the test framework