Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm_of_ocaml support #11093

Merged
merged 26 commits into from
Nov 5, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,94 @@ jobs:
# We disable the Dune cache when running the tests
DUNE_CACHE: disabled

wasm:
name: Wasm_of_ocaml
runs-on: ubuntu-latest
steps:
- name: Install Node
uses: actions/setup-node@v4
with:
node-version: latest

- name: Restore Cached Binaryen
id: cache-binaryen
uses: actions/cache/restore@v4
with:
path: binaryen
key: ${{ runner.os }}-binaryen-version_119

- name: Checkout Binaryen
if: steps.cache-binaryen.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
repository: WebAssembly/binaryen
path: binaryen
submodules: true
ref: version_119

- name: Install Ninja
if: steps.cache-binaryen.outputs.cache-hit != 'true'
run: sudo apt-get install ninja-build

- name: Build Binaryen
if: steps.cache-binaryen.outputs.cache-hit != 'true'
working-directory: ./binaryen
run: |
cmake -G Ninja .
ninja

- name: Cache Binaryen
if: steps.cache-binaryen.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: binaryen
key: ${{ runner.os }}-binaryen-version_119

- name: Set Binaryen's Path
run: |
echo "$GITHUB_WORKSPACE/binaryen/bin" >> $GITHUB_PATH

- name: Checkout Code
uses: actions/checkout@v4
with:
path: dune

- name: Use OCaml 4.14.x
uses: ocaml/setup-ocaml@v3
with:
ocaml-compiler: 4.14.x
opam-pin: false
opam-depext: false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opam depext is unused in setup-ocaml v3

dune-cache: true

- name: Update Dune
working-directory: ./dune
run: opam pin add -n dune . --with-version 3.17.0

- name: Checkout Wasm_of_ocaml
uses: actions/checkout@v4
with:
repository: ocaml-wasm/wasm_of_ocaml
ref: wasm-dune
path: wasm_of_ocaml

- name: Install Wasm_of_ocaml
working-directory: ./wasm_of_ocaml
run: |
opam pin add -n . --with-version `< VERSION`
opam install wasm_of_ocaml-compiler

- name: Set Git User
run: |
git config --global user.name github-actions[bot]
git config --global user.email github-actions[bot]@users.noreply.github.com
- name: Run Tests
working-directory: ./dune
run: opam exec -- make test-wasm
env:
# We disable the Dune cache when running the tests
DUNE_CACHE: disabled

monorepo_benchmark_test:
name: Build monorepo benchmark docker image
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ test-windows: $(BIN)
test-js: $(BIN)
$(BIN) build @runtest-js

test-wasm: $(BIN)
DUNE_WASM_TEST=enable $(BIN) build @runtest-wasm

test-coq: $(BIN)
DUNE_COQ_TEST=enable $(BIN) build @runtest-coq

Expand Down
20 changes: 16 additions & 4 deletions bin/printenv.ml
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,25 @@ let dump sctx ~dir =
|> Action_builder.of_memo
>>= Dune_rules.Menhir_env.dump
and+ coq_dump = Dune_rules.Coq.Coq_rules.coq_env ~dir >>| Dune_rules.Coq.Coq_flags.dump
and+ jsoo_dump =
and+ jsoo_js_dump =
let module Js_of_ocaml = Dune_rules.Js_of_ocaml in
let* jsoo = Action_builder.of_memo (Dune_rules.Jsoo_rules.jsoo_env ~dir) in
Js_of_ocaml.Flags.dump jsoo.flags
let* jsoo = Action_builder.of_memo (Dune_rules.Jsoo_rules.jsoo_env ~dir ~mode:JS) in
Js_of_ocaml.Flags.dump ~mode:JS jsoo.flags
and+ jsoo_wasm_dump =
let module Js_of_ocaml = Dune_rules.Js_of_ocaml in
let* jsoo = Action_builder.of_memo (Dune_rules.Jsoo_rules.jsoo_env ~dir ~mode:Wasm) in
Js_of_ocaml.Flags.dump ~mode:Wasm jsoo.flags
in
let env =
List.concat [ o_dump; c_dump; link_flags_dump; menhir_dump; coq_dump; jsoo_dump ]
List.concat
[ o_dump
; c_dump
; link_flags_dump
; menhir_dump
; coq_dump
; jsoo_js_dump
; jsoo_wasm_dump
]
in
Super_context.context sctx |> Context.name, env
;;
Expand Down
1 change: 1 addition & 0 deletions doc/changes/11093.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Wasm_of_ocaml support (#11093, @vouillon)
1 change: 1 addition & 0 deletions doc/howto/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ These guides will help you use Dune's features in your project.
../sites
../instrumentation
../jsoo
../wasmoo
../melange
../virtual-libraries
../tests
Expand Down
19 changes: 18 additions & 1 deletion doc/reference/dune/env.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,24 @@ Fields supported in ``<settings>`` are:
or not where ``<mode>`` is either ``no``, ``file`` (to generate sourcemap in a ``.map`` file next the the generated javascript file) or ``inline`` (to inline the sourcemap at the end of the generated JavaScript file).

- ``(js_of_ocaml (runtest_alias <alias-name>))`` specifies the alias under which
:ref:`inline_tests` and tests (:ref:`tests-stanza`) run for the `js` mode.
:ref:`inline_tests` and tests (:ref:`tests-stanza`) run for the ``js`` mode.

- ``(js_of_ocaml (enabled_if <blang expression>))`` specifies whether the ``js`` mode is enabled. It is enabled by default.

- ``(wasm_of_ocaml (flags <flags>)(build_runtime <flags>)(link_flags <flags>))``
specifies ``wasm_of_ocaml`` flags. See :ref:`wasmoo-field` for more details.

- ``(wasm_of_ocaml (compilation_mode <mode>))`` controls whether to use separate
compilation or not where ``<mode>`` is either ``whole_program`` or
``separate``.

- ``(wasm_of_ocaml (sourcemap <mode>))`` controls whether to generate sourcemap
or not where ``<mode>`` is either ``no``, ``file`` (to generate sourcemap in a ``.map`` file next the the generated javascript file) or ``inline`` (to inline the sourcemap at the end of the generated JavaScript file).

- ``(wasm_of_ocaml (runtest_alias <alias-name>))`` specifies the alias under which
:ref:`inline_tests` and tests (:ref:`tests-stanza`) run for the ``wasm`` mode.

- ``(wasm_of_ocaml (enabled_if <blang expression>))`` specifies whether the ``wasm`` mode is enabled. It is enabled by default.

- ``(binaries <binaries>)``, where ``<binaries>`` is a list of entries of the
form ``(<filepath> as <name>)``. ``(<filepath> as <name>)`` makes the binary
Expand Down
40 changes: 36 additions & 4 deletions doc/reference/dune/executable.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,15 @@ executable stanzas is as follows:
There can be additional modules in the current directory; you only need to
specify the entry point. Given an ``executable`` stanza with ``(name <name>)``,
Dune will know how to build ``<name>.exe``. If requested, it will also know how
to build ``<name>.bc`` and ``<name>.bc.js`` (Dune 2.0 and up also need specific
configuration (see the ``modes`` optional field below)).
to build ``<name>.bc``, ``<name>.bc.js`` and ``<name>.bc.wasm.js`` (Dune 2.0
and up also needs specific configuration (see the ``modes`` optional field
below)).

``<name>.exe`` is a native code executable, ``<name>.bc`` is a bytecode
executable which requires ``ocamlrun`` to run, and ``<name>.bc.js`` is a
JavaScript generated using ``js_of_ocaml``.
executable which requires ``ocamlrun`` to run, ``<name>.bc.js`` is a
JavaScript generated using ``js_of_ocaml``, and ``<name>.bc.wasm.js`` is a
Wasm loader script generated using ``wasm_of_ocaml`` (the Wasm modules are included in
directory ``<name>.bc.wasm.assets``).

Please note: in case native compilation is not available, ``<name>.exe`` will be
a custom bytecode executable, in the sense of ``ocamlc -custom``. This means
Expand Down Expand Up @@ -91,6 +94,8 @@ files for executables. See

- ``js_of_ocaml``: See the section about :ref:`jsoo-field`

- ``wasm_of_ocaml``: See the section about :ref:`wasmoo-field`

- ``flags``, ``ocamlc_flags``, and ``ocamlopt_flags``: See
:doc:`/concepts/ocaml-flags`.

Expand Down Expand Up @@ -165,6 +170,7 @@ available.
non-OCaml application.
- ``js`` for producing JavaScript from bytecode executables, see
:doc:`/reference/dune-project/explicit_js_mode`.
- ``wasm`` for producing JavaScript from bytecode executables.
- ``plugin`` for producing a plugin (``.cmxs`` if native or ``.cma`` if
bytecode).

Expand All @@ -186,6 +192,7 @@ Additionally, you can use the following shorthands:
- ``byte`` for ``(byte exe)``
- ``native`` for ``(native exe)``
- ``js`` for ``(byte js)``
- ``wasm`` for ``(byte wasm)``
- ``plugin`` for ``(best plugin)``

For instance, the following ``modes`` fields are all equivalent:
Expand Down Expand Up @@ -216,6 +223,7 @@ The extensions for the various linking modes are chosen as follows:
.. (native/best shared_object) %{ext_dll}
.. c .bc.c
.. js .bc.js
.. wasm .bc.wasm.js
.. (best plugin) %{ext_plugin}
.. (byte plugin) .cma
.. (native plugin) .cmxs
Expand Down Expand Up @@ -264,14 +272,38 @@ options using ``(js_of_ocaml (<js_of_ocaml-options>))``.
- ``(sourcemap <config>)`` where ``<config>>`` is one of ``no``, ``file`` or ``inline``.
This is only available inside ``executable`` stanzas.

- ``(enabled_if <blang expression>)`` to specify whether the ``js`` mode is enabled. It is enabled by default.
This is only available inside ``executable`` stanzas.

``<flags>`` is specified in the :doc:`/reference/ordered-set-language`.
``<blang expression>`` is specified using the :doc:`/reference/boolean-language`,

The default values for ``flags``, ``compilation_mode`` and ``sourcemap`` depend on the selected build profile. The
build profile ``dev`` (the default) will enable inline sourcemap, separate compilation and pretty
JavaScript output.

See :ref:`jsoo` for more information.

.. _wasmoo-field:

wasm_of_ocaml
~~~~~~~~~~~~~

In ``library`` and ``executable`` stanzas, you can specify ``wasm_of_ocaml``
options using ``(wasm_of_ocaml (<wasm_of_ocaml-options>))``.

``<wasm_of_ocaml-options>`` are all optional. They are the same as the ``<js_of_ocaml-options>`` above plus:

- ``(wasm_files (<files-list>))`` to specify ``wasm_of_ocaml``
Wasm runtime files.

For the ``(sourcemap <config>)`` option, source maps are generated when ``<config>>`` is either ``file`` or ``inline``. They are put within the ``.bc.wasm.assets`` directory in both cases.

The default values for ``flags``, ``compilation_mode`` and ``sourcemap`` depend on the selected build profile. The
build profile ``dev`` (the default) will enable sourcemaps, separate compilation and pretty Wasm output.

See :ref:`wasmoo` for more information.

executables
-----------

Expand Down
4 changes: 4 additions & 0 deletions doc/reference/dune/library.rst
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,10 @@ order to declare a multi-directory library, you need to use the

Sets options for JavaScript compilation, see :ref:`jsoo-field`.

.. describe:: (wasm_of_ocaml ...)

Sets options for JavaScript compilation, see :ref:`wasmoo-field`.

.. describe:: (flags ...)

See :doc:`/concepts/ocaml-flags`.
Expand Down
3 changes: 2 additions & 1 deletion doc/tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -258,14 +258,15 @@ field. Available modes are:
- ``best`` for running tests in native mode with fallback to byte code,
if native compilation is not available
- ``js`` for running tests in JavaScript using Node.js
- ``wasm`` for running tests in Wasm using Node.js

For instance:

.. code:: ocaml

(library
(name foo)
(inline_tests (modes byte best js))
(inline_tests (modes byte best js wasm))
(preprocess (pps ppx_expect)))


Expand Down
104 changes: 104 additions & 0 deletions doc/wasmoo.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
.. _wasmoo:

***************************************
Wasm Compilation With Wasm_of_ocaml
***************************************

.. TODO(diataxis)

This is an how-to guide.

Wasm_of_ocaml_ is a compiler from OCaml to WebAssembly (Wasm for
short). The compiler works by translating OCaml bytecode to Wasm code.


Compiling to Wasm
=================

Dune has full support for building wasm_of_ocaml libraries and executables transparently.
There's no need to customise or enable anything to compile OCaml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the documentation mention the external dependencies that are required for the build ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I now explicitly point to the github repository above.

libraries/executables to Wasm.

To build a Wasm executable, just define an executable as you would normally.
Consider this example:

.. code:: console

$ echo 'print_endline "hello from wasm"' > foo.ml

With the following ``dune`` file:

.. code:: dune

(executable (name foo) (modes wasm))

And then request the ``.wasm.js`` target:

.. code:: console

$ dune build ./foo.bc.wasm.js
$ node _build/default/foo.bc.wasm.js
hello from wasm

If you're using the js_of_ocaml syntax extension, you must remember to add the
appropriate PPX in the ``preprocess`` field:

.. code:: dune

(executable
(name foo)
(modes wasm)
(preprocess (pps js_of_ocaml-ppx)))

Selective Compilation
=====================

The ``js`` and ``wasm`` modes can be selectively disabled using the ``(js_of_ocaml (enabled_if ...))`` and ``(wasm_of_ocaml (enabled_if ...))`` options. This allows for instance to generate one, or the other dependings on a profile:

.. code:: dune

(env
(js-only
(wasm_of_ocaml
(enabled_if false)))
(wasm-only
(js_of_ocaml
(enabled_if false))))

To be able to invoke the generated code using the same JavaScript script name in all cases, you can add a rule to copy the Wasm launcher script when the js_of_ocaml compilation is disabled.

.. code:: dune

(rule
(action
(copy foo.bc.wasm.js foo.bc.js))
(enabled_if
(= %{profile} wasm-only)))

Separate Compilation
====================

Dune supports two modes of compilation:

- Direct compilation of a bytecode program to Wasm. This mode allows
wasm_of_ocaml to perform whole-program deadcode elimination and whole-program
inlining.

- Separate compilation, where compilation units are compiled to Wasm
separately and then linked together. This mode is useful during development as
it builds more quickly.

The separate compilation mode will be selected when the build profile
is ``dev``, which is the default. It can also be explicitly specified
in an ``env`` stanza (see :doc:`/reference/dune/env`) or per executable
inside ``(wasm_of_ocaml (compilation_mode ...))`` (see :doc:`/reference/dune/executable`)

Sourcemap
=========

Wasm_of_ocaml can generate sourcemaps for the generated Wasm code.
By default, they are generated when using the ``dev`` build profile and are not generated otherwise.
The behavior can explicitly be specified in an ``env`` stanza (see :doc:`/reference/dune/env`)
or per executable inside ``(wasm_of_ocaml (sourcemap ...))`` (see :doc:`/reference/dune/executable`)

.. _wasm_of_ocaml: https://github.com/ocaml-wasm/wasm_of_ocaml
Loading
Loading