How to implement a platform-agnostic gpio/i2c/spi wasi api?

[rahul-thakoor]

Hello!

I am very new to the wasm/wasi ecosystem, but I'm interested in learning some more and help with such an implementation if possible. I saw there is a related closed issue #251

Context:
I have a similar use case where I could like to be able to access GPIO pins or even i2c and SPI based devices from a wasm module. I think using the existing wasi filesystem api I can achieve something like this to some extent. E.g I can implement the linux sysfs-gpio interface partially but then the wasm module is not portable per se - it would only run on linux host for instance.

I am trying to understand how to make such a platform-agnostic API such that the same module works on Linux, Windows, macOS, or even microcontroller hosts as long as there is a runtime that implements the hardware abstraction API.

It feels like there is indeed some overlap with the rust-embedded embedded-hal library and the API could be inspired by that.

Not sure if this is the right place for such a discussion. Feel free to redirect accordingly 😃

Thanks 👍

[sunfishcode]

Something like embedded-hal indeed sounds like a good place to start. In general, WASI prefers to use typed APIs rather than filesystem-oriented abstractions, in part because of the portability issues you mention.

The current official way to write WASI APIs is to write an s-expression-based witx interface, and there are a few tools for using that to generate bindings for various languages. That said, these tools aren't very mature, and at this point are somewhat out of date with respect to interface types' evolution. I suggest taking a look at witx-bindgen, which is a new tool still in development, but it learns several lessons from earlier witx work, and incorporates new ideas like the canonical ABI. Please feel free to ask questions!

[rahul-thakoor]

Thanks for the clarification @sunfishcode

So if I understand correctly, say I were to implement the embedded-hal traits for wasi, I would

1. Need to describe the API in .witx format
2. Use witx-bindgen to generate the bindings
3. Implement the bindings for the runtime I want to use

Is that correct?

I'm not 100% sure how to implement something like gpio access using typed APIs and also make that cross-platform. Is it a case of implementing the traits generated separately for each target platform? is there a reference gpio or other library that works for both unix and windows which can be used as a reference?

Thanks

[sunfishcode]

Yes, that sequence sounds correct. As I mentioned above, the tools are still in development, but that seems like the best path at this time, and as it matures it'll get better.

The nature of WASI is that we'll typically want to use the same interface for all platforms. Can you say more about what makes gpio difficult do in a cross-platform way?

[rahul-thakoor]

Can you say more about what makes gpio difficult do in a cross-platform way?

I'm not sure, to be honest. But I'm thinking for implementing the traits for each platform, we will have to use platform specific apis which do provide different interfaces imo.

The nature of WASI is that we'll typically want to use the same interface for all platforms.

Is there an example of perhaps something simple that uses typed APIs to blink a system led for instance? is this possible by using existing wasi api only?

[rahul-thakoor]

Was trying to look for some more info on canonical ABI? is this https://github.com/WebAssembly/interface-types/blob/40f157ad429772c2b6a8b66ce7b4df01e83ae76d/proposals/interface-types/CanonicalABI.md the most up to date one please?

[SamuraiCrow]

Would a volatile address space be the best way to implement a sandboxed memory-mapped I/O be the best way to implement simple devices to a static driver? I see there's a proposal for multiple address spaces on https://github.com/WebAssembly/multi-memory which might be a means to the end we are seeking.

[emielvanseveren]

What steps should I take to get things started around this api? Should I plan something in the biweekly wasi meetings or create an initial wit impl first?

[shaggygi]

Might want to keep an eye on dotnet\iot as they will most likely add gpio controller and i2c devices for this. I'd say it would be a ways off until the core wasi stuff matures.

[emielvanseveren]

I actually want to help implement the interface in WASI. Not use its implementation.

[linclark]

@emielvanseveren you can find more information on how to champion a proposal in the Contributing guide

[joakimeriksson]

Anyone started to work on this yet? (we are looking into support WASM on IoT-devices). I guess one question is on what level do we want the API's to be at? GPIO/SPI means that device drivers will be in WASM also - but if we go for sensors the drivers will be "below" / provided by host. On the other hand - we could have both to provide WASI-defined APIs from within a WASM instance...

[emielvanseveren]

@joakimeriksson I have planned an agenda item on the 9th of March to approve this for phase 1. That is a good question and is something that needs to be discussed further. My initial thought would be to support both although I'm not sure if moving the drivers to the host fits with the platform agnostic goal of WASI.

[SamuraiCrow]

My initial thought would be to support both although I'm not sure if moving the drivers to the host fits with the platform agnostic goal of WASI.

I personally think that the WASI protocol is best left to application development because driver development would require a totally different type of sandboxing than applications.

Also, creating drivers requires volitile memory accesses to make MMIO while an application compatibility layer never should have volitile memory access. I've given it considerable thought and cross-architecture drivers need a different protocol than WASI.

[emielvanseveren]

You mentioned in an earlier reply that MMIO might be achievable with multi-memory. Any thoughts regarding that aspect?

[SamuraiCrow]

@emielvanseveren The issue there, WebAssembly/multi-memory#9 , is closed but discussed in a different proposal: https://github.com/WebAssembly/memory-control . I commented in the discussion that spawned the propsal as well: WebAssembly/design#1439

[sunfishcode]

Another option for MMIO is to have host functions which do the actual loads and stores. This has several nice properties: it abstracts away the need for the addresses to live inside the linear memory address range, it's virtualizable so you can write a hardware emulator just by implementing the functions rather than having to hook into the Wasm engine to intercept loads and stores, it doesn't require us adding I/O semantics to Wasm linear memory, and the host code can be in full control of the sizes and alignments of the accesses to the MMIO memory.

[joakimeriksson]

@sunfishcode - I would go for this type of abstraction - it would give a more high-level interface to the applications also and still enable "higher-level" drivers for some types of hardware (if done right). We are considering adding WASM and WASI in an IoT-OS and are looking into how the abstractions should be in this context.

[woodsmc]

To help guide us in making decisions on the level of abstraction I think considering the min footprint of an embedded device and some specific use cases we want to support first will help. Perhaps also sharing some of this thinking with the WAMR TSC would be a great way to extend the level of feedback.

The approach @sunfishcode suggests, is what I was mentioning on the Meeting on March 9th. You see the same approach in Microkenel OS architectures. A separation of logic, and physical device drivers, and then these also wrapped and then exposed as services (known interfaces) to the application code (wasm). So something like this... (I'm making this up as an illustration for a disk and file system)...

• Physical layer: Software handling Interrupt handling, DMA systems, GPIO's etc. (Native / host system)
• Logical Layer: Software implementing Block Storage Drive Controller (given the physical controller for any number of drives can provide block storage) (Native / host system)
• Interface Layer: File System (fopen / fclose etc. behind the scenes it's implementing a exFAT format) (Native / host system)
  *Application Layer: WASM.

This works, but has an overhead which will impose minimum requirements on the platforms that can support WASM. The stratification above is often rejected by RTOS implementations who provide a bare-metal fopen function so that they can support as many smaller embedded systems as possible.

Providing a definition of what we mean by an embedded system will help us determine the correct layer of abstraction we'd like to provide. Some of this can be determined by the existing minimum system requirements that, say WAMR has and some of the use cases that the WAMR TSC may want to support.

[yamt]

is this about something at the same layer as https://docs.oracle.com/javame/8.2/api/dio/api/index.html ?

[joakimeriksson]

Is there any activities around this? We are putting some efforts into experimenting with WASM in Contiki-NG - for small/resource constrained IoT devices. It would be great having a discussion on this topic and do a reference implementation - we would be happy to discuss, evaluate and test!

[sunfishcode]

I put together a rough sketch of a gpio/i2c/spi API, and built a prototype:

• An API description, which includes GPIO, I²C, PWM, SPI, and timers.
• A repo with a simple example guest, and a host simulator that can run it.

[Zelzahn]

Greetings everyone, I have submitted a PR to update the I2C proposal (PR). Currently, the WIT is largely based upon the works of @sunfishcode.