Towards a modular ZMK

[innovaker]

Fellow developers, thank you for your patience. The modular proof-of-concept (PoC) is ready for evaluation.

Purpose

This is a (fully functional) proof of concept. Its purpose is to:

• enable everyone to explore what a modular ZMK would entail.
• act as a point of reference for discussions about modularity.
• be the basis for properly evaluating a modular ZMK, including:
  • technical design
  • end-user experiences
  • developer workflows and experiences
  • maintenance
  • best practices
• be the basis for moving forward if we choose to go down this road (it's a high fidelity prototype).

What's "a modular ZMK"?

• Modules:
  • can contain boards, shields, C (code), Kconfig, dts, soc, or arch.
  • can live anywhere in their own git repository (web or filesystems).
  • can be owned, licensed and/or maintained by anyone (distributed development).
  • are facilitated by Zephyr's west tool.
• We distill the ZMK repository down to core components only.
  • Non-core components (boards/shields,etc.) can be moved into external (optional) modules.
• We supersede zmk-config with a simple user "application".
  • This "application" module/repository is owned by the end-user.
  • It's specific to their own build.
  • It sits in the driver's seat.
    • It describes:
      • Which modules their application/build depends on (including boards, shields or extra features).
      • Any configuration or code specific to their build.
    • It's square one for the build process.
  • It can be as simple or complex as the user desires.
• ZMK effectively becomes a core module/library/SDK.

Motivations

I'm suggesting we consider a modular approach for ZMK because of several pressing issues, including:

• Board and Shields
  • As ZMK grows in popularity we're seeing the number of board and shield PRs increase.
  • We're facing pressure around which contributions we should accept into the core.
  • Any time core contributors spend on boards/shields is time taken from core development, slowing the project down.
  • The more boards and shields we accept, the more bloated ZMK becomes and the more maintenance baggage we take on.
  • The more we take on, the more impractical or unaffordable it will become to test them as part of ZMK's CI.
  • Our current approach is great for popular boards/shields, but does little for the smaller players. We risk artificially limiting discoverability.
  • We maintain pretty high standards for PRs, which isn't everyone's cup of tea.
  • What happens when contributions become abandoned? How do we deprecate them?
• Distributed Extensions
  • Currently there are only these options for adding functionality:
    • Add it to your zmk-config.
    • Create a branch/fork of ZMK.
    • Submit a PR to add it to the core (which is an understandably slow process).
  • These approaches get the job done, but none of them are particularly great for sharing with others.
  • It's not easy for developers and testers who often live on the cutting edge from multiple sources.
• Flexibility
  • Modularity would let us explore and adopt a range of approaches for the future, such as:
    • Hardware developer owned & maintained modules (boards/shields).
    • Author owned & maintained modules.
    • Fully distributed boards/shields (each in their own module).
    • Community maintained modules/repositories.
  • Users could leverage other Zephyr modules such as the Nordic nRF Connect SDK without licensing issues for ZMK itself.
  • Users could have more control over the build.

First steps

• I've built a demo which uses the brand new harmless shield. Thanks to @mcrosson for letting me based it on the tidbit.
• The shield itself lives in its own repository/module.

Local development

i.e. developers with an existing ZMK development environment setup on Windows, macOS, Linux or WSL2.

west init --mr master-v2 -m https://github.com/innovaker/my-zmk-keyboard zmk-workspace
cd zmk-workspace/my-zmk-keyboard
west update
west build

You may need to install v0.11.4 of the Zephyr SDK.

GitHub Actions (cloud)

• Fork https://github.com/innovaker/my-zmk-keyboard
• Enable the workflow (only because we're using a fork for the demo)
  • Go to the Actions tab.
  • Click I understand my workflows, go ahead and enable them.
• Run the workflow
  • Go to the Actions tab.
  • Click Build (on the left).
  • Click Run workflow (on the right) and run it on Branch: master-v2
  • Wait 10 seconds for the Build job to appear.
• Make a cup of tea whilst GitHub actions builds firmware for the harmless shield (< 4 minutes).
• Checkout, edit/commit the files and push.
  • You probably only need to edit the matrix list in .github/workflows/build.yml
• Make another cup of tea whilst it rebuilds.

Docker

If you want to try developing in Docker containers ...

• Install Docker for Desktop
• I recommend turning on the WSL2 backend in the options.

VS Code Remote Containers

Due to current shortcomings in VS Code, VS Code Remote Containers, Codespaces, Docker and the Zephyr build system, it's proving tricky to find an intuitive one-size-fits-all approach for containers. However, their potential benefits make them worthy of further exploration. Here's a guide to the known approaches at time of writing ...

Cloning straight from a repository into a container with VS Code

This approach is my personal preference. It's quick to setup, performant and I don't mind my files being inside volumes. It lets you clone a repository straight into a Docker container/volumes without touching the local system or needing any tools besides VS Code. I also use the WSL2 backend which makes it really fast.

• Open VS Code. Make sure you have the Remote - Containers extension installed.
• If you press F1 and enter: > Remote-Containers: Clone
  it'll give you a cloning option.
  Repository URL: https://github.com/innovaker/my-zmk-keyboard
• You can use either cloning options:
  • Create a unique volume
  • Create a new volume ...
    • Enter the volume name: zmk-modular-v2 (it can be anything you like)
    • Enter the target folder name: zmk-workspace/my-zmk-keyboard
      • note the slash: it's a trick you can use to have multiple ZMK workspaces in a single container.
• After launching the container:
  • west update will run as part of the container setup - it will take a while the first time.
  • When the prompt pops up, click Open Workspace.
    • This will let you see the zmk folder and give you access to the other VS Code workspace benefits.
  • Open a terminal and run: west build

Cloning to a local disk first, then opening it in VS Code, then reopening it in a container

Some people prefer this method as it makes more sense to them. But it's not as performant.

• Locally (in your OS): git clone -b master-v2 https://github.com/innovaker/my-zmk-keyboard
• Open the my-zmk-keyboard folder in VS Code.
• When the prompt pops up, click Reopen in Container.
• west update will run as part of the container setup - it will take a while the first time.
• When the prompt pops up, click Open Workspace.
  • This will let you see the zmk folder and give you access to the other VS Code workspace benefits.
• Open a terminal for app and run: west build

Note, I've configured it to mount zephyr and its dependencies into (shared) Docker volumes so that it doesn't take an eternity to run west update. These volumes are shared across containers and not junked when you delete a container. zmk is currently stored within the container itself (so it will be lost/re-cloned if you rebuild the container) but it too can be mounted as a volume by a savvy user (I've left it commented out).

Codespaces + nested branch

I've not personally tested Codespaces yet but my cursory understanding is that it's built around the mono-repository paradigm and doesn't support untracked flat structure workspaces??? In anticipation of that, I've' been exploring a nested version of zmk-app/my-zmk-keyboard. I'm not keen on it personally but I made it as an experiment.

The branch names are nested-v2 if you want to try it but it's largely untested.

Next steps

• Edit: CMakeLists.txt
• Drop in: your own overlay/keymap?
• Rebuild with: west build -p (pristine build)
• Explore: west.yml
• Try: making your own module?

Going forward

• Whilst exciting, modular development is a complex topic which merits careful consideration and debate.
• It would entail a major paradigm shift to our current approach. We shouldn't get carried away and rush into it.
• We should use this PoC to explore the pros and cons together.
• I encourage you to actively try developing with it, particularly to get a feel for its effects on workflow.
• We need to find the pain points and potential blockers.

[innovaker]

For those wishing to dig deeper:

• The ZMK core changes can be found on the modular-v2 branch, they're self documenting. If you want to know the technical aspects, I suggest studying the commit messages.
• The upstream of my-zmk-keyboard is zmk-app
• The upstream of zmk-tidbit is zmk-module
• The general idea behind the upstreaming is that we clone (not fork!) copies of zmk-app or zmk-module to create new user applications or modules. Infrequent upstream updates can then be merged in.
  • for modules, the developers/maintainers can do this and we could provide a convenience GitHub workflow to notify them of updates, maybe even an automatic PR.
  • for applications, we might achieve this with:
    • scripts for the initial clone (similar to setup.sh)
    • a GitHub workflow to create a PR for them to merge (as above)
    • React based system on the website (using API calls to GitHub)
    • scripts embedded into zmk-app itself (for updating / merging) and thus the user's application also.
  • For users who are incapable of merging, we can just suggest creating (cloning) a fresh application and copying their key configuration files or changes across.

[innovaker]

For those out the loop, the general ideas for discoverability of modules has so far been:

• Maintaining an index on the website.
  • Module authors can PR a metadata file into the core.
  • Compilation of the metadata can be driven by webpack/netlify.
  • React/Docusaurus can be used to provide a explorable interface with search, categories and generate appropriate west.yml entries.
  • We could perhaps tie the front-end into the GitHub API as a stretch goal.
  • I appreciate this smells like a package repository.
• Encouraging a naming convention to make it easier to find ZMK modules via GitHub's search.

[innovaker]

I've released v2 of the modular PoC, which is (re-)based onto zmk's main as of today (f3502db which includes Zephyr v2.4.0).

I've updated the previous posts accordingly. If you've already tried v1 with Docker, you may have to remove the old containers before trying v2.

I have no intentions of making another release in the foreseeable future as it's quite an intensive process.

[Nicell]

While testing this on Windows I found an issue with finding module root paths causing mixed styles of path separators, which seems to be fixed in zephyrproject-rtos/zephyr#29243, but it isn't in a stable release of Zephyr yet.

For now I've manually added this fix to my local development workspace, but we should be aware of it when getting closer to a full implementation of this.

[petejohanson]

Some initial thoughts:

• While making ZMK a "core library" is generally a great move, 95% of users are unlikely to ever want to have a custommain entrypoint, versus having their custom code just be in their module w/ init callbacks, etc. Thoughts on having our core library still add a main unless a Kconfig option is set to disable that? In that scenario it's disabled, we would still export an entrypoint that the user main could invoke.
• Doing that possibly removes need for a CMakeLists.txt by default in the "user application".

Upstream modules:

I think templates like this are really important, ideally they can be as minimalist as possible while still offering the full features. Something I don't love about the current templates for the user config repos is them being external to the core repo, so changes that need coordination are harder.

I would propose those templates actually be in the main repo, and creating new downstream copies can pull from the subdirectory of the core.

Discoverability

One idea that came up earlier today was some GH label conventions for repositories that we could then query on ussng their API to find/discover modules for users.

[innovaker]

For the benefit of casual observers, there were some further discussions on Discord:

• https://discord.com/channels/719497620560543766/719544230497878116/806393254593429575
• https://discord.com/channels/719497620560543766/719544230497878116/806483474541707304

[innovaker]

While making ZMK a "core library" is generally a great move, 95% of users are unlikely to ever want to have a custommain entrypoint, versus having their custom code just be in their module w/ init callbacks, etc. Thoughts on having our core library still add a main unless a Kconfig option is set to disable that? In that scenario it's disabled, we would still export an entrypoint that the user main could invoke.

This makes sense and I'm fine with it. I also tested the principle last night. It's serves the majority but facilitates the minority.

[innovaker]

Doing that possibly removes need for a CMakeLists.txt by default in the "user application".

I doubt the bootstrapping code can be removed from the application but I'm not a cmake expert and welcome alternatives.

[innovaker]

I would propose those templates actually be in the main repo, and creating new downstream copies can pull from the subdirectory of the core.

I wasn't aware that pulling from a subdirectory of a git repository was possible (unless you're talking about partial clone?)? That's partly why I went for an upstream module approach in the PoC.

[schengnz]

Is there any possibility of decoupling the key processing engine as a separate module, independent of LED, key-scanning, sensors, display, power management and BLE/USB HID support? That way a key processing engine can be swapped in and out.

[Anutrix]

Any updates?

[Anutrix]

It's been almost 3 years. Has there been no updates or was this just not updated?

[caksoylar]

@Anutrix This is partially implemented in how currently user config repos work (where zmk itself is used as a Zephyr module and they support custom boards, shields and custom code for them). Maybe you should note what exactly are you looking to accomplish and what issue are you running into with the current limitations.

[Anutrix]

I am looking for support on ESP32-S3 which is supported officially by Zephyr Project(https://docs.zephyrproject.org/latest/boards/xtensa/esp32s3_devkitm/doc/index.html; https://github.com/zephyrproject-rtos/zephyr/commits/main/boards/xtensa/esp32s3_devkitm has had most features added by now.) but not ZMK.

It seemed that Modular ZMK would make adding support for it easier before I can deep type into this repo.

[caksoylar]

You can directly use boards from Zephyr in ZMK but you'll typically need to enable certain options, see some examples like for Adafruit KB2040 (overlay and conf). You could even do these overrides in a config repo, under config/boards/*.overlay etc.