[FEATURE] Ability to use Zephyr shell with SOF

[kv2019i]

Is your feature request related to a problem? Please describe.
Add overlay (or build option) to implement Zephyr shell (https://docs.zephyrproject.org/latest/services/shell/index.html) support to SOF, and a backend that allows to attach to the shell from host CPU (over e.g. a memory window backend). This offers a new debugging and development option for SOF development, opening up possibility to observe system state and execute test sequences on the DSP.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[kv2019i]

Working on this now,

[lgirdwood]

@andyross @cujomalainey fyi

[cujomalainey]

Interesting, would it just appear as a tty to the kernel?

[lgirdwood]

Interesting, would it just appear as a tty to the kernel?

That's the intention. The kernel tty would just be a passthrough from the DSP. Similar to how GDB would work.

[andyross]

Ended up here after commenting on this Zephyr PR: (zephyrproject-rtos/zephyr#72158)

This sounds cool. Ironically I've never been a shell person on the Zephyr side (almost always if you're chasing a bug it's easier to just rebuild/reflash the whole app rather than trying to "use it interactively").

But one big feature it would add here is the ability to do whitebox testing on live audio firmware, something that the kernel driver normally tries its hardest to prevent. Having a backchannel from userspace would be really nice, even if you have to use the slightly crufty shell API to do it.

[lgirdwood]

But one big feature it would add here is the ability to do whitebox testing on live audio firmware, something that the kernel driver normally tries its hardest to prevent. Having a backchannel from userspace would be really nice, even if you have to use the slightly crufty shell API to do it.

Ack, we were thinking things like injecting errors in pipelines, dumping state during tuning- really a method to help increase technical velocity in many ways.

[kv2019i]

First Zephyr PR submitted for review zephyrproject-rtos/zephyr#72738 . SOF PR to add a config overlay and (possible) Linux SOF client to create the pty, will come later.

[cujomalainey]

@johnylin76 with relation to #5521

[kv2019i]

Current set of PRs in progress (updated May 31)

• Zephyr: shell: add shell backend for audio DSP using shared memory window zephyrproject-rtos/zephyr#72738
• Zephyr: dependency for above: soc: intel_adsp: tools: merge cavstool.py and acetool.py code zephyrproject-rtos/zephyr#72799
• SOF: dependency to update Zephyr west.ytml: update to Zephyr 53ddff639562 (May 31st) #9179
• SOF: app: add new shell_overlay.conf #9133
• SOF: zephyr: shell: add initial SOF custom commands #9174
• SOF docs: developer_guides: debugability: shell: add usage documentation sof-docs#496

[kv2019i]

Dependencies merged, now waiting for the zephyrproject-rtos/zephyr#72738 to be merged in Zephyr.

Uploaded a sof-docs PR to thesofproject/sof-docs#496 to document shell usage with some examples.

[kv2019i]

While waiting for Zephyr side PR to be merged, here's one more draft on SOF side that adds first custom command:

#9174

[kv2019i]

Shell backend now merged to Zephyr. Next step is to bring in the changes to SOF main via #9179

[kv2019i]

SOF main now has shell support, overlay support marked ready for review: #9133

[kv2019i]

Main feature completed and merged to v2.10. The SOF side config overlay (convenience to build with shell) and additional SOF custom commands, are still in progress, but these goes go out of the scope for this original feature request.

Follow-up PRs:

• SOF: app: add new shell_overlay.conf #9133
• SOF: zephyr: shell: add initial SOF custom commands #9174
• SOF docs: developer_guides: debugability: shell: add usage documentation sof-docs#496