west: sign.py: add "REM" support to pass comments through cpp

[marc-hb]

3 sign.py commits. Main commit:

west: sign.py: add "REM" support to pass comments through cpp

Generated outputs can be difficult to read, preserving comments helps a
lot and they often provide good git grep search keywords.

[marc-hb]

Successfully tested in commit https://github.com/marc-hb/sof/commits/toml-REM/

[andyross]

No complaints from me, seems like a useful feature. I will say though that this smells a little like a workaround for a design flaw: SOF, heh, sort of has a habit[1] of picking weak languages, trying to do complicated things with them, and then having to bolt on some kind of preprocessing to make up for the lack of programmability in the original.

So we have (here) rimage writing its config file in TOML, but then having to suck in non-trivial macrobatics to do what needs to be done, which requires debugging, which requires comments that survive the translation.

Maybe it might be time to think about generating the rimage config in a first class programming language (which you can simplify with a DSL layer as needed) rather than trying to build up toward a first class language with simpler tools?

[1] The true case study here was the attempt to bolt M4 around ALSA config files to generate topologies. The tplg2 stuff walked that back, but now that too is being passed through cpp!

[marc-hb]

Thanks for the review!

but then having to suck in non-trivial macrobatics to do what needs to be done, which requires debugging, which requires comments that survive the translation.

"requires" sounds too strong; this PR is a "quality of life" enhancement.

sort of has a habit[1] of picking weak languages,

Thanks for preaching to the choir (again) :-)

• rimage: add TOML preprocessing #65411 (comment) (where this cpp-based design was discussed)
• New functions to read elf files and parse modules (SP3) thesofproject/rimage#155 (comment)
• https://github.com/thesofproject/rimage/issues?q=crash
• • others if you follow these links

In theory, practice and theory are the same. In practice, people choose the languages they know and (think they) are the most productive with.

To be fair, this cpp-based solution seems very convenient to access config.h and similar. Zephyr is still 100% written in an ancient and weak language after all and it's not short of macrobatics either! "In Rome, do as the Romans do"

https://www.youtube.com/watch?v=w8GgP3h0M8M

• Explicitly allow C++ in tests #66053

The tplg2 stuff walked that back, but now that too is being passed through cpp!

I'm not familiar with topologies, do you have a link?

[nashif]

Each platform has and always had its own build-platform/ build
directory. Keep all such build directories consistent with each other to
simplify installation, checksumming and post-processing. We don't need
any build-mtl/**/mtl.toml path to tell us the platform name twice,
once is enough.

"Fixes" recent commit marc-hb@1533604 ("west: sign.py: generate platf.toml
from platf.toml.h with cc -E")

That commit message is confusing because it talks about how SOF builds things and not upstream Zephyr.

[marc-hb]

That commit message is confusing because it talks about how SOF builds things and not upstream Zephyr.

Thanks, I'll rephrase to be more neutral. Note the .toml pre-processing seems quite SOF-specific for now though.

[marc-hb]

v2; very small changes:

• Commit message fix (@nashif)
• Replace REM words by nothing which makes REM # mandatory (@lyakh)

Approvals lost please review.

[andyross]

refresh +1