TIC80 and languages support overview

[msx80]

Hi there :) It would be nice (both for contributors and users) to know in general what are the plans for the future of TIC80 in regards to languages support. There are already a number of languages (lua, js, wren, squirrel etc), and some more are in development or available as pull requests. Now, it's obviously good to support a wide selection of languages, to widen the audience and offer more choices etc, but there are drawbacks too, for example each new language bloats the build process, have to be maintained both for updates on the language itself and for tic80 (new apis or revision on existing ones). Some might not compile on all platforms or require special care.

I don't think it's feasible to just keep adding languages upon languages, at least with the current setup, but on the other hand stopping new addition is weird, expecially since there are mainstream languages like python currently not included while much more obscure languages like wren are. Btw, it would be good to know the distribution of languages across submitted carts. I would guess it's like 95% lua.

Since it's ultimately a preference choice, it's up to @nesbox to decide, so it would be nice to know what's in his mind :)

If we plan to "keep adding languages", i suggest to make the process more steamlined and decide on some conventions. I've actually tried it on myself while adding support for wasm language, and i encountered some areas that could use some cleaning, for example:

• language support is spread in a lot of different source files in the form of switches etc. Implementing a language require to search them all around the code. It would be better and not much hard to centralize the management. Perhaps each language could provide a common "configuration" struct ( much like the current tic_script_config but with more info), that gets added to a global array if the language is enabled, so that the rest of the code can simply loop the array in a language-agnostic way, without language specific switches.
• the API specifies function that are variadic and dinamically typed, but not all languages support varargs/function overloading, so for example it could be the case that "draw" must be defined with different names for different set or type of parameters. A convention like draw1, draw2 etc could be agreed upon (this was the case for wasm, for example, or it would be for C).
• Ideally each API would translate to an internal, identical call to tic engine, to minimize noise on the "glue" code. While this is often the case, some times it is not, for example languages implementing "btn" go directly to tic80 memory to pick the data instead of calling a "tic_api_btn" function (example). Even worst the "tic_api_btn" exists but does something different, which is pretty confusing (at least, i was confused :) )

So these are my two cents, what do you all think ? :)

[Anrock]

I agree - having single documented place for new language addition would be really nice. So far I've spent more time trying to understand what's going in TIC-80 codebase with my super rusty C skills than actually implementing a new language.

Also +1 on slimming down amount of API calls in favor of direct memory access at some documented location. Memory-mapped I/O is way less magical and more retro-feeley. But that's a view through my "Retro CPU" glasses, so it's highly biased.

[RobLoach]

Web Assembly would be the language to rule them all 😉 ..... It would allow support for Rust, Go, ES2020 JavaScript, etc!

[msx80]

Well running all languages throu a single bytecode/interpreter would be super useful but i don't think it's possible.. Like lua would be doubly interpreted, performances would sink. And it would certainly be cumbersome.

[nesbox]

I think we have to keep adding new languages and it's weird to stop doing this because it will increase the entropy of the universe :)

• let's do some refactoring and put all the lang related code to the one place tic_script_config or smth else :)
• we could try to use named parameters for non-variadic languages
• yes, we have to refactor this noise glue code, will try to do it soon

Thank you for your thoughts 👍

[msx80]

Hi, having picked up tic80 again i checked on this topic and found that there have been some improvements in the code. I noticed that now the script related switches are limited to a couple of places, and SCRIPT_LIST/SCRIPT_DEF is used extensively in the code to "generalize" on languages.
Now it's nicer than before, but i think it can be improved more:

• There are still different places where per-language code is used, for example in getDemoCart() in console.c (to include language-specific demos) and in tic80_config.h, where file extension is defined and single comment is re-defined (it's also in tic_script_config) (at least i think it's the same thing?).
• The macro machinery (SCRIPT_LIST/SCRIPT_DEF) is not particularily readable or practical.

I suggest the following:

• move all language specific stuff to the respective tic_script_config (like: adding the following fields: fileExtension, demoRom, demoRomSize). tic_script_config shall be the one-stop place where everything related to a language engine is defined. Implementing a new language would be basically filling in a tic_script_config.
• create a global static array of tic_script_config that's populated with all _tic_script_config_s available at build (much like tic_core_script_config already does, but privately, in core.c)
• replace the SCRIPT_LIST machinery with access or searches in the array when a specific config has to be found starting from either the file extension or the "script" metatag.
• add a pointer in tic_core pointing at the "current" tic_script_config. This could replace (or be integrated into) the tic_core_script_config() function which dinamically detect which language you're writing in.
• unify the "vm" fields in tic_core, making it a single opaque pointer. Currently the struct has a different, typed pointer for each language. I don't think tic_core should know about languages, it should have a single "opaque" pointer to pass to the correct tic_script_config, which can cast it to the actual type. So tic_core will have: 1) a current tic_script_config and a wm associated to it. Together they can generically define the current running wm.

If you like this proposal i can try and implement it.

[Anrock]

@msx80 yes please. If you also write some kind of tutorial / docs on how to approach implementing new language that would be super sweet.

[nesbox]

Hi, sorry for the delay with the response
Your thoughts look well, one thing, we have places where we generate strings with the supported scripts in compile-time, for example, here

TIC-80/src/studio/screens/console.c

Line 142 in b1d24e6

#define SCRIPT_DEF(name, ...) " "#name

or here

TIC-80/src/studio/screens/console.c

Line 2524 in b1d24e6

#define SCRIPT_DEF(name, ...) #name "|"

Of course, we have to generate these strings in run-time if you move all the things under the tic_script_config.
So, I like the proposal and will appreciate your help. Also, docs about adding new lang will be very helpful.
Thank you.

[msx80]

I've made a pull request with the first batch of these modifications. Man working with strings in C is a painful pain :)
Let me know if it's ok, then i'll tackle the "vm" stuff

[msx80]

I'm writing a small doc on how to include a new language in TIC-80 :)
In the meantime, is there still interest in supporting web assembly? I had a working prototype before, it would be even easier to bring it to the new architecture. If this is the case i could open a new discussion to talk about how to handle the thing in the best way.