-
-
Notifications
You must be signed in to change notification settings - Fork 492
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compiled Languages support / WASM language bindings #1784
Comments
I've written the low-level D bindings. The standard hello demo works on Linux and macOS. PR incoming soon. |
Awesome. Looking forward to it! |
@keithohara If you find the time to work on your C/C++ libs again too in the future that'd be awesome. |
A quick experience report, based on translating Lua examples on the wiki. The tri() example uses sin and cos functions.
One of the mouse examples constructs a simple string indicating current mouse coordinates.
Possibility: Modify TIC-80 so that Wasm3 exports more libc functions like trig and string handling functions. I'm also experimenting with the Free Pascal Wasm compiler. Currently it generates .wasm files that are too big for TIC-80. As an aside, Free Pascal also has a Javascript transpiler named pas2js. The examples implemented in Pascal and transpiled into Javascript work fine. |
This is why projects like https://github.com/WebAssembly/wasi-sdk exist. We need to focus on providing the proper sample build scripts for various languages - we don't need to wrap the entire C standard library ourself. I'm not happy with the current D support that was merged - it's incomplete and not following the WASM-4 examples. The WASM-4 build scripts for D use the WASI SDK and I assume the support over there is much better.
Does this work as intended in WASM-4? If you're familiar with D and wanted to make a PR to get us up to WASM-4 standards that'd be great! It should be pretty easy.
I disagree that we need to "implement" this. I'm not fully convinced this is our problem. It's certainly possible we might want to consider mentioning popular libraries in our sample projects, but ultimately users can use whichever libraries they want.
Are you using small/production builds? The easiest way to support any new languages is to get someone to do the work over in WASM-4 (it's increased popularity for WASM) and then port that to TIC-80. |
@PierceNg Ah it was your PR... It still needs to be fully brought up to the WASM-4 examples and then I think we'll be in a lot better place for D. Are you still wanting to finish that work? |
Yes. That's why I'm translating some of the Lua examples, to verify that dub.json and Makefile work for more than one example.
WASM-4's D Makefile doesn't work for me out of the box. Have to add
I meant the game programmer needs to implement string handling in their Zig code. Could be what I initially suggested, that TIC-80 exports libc functions, or as you say, which I agree is the better approach, have the D/Zig code use WASI SDK's libc. |
I was referring does using standard library stuff work better since I presume they are linking with WASI, etc... I assume from your PR the answer is yes. :-) |
Hi, I'm working on a Go binding for this. Looking at the WASM4 template for the Go programming language specifically uses tinygo. This compiler has the ability to specify a custom build target, which WASM4 defines like so:
In addition to the memory flags, what else should be added/adjusted? |
I think only the memory and stack flags... we're trying to follow WASM-4's lead on these things since there is a lot more active work going on there in WASM so far than here. So both memory would increase to 256kb and stack-size would increase to 96kb + 8kb I think? (our reserved IO RAM + 8kb stack)... therefore giving the stack 8kb to grow before it heads down into the 96kb of reserved RAM. Technically one could get away with less (since there is 12kb of reserved memory we aren't using yet at the top of that 96kb), but our defaults should be sane |
So is the
|
Yes, we want our stack at the bottom of RAM. But since the MMIO is all there that's why you have to add 96kb to the size... |
So like this?
|
I've started working on a Rust binding, and I have the core functionality working. I've only bound a few functions to test it so far, but I think the rest of the low-level bindings should be fairly easy to implement. I just wanted to get some input on a couple of details:
|
I'm not familiar with how Rust works... in most languages we're doing low RAM stack... so after the memory mapped IO, and stack... the rest is the heap... I thought Rust pioneered the low RAM stack... so perhaps I'm missing something here? Is the stack grows down, isn't the rest heap by default?
I'm not sure... so far we're ok with "basic" and "nicer" implementations... but if you're suggesting a thirst "nicer + perfectly rusty"... I'm not sure what category that falls into really. |
Well, normally, but in the WASM-4 Rust template and by extension my TIC-80 Rust template, a different memory allocator called |
Well sure - if you have to hard-code the heap size then we'd want to hard code it to take advantage of all the free RAM we have... but I'd also be curious to know how they got there. |
Okay, found the reasoning: aduros/wasm4#78. Also, for now I'm going to write nice wrappers but still require |
I've run into quite a big issue with my Rust template. In WASM-4, the first 4 bytes are unused, but in TIC-80, the memory map begins right away at address 0x00000. The problem here is that Rust makes some assumptions about the platforms it runs on, one of those being that address 0 is always a null pointer. In some situations it is possible to write code that accesses 0x00000, but it can't be guaranteed, since the compiler likes to optimise away any such access. All of this means that the first two pixels on the screen can't be accessed directly, only through the C API. At this point, I can see a couple of possibilities:
Of course, the WASM memory layout could be changed to avoid storing data at 0x00000, but that would be a major change and break binary compatibility, so probably not really an option. It definitely seems that I won't be able to get this perfect, so what should the priority be here? Keep small file sizes, or fully support the memory map? |
Yeah, changing the memory layout of the entire machine to make a single language happy isn't likely to happen.... perhaps if we were starting over from scratch. Are you sure there isn't some other way/third choice? I struggled with this in Zig for quite a while and then finally found the right magic "volative" syntax to say "no really let me have a pointer at 0 and stop complaining about it"... I don't think it had any effect on the compiled size. Accessing the framebuffer directly is kind of a key thing you'd want to do from a compiler language, so that seems a key thing to have in our library support... |
Really fairly sure, at least not without major modifications to the compiler itself. The null = 0 assumption runs pretty deep through the language and its optimisation features. I wonder if there's a less extreme way to tweak the memory map. Maybe a cart configuration option like |
That would require a LOT of special cases - the memmap is like hard coded into so much of the codebase. It's a literal chunk of 96kb in RAM in C. |
19kb out of 256kb isn't TERRIBLE if it's a one time cost... |
Correct me if I'm wrong, but since the code is stored in the BINARY chunk, isn't the limit actually 64k?
I don't need to change anything about that 96 KB block though. I think that the 96 KB block could be moved to the opposite end of the 256 KB block used by the WASM runtime with very little additional code. Then, the data on the C side is identical, but the Rust code sees all of the MMIO addresses shifted by 160 KB. This fixes the address zero issue, and as a bonus, the stack will no longer attempt to grow into the reserved memory. I might be wrong, but I'll put the Rust side on hold for now, and see if I can implement an alternate layout option. It feels odd to have a special case for one language, but I think it's a minor enough change that it's okay? |
When needed it's distributed across multiple chunks... so for a large cartridge where would be 4 banks of 64kb BINARY chunks, totaling 256kb. And even that is an arbitrary limit... we can have 512kb easy enough - more than that I'm not entirely sure.
All the memory related code would have to change - memcpy, peek, poke - which now introduces a slowdown for ALL platforms. The memory map inside WASM should not be any different than the memory map we expose via the internal API. IE, peek(0) and pointer to address 0 should be the exact same thing - reading the memory at address 0.
Personally I'd veto it and tell you not to bother, but nesbox always the final say. This topic came up during development though IIRC and again IIRC we were not interested in changing the memory map to make any particular language happy at that time. |
Oh yeah, forgot the memory related functions. In that case I'll just finish off the Rust bindings using Relying on this to access 0x00000 is probably not right, since that's still undefined behaviour, but I imagine it's highly unlikely to ever change. Plus, it really does seem to be the only remaining option, since even Basically, Rust is definitely not the right fit for a system with this sort of memory map. Ah well, at least it's somewhat functional now. |
Well I meant is it MOSTLY the 15kb upfront increase... if every bit of code is just a bit larger that's annoying but not terrible... IE 500 => 15kb OK... but 5000 = > 150KB BAD. 5000 => 5000 + 20kb "ok". |
So, I have a question about that in the Go binding I am working on. Go by design forbids implicit conversion (particularly between integer types). That would mean, as the native WASM API is currently implemented, care must be taken when calling any native API function to convert values as necessary. For example: tic80-go/tic80/tic80.go // tic80 Implements the native WASM API binding.
package tic80
// Btn executes the btn API call.
// go:export btnn
func Btn(id int32) int32 tic80-go/main.go package main
import (
"tic80-go/tic80"
)
// go:export TIC
func main() {
var buttonId int = 4
if (tic80.Btn(int32(id)) > 0) {
// More code...
}
} Am I permitted to write my own version of the calls in Go, defaulting to the native API only when necessary? Or should I wait until the WASM API can be standardized? |
How would that help the situation? I think I'd need a specific example. If this is just a Go thing - it sounds pretty annoying... I'd expect that the per language wrapper (one layer above WASM) would use the "most natural types" for each API call - and then do the conversions itself to what WASM wants... so if ints are commonly used everywhere (vs the specific int32) then the wrappers job would be to deal with that annoyance, so users of the library don't have to think about it. So I'm not seeing the need to "write your own version" so much as to provide a wrapper that deals with common types or does type conversions that people would typically expect from a Go library - and then call the native APIs. |
In the process of fixing another function signature bug, I encountered some issues with the Zig template:
By the way, I should hopefully be done with my Rust template soon, and I'm borrowing the "structs as default arguments" idea from the Zig template for my wrappers, so thanks for that! |
PR welcome.
I think it might be nice to have both but if that's just too much then I agree that the demo using the wrapped version would be nicer. And yes, the wrapped version came after everything was all working.
I'm slowly forgetting it already, but yes to pass a struct inline to a function I think you need the |
Yeah, I'll probably open a PR soon with a collection of fixes for the Zig template. I figured I'd just check here first to see if anyone had any further comments before I started making changes.
That would be best, and personally I think it's not too much. I'm just not sure of the best way to structure it, since so far there's only one template per language. Maybe just dropping an extra source file in alongside the current main.zig? That makes sure that the tic80.zig file doesn't need to be duplicated. Thoughts? |
Sure, but how does that work with the build system? |
Poorly, I haven't quite found a solution that works well with existing build systems, particularly when considering how it will work for other languages that may be supported in future. For now I've decided to just use the wrapped version of the API, and not include an example of the raw one, since I imagine that will be what most users of the WASM templates want to work with. |
I think that's fine, that's what I'd recommend to 99% of people... raw is just there for completeness I think. |
I was also looking into making a rust template. The This fixes the problem, since only the rust compiler has issues. |
Hmm, that's a cool way to solve it. At this point though, at least until #1956 is worked on, I'll stick to using the standard library wrapper version I currently have. While they're slightly slower, they are also far more flexible since they can access the entire 256K. |
After working on the Go binding for quite some time, I cannot seem to get it to work completely. I have made the work I have done available as a repository here. Most API calls work, it's just that any calls that require transparency or strings just state "missing imported function" with no further information. Hope someone can get this to work as I really would like to use Go. I am considering giving Rust a try though (by the way, nice work). |
Can you give me a specific API call and point me to the exactly game code (line # please)... lets start with transparency and see where we get... For example, do you have a problem with nothing but: https://github.com/sorucoder/tic80-go/blob/master/main.go#L27:
Will that line alone produce the issue? |
Usually this indicates a data type misalignment or passing the wrong # of arguments... have you looked at the Go code on the WASM4 to see if they're passing things any differently? (or perhaps their API has no some more complex data types?) |
Like I said, any code that uses transparency or strings causes an error. If I take out the calls to func Spr(id, x, y int, options *SpriteOptions) {
if options == nil {
options = &defaultSpriteOptions
}
transparentColorBuffer, transparentColorCount := toCBuffer(&options.transparentColors)
rawSpr(int32(id), int32(x), int32(y), transparentColorBuffer, int8(transparentColorCount), int32(options.scale), int32(options.flip), int32(options.rotate), int32(options.width), int32(options.height))
} It is simply a wrapper to the real API call. The other lines of code do the conversion work to conform to the real API call. We can ignore the transparentColorBuffer, transparentColorCount := toCBuffer(&options.transparentColors) It converts a func toCBuffer(goBytes *[]byte) (buffer unsafe.Pointer, count int) {
sliceHeader := (*reflect.SliceHeader)(unsafe.Pointer(goBytes))
buffer = unsafe.Pointer(sliceHeader.Data)
// For some odd reason, tinygo considers the type of reflect.SliceHeader.Len to be uintptr,
// instead of int. Using the builtin len function instead.
count = len(*goBytes)
return
} All that is left is to analyze the calling signature of the real m3ApiRawFunction(wasmtic_spr)
{
m3ApiGetArg (int32_t, index)
m3ApiGetArg (int32_t, x)
m3ApiGetArg (int32_t, y)
m3ApiGetArgMem (u8*, trans_colors)
m3ApiGetArg (int8_t, colorCount)
if (trans_colors == NULL) {
colorCount = 0;
}
m3ApiGetArg (int32_t, scale)
m3ApiGetArg (int32_t, flip)
m3ApiGetArg (int32_t, rotate)
m3ApiGetArg (int32_t, w)
m3ApiGetArg (int32_t, h)
tic_mem* tic = (tic_mem*)getWasmCore(runtime);
// defaults
if (scale == -1) { scale = 1; }
if (flip == -1) { flip = 0; }
if (rotate == -1) { rotate = 0; }
if (w == -1) { w = 1; }
if (h == -1) { h = 1; }
tic_api_spr(tic, index, x, y, w, h, trans_colors, colorCount, scale, flip, rotate) ;
m3ApiSuccess();
} compared to mine: func rawSpr(id, x, y int32, transparentColorBuffer unsafe.Pointer, transparentColorCount int8, scale, flip, rotate, width, height int32) TLDR; I'm in over my head here. |
Can you get it working if you change the code to just pass two integer parameters and just pass zero for both of them? |
Next ideas:
|
Can anyone make an Assembly Script binding for tic 80 wasm 4 or help me pls |
I had someone help me fix the Go binding I have been working on. It now works and should be ready to go. The repository is here. |
@sorucoder Looks awesome! |
@soxfox42 I have found how to deal with this issue. Since Rust 1.66 we have stable #[export_name = "TIC"]
pub fn tic() {
unsafe {
let nullptr = std::hint::black_box(0 as *mut u8);
// Same as `tic80::poke4(0, 12)`
*nullptr = 12;
};
} Although it is stated in Rust doc that this function should not be relied upon to control critical program behavior, I think it is still better than having no optimizations at all. I would make a PR but I do not know if this kind of hack is acceptable. |
Which parts of the API are we talking about using this hack, exactly? |
I think I found another way a while ago and never got around to updating the docs. I believe |
@joshgoebel |
Any chance we can get AssemblyScript bindings, or at least have it added to the list in the first post? https://www.assemblyscript.org/ |
Done. These bindings are all community provided... if you want to contribute... |
I've recently made one game with Odin lang and TIC-80 for game jam and have made some kind of bindings for that purpose. It's not very clean and does not cover 100% of TIC-80 API and have some extra methods that I've found convenient but if I'll have enough time I can try to make a fully covered bindings and project template from that. That language really deserves some attention in gamedev space in my opinion. |
Creating a master topic to link to from our wiki/documentation for those wishing to contribute language templates/libraries.
We should be able to support all the same languages as the WASM-4 project. The list:
--script: wat
to support raw textual web assembly #1849.Essentially any language that compiles to WASM and allows you to configure the memory layout is a potential.
What is needed? Port over the build/template scripts from the WASM-4 project templates and update for TIC-80. These templates are typically well thought out and well maintained so they are a great starting point.
templates
wasm4.h
library headertic80.h
library headerIf you'd like to help speak up on this thread.
What is the core API?
Reference:
The ZIG API definition is pretty easy to read:
It follows VERY closely to the native C API which you can find in:
core.c
,io.c
, anddraw.c
It's often quite a bit lower level than the API docs on the wiki, so don't rely on those TOO much.
The text was updated successfully, but these errors were encountered: