Add callback to handle memory.grow failures

[eloparco]

When embedding WAMR, this PR allows to register a callback that is invoked when memory.grow fails.

In case of memory allocation failures, some languages allow to handle the error (e.g. by checking the return code of malloc/calloc in c), some others (e.g. Rust) just panic. For the latter, it is useful to have a callback function that would be called before the wasm execution crashes, allowing the embedder to detect and handle the memory.grow error. It comes in handy when using shared memory since shared memory forces the maximum linear memory size to be set at compilation time. Setting a value that is too low can lead to memory allocation failures (i.e. memory.grow failures).

It would be nice to capture the crash in Rust directly, but the feature is experimental (rust-lang/rust#51540).

[yamt]

allowing the embedder to detect and handle the memory.grow error

do you mean just printing a diagnostic message?
or are you thinking about more interesting error handling?

[eloparco]

do you mean just printing a diagnostic message?
or are you thinking about more interesting error handling?

My use case is more about logging, storing some stats and doing some cleanup in the embedder. Without that, the execution can crash at any point (in which there's an allocation) and from the embedder there won't be any way to automatically understand that the cause is the memory.grow failure.
I guess there are other possible scenarios and exposing the callback can unlock them.

[yamt]

do you mean just printing a diagnostic message?
or are you thinking about more interesting error handling?

My use case is more about logging, storing some stats and doing some cleanup in the embedder. Without that, the execution can crash at any point (in which there's an allocation) and from the embedder there won't be any way to automatically understand that the cause is the memory.grow failure. I guess there are other possible scenarios and exposing the callback can unlock them.

for logging purpose, i guess it's better to provide more info like:

• context
  • wasm_exec_env_t
• memory
  • module instance
  • memory idx (currently always 0 though)
  • current size of the memory
• the cause of the failure

[Zzzabiyaka]

just a general note: is there any specific reason why we need goto statement here?

why cannot we call a separate function and then just return false?

maybe I'm missing some context

[eloparco]

just a general note: is there any specific reason why we need goto statement here?

It just looked cleaner than calling enlarge_memory_error_cb() in all the places where wasm_enlarge_memory_internal() returns.

[wenyongh]

LGTM

[yamt]

lgtm

[lum1n0us]

Without that, the execution can crash at any point (in which there's an allocation) and from the embedder there won't be any way to automatically understand that the cause is the memory.grow failure.

Be curious about the use case. Does it like

• memory.grow failed in runtime
• rust(user code or std) detects the failure and panic!
• embedded doesn't know anything but WASM app exits.

I guess there are other possible scenarios and exposing the callback can unlock them.

It seems like a general global exception handler with variants exceptions is better than a set of callbacks for every possible scenarios.

[eloparco]

Be curious about the use case. Does it like

• memory.grow failed in runtime
• rust(user code or std) detects the failure and panic!
• embedded doesn't know anything but WASM app exits.

Yes, that's my use case

It seems like a general global exception handler with variants exceptions is better than a set of callbacks for every possible scenarios.

Do you mean to implement a general global exception handler in the runtime?
The best option would probably be to override the exception handler for memory allocation in Rust, but that feature is experimental. The problem doesn't appear in C since there we can just check the return code of malloc/realloc in user code.

[lum1n0us]

The problem doesn't appear in C since there we can just check the return code of malloc/realloc in user code.

Does the uncheckable exception only happen during allocation or it is a general behaviour in Rust? Is it safe to say panic!->abort() (Rust->Wasm) is the root cause of such scenarios?

[eloparco]

Does the uncheckable exception only happen during allocation or it is a general behaviour in Rust?

Yes, the one that I was trying to address happens in Rust whenever a memory allocation fails. It can be raised by any library included (e.g. standard library) and currently there's no way to catch it. To reproduce it, you can just set the maximum linear memory size to a small value and compile some Rust code that allocates memory.

Is it safe to say panic!->abort() (Rust->Wasm) is the root cause of such scenarios?

I think so.

[loganek]

I think it's worth to mention in this thread that (a cross-runtime) alternative for that would be to add a compiler pass that wraps memory.grow opcode with some additional error checking logic. And that can be extended to handle any other future usecases.

[lum1n0us]

Is it safe to say the embedded will meet an un-handled exception when the rust code hit a panic!? no matter it is caused by allocation failure, out-of-length index accessing or anything else.

If so, shall we avoid panic! especially when plan to be transferred to Wasm? Maybe use Result<T,E> as a return instead

[eloparco]

If so, shall we avoid panic! especially when plan to be transferred to Wasm? Maybe use Result<T,E> as a return instead

The panic! is not called explicitly (in the user code compiled to wasm) but happens whenever Rust fails memory allocation. Failure to allocate memory is an unrecoverable error in Rust.

[lum1n0us]

Is it safe to say the embedded will meet an un-handled exception when the rust code hit a panic! no matter if it is called by user code(compiled to wasm) or not.

[eloparco]

Is it safe to say the embedded will meet an un-handled exception when the rust code hit a panic! no matter if it is called by user code(compiled to wasm) or not.

Yes