Bytes as ArrayBuffer

[Scooletz]

This PR introduces a capability to convert raw byte chunks like byte[] as an ArrayBuffer. It does it by implementing an object converter directly in Jint core project. This is done to not make the ArrayBufferInstance public and keep the API change as minimal.

Open questions:

1. Why the test is failing? I tried to debug it and I fail to understand why an ArrayBuffer is not an instance of ArrayBuffer. I think it's not realm connected. I observed that Object.prototype.toString returns something different than expected as well. I'd appreciate some help.
2. Are we ok with introduction of a custom object converter in the core?
3. Are we ok to implement a similar approach for Memory<byte> which would require offset bookkeeping in the ArrayBuffer (I can implement it).

[lahma]

This PR introduces a capability to convert raw byte chunks like byte[] as an ArrayBuffer. It does it by implementing an object converter directly in Jint core project. This is done to not make the ArrayBufferInstance public and keep the API change as minimal.

Nice! Here's some feedback from the top of my head.

I think it would also be fine if we rename ArrayBufferInstance to more idiomatic JsArrayBuffer and expose it as public, generally things are internal if they haven't been API-checked for general consumption. We've already changed DateInstance to JsDate etc to get clean API. This would be for the case if you want to return from your c# code something like engine.Realm.Intrinsics.ArrayBuffer.Construct(bytes); (see below).

1. Why the test is failing? I tried to debug it and I fail to understand why an ArrayBuffer is not an instance of ArrayBuffer. I think it's not realm connected. I observed that Object.prototype.toString returns something different than expected as well. I'd appreciate some help.

The constructed object is not properly initialized, I'd suggest adding new Construct method to ArrayBufferConstructor:

public ArrayBufferInstance Construct(byte[] data)
{
    var obj = OrdinaryCreateFromConstructor(
        this,
        static intrinsics => intrinsics.ArrayBuffer.PrototypeObject,
        static (engine, _, state) => new ArrayBufferInstance(engine, state!),
        data);

    return obj;
}

And changing your converter logic to:

if (value is byte[] bytes)
{
    result = engine.Realm.Intrinsics.ArrayBuffer.Construct(bytes);
    return true;
}

Note, now ArrayBufferConstructor could also be made public if there's use case for such construction and everything "unnecessary" is hidden as internal/private.

2. Are we ok with introduction of a custom object converter in the core?

I think there might be small problem with ambiguity here. You could also want to have your byte[] to be converted to Uint8Array or Uint8ClampedArray - so automatic conversion during runtime might not be obvious, or one could want to have both possibilities (converting to both ArrayBuffer or Uint8Array).

3. Are we ok to implement a similar approach for Memory<byte> which would require offset bookkeeping in the ArrayBuffer (I can implement it).

Everything what makes Jint more usable in interop scenarios sounds good to me.

[Scooletz]

Thank you for a prompt and in-depth reply @lahma . This is my first more serious work with Jint codebase so that I'm stepping into unfamiliar territory. Let me dive deeper with your comments in mind and get this PR altered over the weekend.

[lahma]

With #1591 I did some preliminary work and did the rename and expose of type. If you find it useful for future, could also unseal it if there's some benefit with different backing store implementations.

So if you rebase, you should have things readily available.

[sfmskywalker]

I stumbled upon this PR via this discussion while looking for a way to pass in CLR byte[] values into the Engine and then return it from another JS method, e.g.

byte[] fileBytes = await LoadLargeFileAsync();
engine.SetValue("fileBytes", fileBytes);
var result = engine.Evaluate("return fileBytes;");
var bytes = result.ToObject(); // becomes object[], but I want byte[].

Currently, I am solving this using a custom converter that is close to implementation as proposed via this PR:

internal class ByteArrayConverter : IObjectConverter
{
    public bool TryConvert(Engine engine, object value, [NotNullWhen(true)] out JsValue? result)
    {
        if (value is byte[] bytes)
        {
            result = engine.Intrinsics.Uint8Array.Construct(bytes);
            return true;
        }

        result = JsValue.Null;
        return false;
    }
}

However, looking at the Uint8Array.Construct method, it internally copies the source byte array into the typed array instance, which might be not very efficient for large byte arrays given all of the processing of each individual byte, which seems unnecessary in my case (correct me if I'm wrong).

The proposed PR, on the other hand, using a new ArrayBufferInstance type, seems to be exactly what I am looking for.

@Scooletz do you still want to complete this PR, and/or would you like me to take it from here? I could fork your fork or perhaps you can give me push access to your fork so that I can update this PR directly.

[sfmskywalker]

@lahma It looks like that if I can somehow instantiate the renamed JsArrayBuffer from my custom object converter, I would be good to go.

I'm also wondering if it would make sense to update DefaultObjectConverter with a type map between byte[] and JsArrayBuffer?

[lahma]

I'll try to create a intermediate step PR to improve APIs to allow passing in existing items.

[lahma]

@sfmskywalker please see #1890 and whether it would help you achieve your goals. The automatic byte[] to ArrayBuffer would be a breaking change as it would also mean all other number array types would/could get the same conversion.

[sfmskywalker]

@lahma Your PR is spot-on to achieving my goals 🙌🏻 Thank you for the super fast turnaround!

[Scooletz]

I don't have capacity to push it further. Should I close?

[lahma]

I'll close this as I think the most important bits are now supported. Thanks for bringing this up and pushing the capabilities further.