Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ext/ffi): Callbacks #14663

Merged
merged 72 commits into from
Jun 20, 2022
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
291ca64
Start rewrite
aapoalas May 13, 2022
6f955d0
Further work
aapoalas May 13, 2022
352d98e
Scope in stack or not
littledivy May 14, 2022
15261ea
Functioning synchronous registered callbacks
aapoalas May 14, 2022
f98424f
Don't borrow across callback call
littledivy May 14, 2022
8b46001
Move forward with parameter types in FFI callbacks
aapoalas May 17, 2022
7a10c38
Adapt, does not function properly
aapoalas May 17, 2022
2335d2a
Use serde_v8::Value
aapoalas May 18, 2022
f63ceba
fmt
aapoalas May 19, 2022
cd8260d
Pick ops stuff from sqlite bindings
aapoalas May 21, 2022
78a12cd
Try async fixing
aapoalas May 21, 2022
4de8ef8
Work work
aapoalas May 21, 2022
8a6dace
Somewhat handle nonblocking calls
aapoalas May 22, 2022
90c932d
Not really well working
aapoalas May 22, 2022
3c10894
Fix lifetime issues with ffi_parse_args
aapoalas Jun 7, 2022
e3b55f6
Error handling
aapoalas Jun 7, 2022
8e63797
async work, thanks divy
aapoalas Jun 8, 2022
15d0fbf
Return lifetimes
aapoalas Jun 8, 2022
69ae9bf
fmt & lint
aapoalas Jun 8, 2022
48f439d
fmt & lint
aapoalas Jun 8, 2022
23f5800
Actually pass RegisteredCallback RID to FFI
aapoalas Jun 8, 2022
6971d16
Actually functioning registered callbacks!
aapoalas Jun 9, 2022
e271cc7
fmt & lint
aapoalas Jun 9, 2022
98110af
Remove likely unnecessary active scope tracking from ffi calls
aapoalas Jun 10, 2022
ccdd122
ops: support a result returning a future returning a result.
littledivy Jun 11, 2022
283dc15
reenable nonblocking ops as hybrids
littledivy Jun 11, 2022
579015c
Merge branch 'main' into feat/ffi-register-callback-rewrite
aapoalas Jun 11, 2022
4fd175f
Rewrite ext/ffi to return serde_v8::Value
aapoalas Jun 11, 2022
f4a8fe2
Fix
aapoalas Jun 11, 2022
f1eed63
Move parameter validity checks to JS side
aapoalas Jun 11, 2022
7478ead
Fix nonblocking calls by reverting them to json::Value and U32x2 retu…
aapoalas Jun 11, 2022
c98241b
Add extensive FFI baseline benchmarks
aapoalas Jun 11, 2022
6c0ffe8
fmt & lint
aapoalas Jun 11, 2022
297dab7
Minor improvements
aapoalas Jun 12, 2022
87e10ca
Fix tests FFI calls
aapoalas Jun 12, 2022
f357cea
Merge branch 'main' into feat/ffi-register-callback-rewrite
aapoalas Jun 12, 2022
7b2c176
fmt
aapoalas Jun 12, 2022
3d53899
Fix more FFI ops call signatures
aapoalas Jun 12, 2022
2f370c9
Fix check_unstable2 usage
aapoalas Jun 12, 2022
e6fad7a
Fix mistaken Symbol usage with ptr FFI calls
aapoalas Jun 12, 2022
06dee1f
Fix buffer returning with quite manual V8 value picking
aapoalas Jun 12, 2022
c29d8fd
Add callback tests
aapoalas Jun 12, 2022
69eec01
fmt & lint
aapoalas Jun 12, 2022
6db458b
Expose registered callback pointer values
aapoalas Jun 12, 2022
84ddbda
Use primordials
aapoalas Jun 12, 2022
bcc5caf
Improve deno_ffi_callback inline comment
aapoalas Jun 12, 2022
77f3f51
Catch and rethrow errors from FFI callbacks
aapoalas Jun 12, 2022
ce5b3cb
Fix idiosyncracies, add some types at least
aapoalas Jun 13, 2022
3f11770
Use pointer values for RegisteredCallbacks, removing resource_table u…
aapoalas Jun 13, 2022
53bfa8f
Fix errors in i64 / u64 callback return values
aapoalas Jun 13, 2022
7d75f7d
Review comments
aapoalas Jun 14, 2022
3b8a881
Rename RegisteredCallback to UnsafeCallback
aapoalas Jun 14, 2022
6ac3471
Use specific APIs for ints
aapoalas Jun 14, 2022
a781463
Use specific APIs for ints in callbacks, avoid serde_v8 in callback p…
aapoalas Jun 14, 2022
0c67852
Implicit drop
aapoalas Jun 15, 2022
d936617
Review fixes, light renaming
aapoalas Jun 15, 2022
798ebf5
Safety notice on UnsafeCallbackResource close()
aapoalas Jun 15, 2022
56004f2
Missing permissions check, remove left-over op, add notice about Unsa…
aapoalas Jun 15, 2022
3cf98c4
Return v8::Value directly in UnsafeCallback constructor
aapoalas Jun 15, 2022
3582d94
Merge branch 'main' into feat/ffi-register-callback-rewrite
littledivy Jun 16, 2022
179074d
Merge branch 'main' into feat/ffi-register-callback-rewrite
littledivy Jun 16, 2022
9f5c8df
Fix UnsafeCallback types and add extensive type tests
aapoalas Jun 16, 2022
a84c3f3
Merge branch 'main' into feat/ffi-register-callback-rewrite
aapoalas Jun 16, 2022
fc2f2e1
Fix format
aapoalas Jun 16, 2022
27accd9
Remove unnecessary try-catch from deno_ffi_callback
aapoalas Jun 16, 2022
4131bfa
Undo unrelated formatting changes
piscisaureus Jun 17, 2022
2eeffd0
Merge branch 'main' into feat/ffi-register-callback-rewrite
littledivy Jun 18, 2022
bd4c1f7
Merge branch 'main' of github.com:denoland/deno into feat/ffi-registe…
littledivy Jun 18, 2022
5184ddd
Add test for error propogation
littledivy Jun 18, 2022
40aac23
Format
littledivy Jun 18, 2022
0780dbc
throwCallback.close()
littledivy Jun 19, 2022
3218386
Merge branch 'main' into feat/ffi-register-callback-rewrite
littledivy Jun 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 44 additions & 17 deletions cli/dts/lib.deno.unstable.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,7 @@ declare namespace Deno {
* });
* ```
*/
export function bench(
name: string,
fn: () => void | Promise<void>,
): void;
export function bench(name: string, fn: () => void | Promise<void>): void;

/** Register a bench which will be run when `deno bench` is used on the command
* line and the containing module looks like a bench module.
Expand Down Expand Up @@ -201,9 +198,7 @@ declare namespace Deno {
* const { columns, rows } = Deno.consoleSize(Deno.stdout.rid);
* ```
*/
export function consoleSize(
rid: number,
): {
export function consoleSize(rid: number): {
columns: number;
rows: number;
};
Expand Down Expand Up @@ -346,9 +341,16 @@ declare namespace Deno {
| "f64"
| "pointer";

type NativeParameterType =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking, maybe it makes things more explicit if we had NativeParameterType and NativeReturnType, as in:

  • NativeParameterType can be sent from JS to C
  • NativeReturnType can be sent from C to JS

Of course in case of callbacks the types would be reversed: parameters would be NativeReturnType and the return value would have type NativeParameterType.

So maybe we should considerToNativeType and FromNativeType or something.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, and do plan on doing a rework on the FFI TS types. f.ex. Currently NativeType is the "base type" and NativeNumberType is a narrowed down type from that. I will do a follow-up to rework the types into something that hopefully resembles a reasonable whole.

A ToNativeType and FromNativeType sounds splendid for that.

| NativeType
| {
function: Omit<ForeignFunction, "nonblocking">;
};

/** A foreign function as defined by its parameter and result types */
export interface ForeignFunction<
Parameters extends readonly NativeType[] = readonly NativeType[],
Parameters extends readonly NativeParameterType[] =
readonly NativeParameterType[],
Result extends NativeType = NativeType,
NonBlocking extends boolean = boolean,
> {
Expand Down Expand Up @@ -395,7 +397,9 @@ declare namespace Deno {
: unknown;

/** Infers a foreign function parameter list. */
type StaticForeignFunctionParameters<T extends readonly NativeType[]> = [
type StaticForeignFunctionParameters<
T extends readonly NativeParameterType[],
> = [
...{
[K in keyof T]: StaticForeignFunctionParameter<T[K]>;
},
Expand All @@ -412,8 +416,10 @@ declare namespace Deno {
: T extends ForeignStatic ? StaticForeignFunctionResult<T["type"]>
: never;

type ConditionalAsync<IsAsync extends boolean | undefined, T> =
IsAsync extends true ? Promise<T> : T;
type ConditionalAsync<
IsAsync extends boolean | undefined,
T,
> = IsAsync extends true ? Promise<T> : T;

/** Infers a foreign library interface */
type StaticForeignLibraryInterface<T extends ForeignLibraryInterface> = {
Expand Down Expand Up @@ -513,6 +519,31 @@ declare namespace Deno {
>;
}

/**
* **UNSTABLE**: Unsafe and new API, beware!
*
* An unsafe function pointer for passing JavaScript functions
* as C function pointers to ffi calls.
*
* The function pointer remains valid until the `close()` method is called.
*/
export class UnsafeCallback<Fn extends ForeignFunction> {
constructor(
definition: Fn,
callback: (
...args: StaticForeignFunctionParameters<Fn["parameters"]>
) => StaticForeignFunctionResult<Fn["result"]>,
);

pointer: UnsafePointer;
definition: Fn;
callback: (
...args: StaticForeignFunctionParameters<Fn["parameters"]>
) => StaticForeignFunctionResult<Fn["result"]>;

close(): void;
}

/** A dynamic library resource */
export interface DynamicLibrary<S extends ForeignLibraryInterface> {
/** All of the registered library along with functions for calling them */
Expand Down Expand Up @@ -848,12 +879,8 @@ declare namespace Deno {
* ```
*
* Requires `allow-net` permission for "tcp" and `allow-read` for "unix". */
export function connect(
options: ConnectOptions,
): Promise<TcpConn>;
export function connect(
options: UnixConnectOptions,
): Promise<UnixConn>;
export function connect(options: ConnectOptions): Promise<TcpConn>;
export function connect(options: UnixConnectOptions): Promise<UnixConn>;

export interface ConnectTlsOptions {
/** PEM formatted client certificate chain. */
Expand Down
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_10.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_i16", [0, 0]);
Deno.core.opSync("op_ffi_read_i16", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_11.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_u32", [0, 0]);
Deno.core.opSync("op_ffi_read_u32", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_12.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_i32", [0, 0]);
Deno.core.opSync("op_ffi_read_i32", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_13.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_u64", [0, 0]);
Deno.core.opSync("op_ffi_read_u64", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_14.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_f32", [0, 0]);
Deno.core.opSync("op_ffi_read_f32", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_15.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_f64", [0, 0]);
Deno.core.opSync("op_ffi_read_f64", 0n);
13 changes: 4 additions & 9 deletions cli/tests/testdata/unstable_ffi_2.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
Deno.core.opSync("op_ffi_call_ptr", {
pointer: [0, 0],
def: {
name: null,
parameters: [],
result: "void",
},
Deno.core.opSync("op_ffi_call_ptr", 0n, {
name: null,
parameters: [],
buffers: [],
});
result: "void",
}, []);
13 changes: 4 additions & 9 deletions cli/tests/testdata/unstable_ffi_3.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
Deno.core.opAsync("op_ffi_call_ptr_nonblocking", {
pointer: [0, 0],
def: {
name: null,
parameters: [],
result: "void",
},
Deno.core.opAsync("op_ffi_call_ptr_nonblocking", 0n, {
name: null,
parameters: [],
buffers: [],
});
result: "void",
}, []);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_5.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_buf_copy_into", [[0, 0], new Uint8Array(0), 0]);
Deno.core.opSync("op_ffi_buf_copy_into", 0n, new Uint8Array(0), 0);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_6.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_cstr_read", [0, 0]);
Deno.core.opSync("op_ffi_cstr_read", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_7.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_u8", [0, 0]);
Deno.core.opSync("op_ffi_read_u8", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_8.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_i8", [0, 0]);
Deno.core.opSync("op_ffi_read_i8", 0n);
2 changes: 1 addition & 1 deletion cli/tests/testdata/unstable_ffi_9.js
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Deno.core.opSync("op_ffi_read_u16", [0, 0]);
Deno.core.opSync("op_ffi_read_u16", 0n);
Loading