Merge pull request #99 from Taaitaaiger/opaque-with-params

Many changes, see changelog for more information

.cargo/config

@@ -1,2 +1,23 @@
 [registries.crates-io]
 protocol = "sparse"
+
+[target.x86_64-unknown-linux-gnu]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]
+
+[target.i686-unknown-linux-gnu]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]
+
+[target.aarch64-unknown-linux-musl]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]
+
+[target.arm-unknown-linux-musleabihf]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]
+
+[target.armv7-unknown-linux-musleabihf]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]
+
+[target.i686-unknown-linux-musl]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]
+
+[target.x86_64-unknown-linux-musl]
+rustflags = [ "-C", "link-args=-Wl,-export-dynamic" ]

.gitignore

@@ -22,4 +22,5 @@ matplotlibrc
 releasenotes.md
 ARCHITECTURE.md
 playground
-libjulia_module_test.so
+libjulia_module_test.so
+expanded.rs

CHANGELOG.md

@@ -1,3 +1,52 @@
+#### v0.19
+
+- A GC-safe `GcSafeRwLock`, `GcSafeMutex`, `GcSafeFairMutex`, and `GcSafeOnceLock` have been added. These synchronization primitives allow for garbage to be collected while waiting for access to be granted.
+
+- The `ConstructType` trait has gained an associated type and constant, and two new trait methods. The associated `Static` type provides a unique, static Rust type for the type constructor, which must be `Self` with static lifetimes. The associated constant `CACHEABLE` indicates whether the constructed type should be cached if it's a leaf type. The `construct_type` method uses the cached entry if it's available, or construct and cache it when supported otherwise. The `construct_type_uncached` method always constructs the type without checking if a cached entry exists. The `type_id` method returns the type id of the associated `Static` type, which is used internally as a key for the cache.
+
+- `Value`s can be instantiated from a layout and a type constructor with `Value::try_new_with`. This method copies the provided layout from Rust to Julia if the provided type constructor is compatible with the provided layout. This method is also available for `TypedValue`.
+
+- The `ValidLayout` trait has gained a `type_object` method. This method must return the Julia type object associated with this type. Its main use is correctly handling unions in `Value::try_new_with`.
+
+- The `ParametricBase` and `ParametricVariant` traits have been added which let you create opaque types with type parameters.
+
+- Opaque types, functions, methods and async functions with type parameters can be exported by iterating over an array of types with a `for` loop in `julia_module`. These loops can be nested, types provided by outer loops can be used in inner loops. Duplicate implentations are filtered as long as the item is only exported once.
+
+- Constantly-sized frames can be allocated on the stack by creating a local scope, all targets allow creating a new local scope. This construct is used for all functions that used to take an `ExtendedTarget`, they now only need a `Target`. This involves three new targets: `LocalGcFrame`, `LocalOutput`, and `LocalReusableSlot`. Trying to reserve a new slot when the frame is full causes a `panic`.
+
+- Functions and methods exported by the `julia_module` macro no longer need to return a `RustResultRet` to throw an exception. The exception can now be thrown directly from Rust. The return type of the exported function no longer needs to account for exceptions if they can only occur due to tracking `self`. If a function should throw an exception, the function can return either a `JlrsResult<RetTy>` or a `Result<RetTy, ValueRet>`. Such a result will either be converted to the output type or thrown as an exception.
+
+- The `c-unwind` feature has been added. When this feature is enabled, all C functions use the `C-unwind` ABI rather than the `C` ABI.
+
+- `Symbol`s and constructed types are cached. Accessing a global in a module can also be cached with `Module::typed_global_cached`.
+
+- Exceptions can be caught by calling `catch_exceptions`.
+
+- Exported functions and methods can be declared to be GC-safe with the `#[gc_safe]` attribute.
+
+- Exported methods that take `self` can skip tracking with the `#[untracked_self]` attribute.
+
+- Types that have no pointer fields can implement `IsBits`. This trait is automatically derived if appropriate by `JlrsCore.Reflect.reflect`.
+
+- Type constructors can be associated with their layout by implementing `HasLayout`. This trait is automatically derived if appropriate by `JlrsCore.Reflect.reflect`, or implemented by blanket implementation if the type constructor and layout are the same type.
+
+- `Value::new_named_tuple` takes a sized array of key-value pairs to avoid heap-allocations.
+
+- A `DimsExt` trait has been added to optimize creating new arrays. While `Dims` can be implemented by crates that use jlrs, `DimsExt` is sealed.
+
+- When a runtime feature is enabled fast TLS is enabled. Crates like rustfft-jl that don't embed Julia must not enable any runtime features.
+
+- Exported functions and methods that should throw can do so by returning a `JlrsResult` or `Result<_, ValueRet>`. `RustResult` has been deprecated in favor of returning one of these types.
+
+- Some functions need to insert additional arguments before the provided arguments. Examples include calling functions with keyword arguments and functions that call a Julia function asynchronously. These functions now take their arguments as an implementation of `Values`, which can add the extra arguments without having to allocate space for them on the heap if the number of arguments is known at compile-time.
+
+- `Call::call_unchecked` has been added to call a Julia function without catching the exception if one is thrown.
+
+- Type aliases can be defined with the `julia_module` macro.
+
+- The `full-no-rt` feature has been added to allow selecting all features except runtimes.
+
+
 #### v0.18
 
  - jlrs is compatible with Julia 1.7 again, but this version isn't actively tested or supported. Version features have been added to select a particular version of Julia, picking a specific version is required.

Cargo.toml

@@ -1,4 +1,5 @@
 
 [workspace]
+# members = ["benches", "examples", "jl_sys", "jlrs", "jlrs_macros", "julia_module_test"]
 members = ["examples", "jl_sys", "jlrs", "jlrs_macros"]
-exclude = ["julia_module_test"]
+exclude = ["julia_module_test", "benches"]

README.md

@@ -6,7 +6,7 @@
 jlrs is a crate that provides access to most of the Julia C API, it can be used to embed Julia
 in Rust applications and to use functionality it provides when writing `ccall`able
 functions in Rust. Currently this crate is only tested in combination with Julia 1.6 and 1.9,
-but also supports Julia 1.7 and 1.8. Using the current stable version is highly recommended.
+but also supports Julia 1.7, 1.8 and 1.10. Using the current stable version is highly recommended.
 The minimum supported Rust version is currently 1.65.
 
 The documentation assumes you're already familiar with the Julia and Rust programming
@@ -40,7 +40,7 @@ recently released version.
 ## Prerequisites
 
 Julia must be installed before jlrs can be used, jlrs is compatible with Julia 1.6 up to and
-including Julia 1.9. The JlrsCore package must also have been installed, if this is not the
+including Julia 1.10. The JlrsCore package must also have been installed, if this is not the
 case it will automatically be added when jlrs is initialized by default. jlrs has not been
 tested with juliaup yet on Linux and macOS.
 
@@ -99,6 +99,7 @@ jlrs. The following version features currently exist:
  - `julia-1-7`
  - `julia-1-8`
  - `julia-1-9`
+ - `julia-1-10`
 
 Exactly one version feature must be enabled. If no version is enabled, or multiple are, jl-sys
 will fail to compile.
@@ -112,6 +113,7 @@ julia-1-6 = ["jlrs/julia-1-6"]
 julia-1-7 = ["jlrs/julia-1-7"]
 julia-1-8 = ["jlrs/julia-1-8"]
 julia-1-9 = ["jlrs/julia-1-9"]
+julia-1-10 = ["jlrs/julia-1-10"]
 ```
 
 In this case you must provide this feature when you build or run your crate:
@@ -219,19 +221,20 @@ In addition to these runtimes, the following utility features are available:
   Flag that must be enabled when compiling with BinaryBuilder.
 
 You can enable all features except `debug`, `i686`, `windows`, `no-link` and `yggdrasil` by
-enabling the `full` feature.
+enabling the `full` feature. If you don't want to enable any runtimes either, you can use
+`full-no-rt`.
 
 
 ## Using this crate
 
 If you want to embed Julia in a Rust application, you must enable a runtime and a version
 feature:
 
-`jlrs = {version = "0.18.0", features = ["sync-rt", "julia-1-8"]}`
+`jlrs = {version = "0.19.0", features = ["sync-rt", "julia-1-8"]}`
 
-`jlrs = {version = "0.18.0", features = ["tokio-rt", "julia-1-8"]}`
+`jlrs = {version = "0.19.0", features = ["tokio-rt", "julia-1-8"]}`
 
-`jlrs = {version = "0.18.0", features = ["async-std-rt", "julia-1-8"]}`
+`jlrs = {version = "0.19.0", features = ["async-std-rt", "julia-1-8"]}`
 
 When Julia is embedded in an application, it must be initialized before it can be used. The
 following snippet initializes the sync runtime:
@@ -483,7 +486,7 @@ impl PersistentTask for AccumulatorTask {
         // A `Vec` can be moved from Rust to Julia if the element type
         // implements `IntoJulia`.
         let data = vec![0usize; self.n_values];
-        let array = TypedArray::from_vec(frame.as_extended_target(), data, self.n_values)?
+        let array = TypedArray::from_vec(&mut frame, data, self.n_values)?
             .into_jlrs_result()?;
 
         Ok(AccumulatorTaskState { array, offset: 0 })
@@ -587,6 +590,8 @@ to your crate's `Cargo.toml`. It's also recommended to abort on panic:
 panic = "abort"
 ```
 
+You must not enable any runtime features.
+
 The easiest way to export Rust functions like `call_me` from the previous example is by
 using the `julia_module` macro. The content of the macro is converted to an initialization
 function that can be called from Julia to generate the module.
@@ -622,7 +627,7 @@ function:
 @assert CallMe.call_me(false) == -1
 ```
 
-This macro has many more capabilities than just exporting extern "C" functions, for more
+This macro has many more capabilities than just exporting functions, for more
 information see the [documentation](https://docs.rs/jlrs-macros/latest/jlrs_macros/macro.julia_module.html). A practical example that uses this macro is the
 [rustfft-jl](https://github.com/Taaitaaiger/rustfft-jl) crate, which uses this macro to expose RustFFT to Julia. The recipe for
 BinaryBuilder can be found [here](https://github.com/JuliaPackaging/Yggdrasil/tree/master/R/rustfft).

benches/Cargo.toml

@@ -0,0 +1,51 @@
+[package]
+name = "benches"
+version = "0.0.0"
+publish = false
+edition = "2018"
+
+[dependencies]
+jlrs = { version = "0.19", path = "../jlrs", features = ["full"] }
+
+[dev-dependencies]
+criterion = { version = "0.5", features = ["html_reports"] }
+pprof = { version = "0.12", features = ["flamegraph", "criterion"] }
+
+[profile.bench]
+debug = true
+
+[[bench]]
+name = "call_function"
+harness = false
+
+[[bench]]
+name = "arrays"
+harness = false
+
+[[bench]]
+name = "array_access"
+harness = false
+
+[[bench]]
+name = "frames"
+harness = false
+
+[[bench]]
+name = "module"
+harness = false
+
+[[bench]]
+name = "symbol"
+harness = false
+
+[[bench]]
+name = "track_array"
+harness = false
+
+[[bench]]
+name = "type_construction"
+harness = false
+
+[[bench]]
+name = "value"
+harness = false