-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support defining C-compatible variadic functions in Rust #2137
Conversation
A bit of background: this was inspired by a co-worker who wanted to define a debugging library to intercept certain C calls, for use with |
Having the compiler enforce special lifetime rules for one function sounds really dubious. Either don't enforce lifetimes or make it work with the normal rules - either with a API that takes |
@comex I didn't intend it to be a "special lifetime rule", so much as the compiler simply defining and passing the appropriate lifetime. I don't really think an API based around calling a closure would work very smoothly, but I'd be open to switching to the |
I'd also appreciate feedback from appropriate compiler folks, regarding what solution might make the most sense from a compiler-internals point of view, in terms of preventing the type from outliving the variadic function associated with it. |
People have talked about introducing a special ie. impl<'a> VaList<'a> {
pub unsafe fn start() -> VaList<'fn>
} |
text/0000-variadic.md
Outdated
impl<'a> Drop for VaList<'a>; | ||
|
||
/// The type of arguments extractable from VaList | ||
trait VaArg; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could be an unsafe trait
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
text/0000-variadic.md
Outdated
the raw C types corresponding to the Rust integer and float types above: | ||
|
||
```rust | ||
impl VaArg for c_char; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless these type aliases became concrete types, I don't think the std
and libc
crates need to provide any additional impls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
text/0000-variadic.md
Outdated
let args = VaList::start(); | ||
let x: u8 = args::arg(); | ||
let y: u16 = args::arg(); | ||
let z: u32 = args::arg(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean
let mut args = VaList::start();
let x: u8 = args.arg();
let y: u16 = args.arg();
let z: u32 = args.arg();
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, thank you.
text/0000-variadic.md
Outdated
```rust | ||
let closure = |arg, arg2, ...| { | ||
// implementation | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can't cast a closure to extern "C" fn
, so I don't think supporting variadic closure is necessary in this RFC.
fn main() {
unsafe {
let _x = (|| {}) as extern fn(); //~ ERROR E0605
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Plus, I think extern "C" variadic functions should always be unsafe functions, and closures currently can't be unsafe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't realize this; I'd assumed that RFC 1558 made this possible, but digging through it, it doesn't allow coercing to extern fn
, only to fn
. I filed rust-lang/rust#44291 to allow coercing to extern fn
, but in the meantime, I'll drop this for now and move it to an open for the future. Would have been nice for more convenient callbacks.
The only safe way that comes to mind is requiring the function take a EDIT: there would need to be a way to mark the function as variadic, |
The RFC says that only |
Rust doesn't support passing a closure as an `extern "C" fn`, so these wouldn't serve any purpose yet.
@mark-i-m Fixed now, since libc's types are just type aliases. |
…times This allows the compiler to pass an appropriate lifetime without as much magic.
@eddyb OK, based on your comment, I've tried reworking the RFC so that it accepts a |
text/0000-variadic.md
Outdated
use std::intrinsics::VaArg; | ||
|
||
#[no_mangle] | ||
pub unsafe extern "C" fn func(fixed: u32, args: ...) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be mut args: ...
then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call! Fixed.
text/0000-variadic.md
Outdated
Such a declaration looks like this: | ||
|
||
```rust | ||
pub unsafe extern "C" fn func(arg: T, arg2: T2, args: ...) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For symmetry, I think we should also allow
extern {
fn func(arg: u8, arg2: u16, args: ...);
// ^~~~~
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, for external variadic functions with the current mechanism?
I'm not sure that makes sense to add, since the argument isn't named in C. What would be the advantage of this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@joshtriplett Just for consistency, the args:
has no meaning here 😄. This is really outside of the scope of this RFC, feel free to ignore it.
text/0000-variadic.md
Outdated
|
||
Note that extracting an argument from a `VaList` follows the platform-specific | ||
rules for argument passing and promotion. In particular, many platforms promote | ||
any argument smaller than a C `int` to an `int`. On such platforms, extracting |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is true for all platforms, it's required by the C standard.
Basically, any integral type smaller than an int
is converted to int
(and any floating point type smaller than a double
(basically float
or short float
, for implementations that provide the latter, is converted to double
)).
Side note, this paragraph points out something important:
emphasis on class type
and enumeration
- this means that it should be valid for programs to impl VaArg
for their #[repr(C|uN|iN)]
structure and simple enumeration types.
I'd also argue that VaArg
should be implemented for &T
, &mut T
, Option<&T>
, and Option<&mut T>
. This is all unsafe anyways :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll phrase the bit about promotions more carefully, and put some thought into the structure and enum cases.
I'd prefer not to implement VaArg
for references; there are plenty of ways to convert a pointer to a reference in unsafe code, if you really want a reference, and those ways allow you to inject the appropriate lifetime of that reference more easily.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer not treating references specially - they are just pointers after all. And it just makes sense sometimes - if you're implementing printf
:
#[no_mangle]
unsafe extern "C" printf(fmt: &CStr, args: ...) -> c_int {
for ch in fmt {
// ...
// found a %s:
{
print!("{}", args.next::<&CStr>());
}
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Except &CStr
is a fat pointer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jethrogb for now ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ubsan If, at some point in the future, there's a reasonable possibility of extracting a reference directly from a variadic function argument and getting a reasonable result, then we can add such a mechanism at that time. Until then, though, I'd like to stick with only allowing raw pointers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ubsan I clarified the RFC's details about promotion to take your explanation into account, and to avoid suggesting "platform-specific".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@joshtriplett I disagree with the reasoning there. There is no reason not to allow references. You almost never actually want raw pointers when dealing with C; you want Option<&T>
(and given that references are valid in C functions...)
text/0000-variadic.md
Outdated
|
||
impl<'a> VaList<'a> { | ||
/// Extract the next argument from the argument list. | ||
pub unsafe fn arg<T: VaArg>(&mut self) -> T; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't like this name - I'd prefer next
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bikeshed: next_unchecked
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think arg
was chosen to match the C name: va_arg
... If we are going to deviate from this, we could change the interface altogether...
fn my_vararg_fn(args: VaList<c_int>) {...}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but you can pass any type through variable arguments, not just one type.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was chosen to match va_arg
, yes. I intentionally didn't use next
, because I don't want to give the impression that this works anything like an iterator, rather than a razor-sharp ball of unsafety.
text/0000-variadic.md
Outdated
impl<'a> Drop for VaList<'a>; | ||
|
||
/// The type of arguments extractable from VaList | ||
unsafe trait VaArg; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's the point of the VaArg
trait anyways? No Rust types should be invalid to pass/take in variadic functions (although perhaps non-#[repr(C)]
types shouldn't be valid? there's no technical reason though).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 for replacing the trait with a #[repr(C)]
check.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ubsan I don't think it makes sense to accept arbitrary Rust types, and in particular not references, or arbitrary structures. But accepting #[repr(C)]
structures makes sense (that's possible in C), as does accepting C-style enums with a declared size.
@joshtriplett Not bad, I don't have any more suggestions at this time, as I believe that solves the problem of the short-lived data escaping the function. |
text/0000-variadic.md
Outdated
[reference-level-explanation]: #reference-level-explanation | ||
|
||
LLVM already provides a set of intrinsics, implementing `va_start`, `va_arg`, | ||
`va_end`, and `va_copy`. The implementation of `VaList::start` will call the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you've changed the RFC to args: ...
, the VaList::start
function should no longer be mentioned. Just say va_start
will be automatically inserted at the start of the function.
text/0000-variadic.md
Outdated
name along with the `...`: | ||
|
||
```rust | ||
pub unsafe extern "C" fn func(fixed: u32, ...args) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VaList::start
should now be an alternative itself. This is at best an "alternative syntax".
This is getting long, so I write it as the main comment here. You mentioned
This will conflict with unsafe extern fn forward_to_printf(fmt: *const c_char, ap: ...) -> c_int {
vprintf(fmt, ap)
} Here int forward_to_printf(const char* fmt, ...) {
va_list ap;
va_start(ap, fmt);
vprintf(fmt, ap);
va_end(ap);
} That means impl VaList {
unsafe fn as_va_list(&mut self) -> va_list;
} so the caller still retains the ownership of unsafe extern fn forward_to_printf(fmt: *const c_char, ap: ...) -> c_int {
vprintf(fmt, ap.as_va_list())
} (Some non-standard stuff)Uniquely-borrowing the VaList instead of moving it will allow you to translate weird things like this into Rust: void just_print_one(va_list ap) {
printf("%d\n", va_arg(ap, int));
}
void print_nums(size_t n, ...) {
va_list ap;
va_start(ap, n);
for (size_t i = 0; i < n; ++ i) {
just_print_one(ap);
}
va_end(ap);
}
int main() {
print_nums(4, 1, 2, 3, 4);
} Note that this violates the C standard §7.16/3, so it is actually invalid. But this works on my machine™.
Actually, would it be even better if |
This RFC has been merged! The tracking issue keeps the syntax explicitly opened as a not-completely-resolved question; further discussion on that point should move over to the tracking issue. Thanks @joshtriplett! |
The RFC as accepted proposes putting (I'd like to avoid bikeshedding this if possible. It definitely has to go in |
@joshtriplett that seems reasonable, although I want to make sure - it's still on unstable, right?
|
@ubsan It's still going to be unstable until stabilized, yes. The issue was that it'd be non-trivial to ever stabilize bits of Also, I think |
@ubsan The |
@joshtriplett right, that makes sense. We really don't want to stabilize bits of |
I've started implementing variadic functions in rust, but some people have suggested changing the syntax to |
@dlrobertson Writing Could you explain more about the complexity that would create? |
@joshtriplett I have git branches with each implemented, just not generating the
I'm not sure what is meant by this? The fn test(fixed: i32, ap: ...VaList) -> VaList { ap }
// or even
fn test<'a>(fixed: i32, ap: ...VaList<'a>) -> VaList<'a> { ap }
The complexity is just a side point and mainly has to do with the internals of parsing in the compiler. Either way we've converged after typeck.
Adding a new |
I believe the issue is that you're writing a type that has a lifetime parameter, except it's not possible to write the correct lifetime as there is no name for it. By writing |
Good point. That could indeed be confusing. For now I'll continue with just |
On December 10, 2018 9:04:00 PM PST, Dan Robertson ***@***.***> wrote:
@joshtriplett I have git branches with each implemented, just not
generating the `va_start`/`va_end` yet. My main concern is the
diverging syntaxes for variadic generics and variadic arguments.
That's a feature. This is specific to extern C functions, and Rust type-safe variadics of any kind can and should be different.
|
I also see it as a good thing that C variadics get a syntax which is similar enough to variadic generics to be unsurprising, but still distinct enough that it's hard to get confused about which feature you're looking at. Because they are two very different and fundamentally incompatible features. |
This is easily solved by checking that the lifetime parameter of It needs to be an "universally quantified" lifetime anyway, just like @dlrobertson got a bit into the weeds of the implementation, but this is my perspective on why I think it's better for the user to write out With e.g. We could even rename (Also, a random note: |
That seems like revisiting the same argument that was made and explored during the RFC. Is there a new argument here that wasn't previously explored? |
Added a comment on rust-lang/rust#57760, but realized I should probably post here too. We need a library feature-gate for
|
@dlrobertson Sounds good to me, go for it. :) |
Just FYI @varkor fixed the issue with feature-gates so the |
Support defining C-compatible variadic functions in Rust, via new intrinsics.
Rust currently supports declaring external variadic functions and calling them
from unsafe code, but does not support writing such functions directly in Rust.
Adding such support will allow Rust to replace a larger variety of C libraries,
avoid requiring C stubs and error-prone reimplementation of platform-specific
code, improve incremental translation of C codebases to Rust, and allow
implementation of variadic callbacks.
This RFC does not propose an interface intended for native Rust code to pass
variable numbers of arguments to a native Rust function, nor an interface that
provides any kind of type safety. This proposal exists primarily to allow Rust
to provide interfaces callable from C code.
Rendered