Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need calling convention for stdcall methods (different from functions!) #1342

Open
retep998 opened this issue Oct 28, 2015 · 7 comments
Open
Labels
T-dev-tools Relevant to the development tools team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC.

Comments

@retep998
Copy link
Member

The crux of the matter is that stdcall methods on Windows using msvc handle struct returns differently than stdcall functions. This is particularly noticeable in COM, where although most of COM doesn't use struct returns, a few of them do, causing incorrect behavior when you attempt to call them from Rust (or even C for the matter).

Thus I propose that we add a new calling convention specifically for stdcall methods. When faced with this new calling convention Rust should do what Clang does:

(2:07:35 PM) rnk: right, this is sret vs. this
(2:07:47 PM) rnk: clang handles this by swapping the order of the arguments
(2:07:54 PM) rnk: and moving the sret attribute to the second parameter

Relevant discussion in #LLVM https://gist.github.com/retep998/604861ea06aa984ee6c7
Relevant discussion on discourse https://internals.rust-lang.org/t/need-custom-calling-convention-for-com/2389

gist demonstrating the calling convention difference: https://gist.github.com/retep998/9503145841a61551d3c6

@eefriedman
Copy link
Contributor

extern "stdcall_method" and friends seems reasonable to me.

@mzabaluev
Copy link
Contributor

Notably, the MIDL compiler seems to have no problem annotating __stdcall in the plain-C bindings for COM vtables. If it turns out that on -msvc targets rustc or LLVM does not do what MSVC does for any C function, then it's simply a matter of fixing "stdcall".

@retep998
Copy link
Member Author

MSVC for C actually suffers from the same issue. I tested it myself. Functions in COM that return structs have incorrect behavior when called from the official Microsoft C bindings using the MSVC compiler. I'm assuming nobody at Microsoft noticed or cared about this since very few people use C when working with COM.

@retep998 retep998 mentioned this issue Mar 7, 2016
47 tasks
@DemiMarie
Copy link

What about thiscall? That is what LLVM calls it I believe.

@retep998
Copy link
Member Author

@DemiMarie thiscall is not the same as an stdcall method.

@retep998
Copy link
Member Author

retep998 commented Jul 6, 2017

It turned out that the workaround I use for this in winapi was actually slightly wrong. When the function takes parameters beyond this, those parameters need to go after the return value parameter. This just goes to show how important language support for this is, because it would remove the need to implement hacky workarounds that might be broken.
retep998/winapi-rs@6754bdd

@Bastacyclop
Copy link

Hi everyone, any updates on this? Still not sure how to fix Bastacyclop/rust_box2d#3 . Do I need to transform Struct f(); into void f(Struct* ret); on the C side ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-dev-tools Relevant to the development tools team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC.
Projects
None yet
Development

No branches or pull requests

6 participants