Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serializable Lambda expression #1022

Open
hyunsik opened this issue Mar 27, 2015 · 8 comments
Open

Serializable Lambda expression #1022

hyunsik opened this issue Mar 27, 2015 · 8 comments
Labels
T-lang Relevant to the language team, which will review and decide on the RFC. T-libs-api Relevant to the library API team, which will review and decide on the RFC.

Comments

@hyunsik
Copy link

hyunsik commented Mar 27, 2015

Rust basically is very strong for parallel programming. If Rust supports Serializable Lambda expression, it would be very strong even for distributed programming.

Here was a little discussion.
rust-lang/rust#23759

Python and Java 8 already have similar concepts as follows:

@kmcallister
Copy link
Contributor

See also #668.

@Rufflewind
Copy link

Rufflewind commented Mar 13, 2017

One could do add some compiler magic to support arbitrary object serialization in Rust, with the caveat that such an object can only be deserialized by the same program. This would be like Any with persistence. The Haskell recently added support for these kinds of “static pointers”.

But the downside of this is that the user has no control over the serialization protocol, and it would require making a serializer that works over arbitrarily complicated types, even though much of it could be done outside of the standard library.

A more flexible approach would be to expose two new mechanisms to Rust:

  • The ability to have an extensible lookup table for a selection of types, a “global type registry” if you will. This I think is the main missing ingredient for Encodable trait objects #668.

  • The ability to construct and deconstruct closures into a tuple of its upvars (Closure::Data). Typically, Closure::Data would be serializable through a #[derive] mechanism.

    pub trait Closure {
        type Data;
        fn construct(data: Self::Data) -> Self;
        fn deconstruct(self) -> Self::Data;
    }
    
    impl Closure for /* all unboxed closures */ {
        type Data = (/* upvars ... */);
        /* ... */
    }
    
    impl Closure for /* all fn(_) -> _ */ {
        type Data = ();
        /* ... */
    }

Edit: no need store a separate function pointer since the closure type itself would suffice.

@Centril Centril added T-lang Relevant to the language team, which will review and decide on the RFC. T-libs-api Relevant to the library API team, which will review and decide on the RFC. labels Feb 23, 2018
@ariesdevil
Copy link

Any updates on it? With write spark like system in rust, Serializable Lambda expression is needed.

@habnabit
Copy link

habnabit commented Jun 26, 2018

(python's pickle does not serialize functions' code to a wire format; it only serializes the name of a function.)

@ariesdevil
Copy link

ariesdevil commented Jun 26, 2018

@habnabit but you can implement it easier,see https://github.com/douban/dpark/blob/0.5.0/dpark/serialize.py#L250

@alecmocatta
Copy link

I created a crate serde_closure that makes closures serializable by wrapping them with the macro Fn!(...).

If the resulting serializable closure is upcast to Box<dyn serde_traitobject::Fn()> it can then be serialized and sent between identical distributed processes using serde_traitobject, though this currently requires nightly.

@Rufflewind
Copy link

Interesting. I had tried a similar idea a while back but deemed it too questionable for production use.

@alecmocatta
Copy link

@Rufflewind Yes, thanks for your crate, it was good inspiration for serde_traitobject.

I just made a rust PR that would make serde_traitobject safe: rust-lang/rust#66113. Keen to hear if you have any thoughts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-lang Relevant to the language team, which will review and decide on the RFC. T-libs-api Relevant to the library API team, which will review and decide on the RFC.
Projects
None yet
Development

No branches or pull requests

9 participants