Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Export C API #1113

Closed
j-a-m-l opened this issue Oct 14, 2021 · 7 comments
Closed

Export C API #1113

j-a-m-l opened this issue Oct 14, 2021 · 7 comments
Labels
enhancement New feature or request

Comments

@j-a-m-l
Copy link
Contributor

j-a-m-l commented Oct 14, 2021 •
edited
Loading

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We (me and @kou ) would like to export the C API to allow other language bindings (#1108, #1114).

Describe the solution you'd like
We will use the FFI that allow exposing a C API (https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c).

Describe alternatives you've considered
None, since not all languages have direct bindings/FFI to Rust
@j-a-m-l j-a-m-l added the enhancement New feature or request label Oct 14, 2021
@j-a-m-l j-a-m-l mentioned this issue Oct 14, 2021
Closed
@houqp
Copy link
Member

houqp commented Oct 15, 2021

Pretty cool, let us know if you need any help with it.

@jimexist
Copy link
Member

I'm not sure tokio and async functions can be easily modeled.

@houqp
Copy link
Member

houqp commented Oct 16, 2021

i think we would need to do pull the same trick we have with our python binding.

@dsisnero
Copy link

there are several libraries made in ruby similar to py03 in python
https://github.com/danielpclark/rutie
https://github.com/matsadler/magnus

kou added a commit to kou/arrow-datafusion that referenced this issue May 26, 2022
@kou
Add minimum C API
9ccdb2a 
Closes apache#1113

This exports minimum C API to write the following Rust code in C:

    use datafusion::prelude::*;

    #[tokio::main]
    async fn main() -> datafusion::error::Result<()> {
      // register the table
      let mut ctx = ExecutionContext::new();

      // create a plan to run a SQL query
      let df = ctx.sql("SELECT 1").await?;

      // execute and print results
      df.show().await?;
      Ok(())
    }

See datafusion/c/examples/sql.c for C version. You can build and run
datafusion/c/examples/sql.c by the following command lines:

    cargo build
    cc -o target/debug/sql datafusion/c/examples/sql.c -Idatafusion/c/include -Ltarget/debug -Wl,--rpath=target/debug -ldatafusion_c
    target/debug/sql

This implementation doesn't export Future like
datafusion-python. Async functions are block_on()-ed in exported
API. But I think that we can export Future in follow-up tasks.

Follow-up tasks:

  * Add support for testing by "cargo test"
  * Add support for building and running examples by "cargo ..."
  * Add support for installing datafusion.h
kou added a commit to kou/arrow-datafusion that referenced this issue May 26, 2022
@kou
Add minimum C API
1b439bb 
Closes apache#1113

This exports minimum C API to write the following Rust code in C:

    use datafusion::prelude::*;

    #[tokio::main]
    async fn main() -> datafusion::error::Result<()> {
      // register the table
      let mut ctx = ExecutionContext::new();

      // create a plan to run a SQL query
      let df = ctx.sql("SELECT 1").await?;

      // execute and print results
      df.show().await?;
      Ok(())
    }

See datafusion/c/examples/sql.c for C version. You can build and run
datafusion/c/examples/sql.c by the following command lines:

    $ cargo build
    $ cc -o target/debug/sql datafusion/c/examples/sql.c -Idatafusion/c/include -Ltarget/debug -Wl,--rpath=target/debug -ldatafusion_c
    $ target/debug/sql
    +----------+
    | Int64(1) |
    +----------+
    | 1        |
    +----------+

This implementation doesn't export Future like
datafusion-python. Async functions are block_on()-ed in exported
API. But I think that we can export Future in follow-up tasks.

Follow-up tasks:

  * Add support for testing by "cargo test"
  * Add support for building and running examples by "cargo ..."
  * Add support for installing datafusion.h
@kou kou mentioned this issue May 26, 2022
Closed
kou added a commit to kou/arrow-datafusion that referenced this issue May 26, 2022
@kou
Export minimum C API
4d41586 
Closes apache#1113

This exports minimum C API to write the following Rust code in C:

    use datafusion::prelude::*;

    #[tokio::main]
    async fn main() -> datafusion::error::Result<()> {
      // register the table
      let mut ctx = ExecutionContext::new();

      // create a plan to run a SQL query
      let df = ctx.sql("SELECT 1").await?;

      // execute and print results
      df.show().await?;
      Ok(())
    }

See datafusion/c/examples/sql.c for C version. You can build and run
datafusion/c/examples/sql.c by the following command lines:

    $ cargo build
    $ cc -o target/debug/sql datafusion/c/examples/sql.c -Idatafusion/c/include -Ltarget/debug -Wl,--rpath=target/debug -ldatafusion_c
    $ target/debug/sql
    +----------+
    | Int64(1) |
    +----------+
    | 1        |
    +----------+

This implementation doesn't export Future like
datafusion-python. Async functions are block_on()-ed in exported
API. But I think that we can export Future in follow-up tasks.

Follow-up tasks:

  * Add support for testing by "cargo test"
  * Add support for building and running examples by "cargo ..."
  * Add support for installing datafusion.h
@kou
Copy link
Member

kou commented May 26, 2022

I've implemented minimum C API as the first step: #2622

@alamb
Copy link
Contributor

alamb commented May 31, 2022

There is an excellent comment about "C API as a way to enable embedding across many languages" from @loic-sharma here #2622 (comment) ❤️

@kou
Copy link
Member

kou commented Jun 3, 2022

I close this because we work on this at https://github.com/datafusion-contrib/datafusion-c .

@kou kou closed this as completed Jun 3, 2022
@jychen7 jychen7 mentioned this issue Jul 3, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Export minimum C API and examples for C, Ruby and Python kou/arrow-datafusion
6 participants
@dsisnero @kou @alamb @jimexist @houqp @j-a-m-l