Cross-compiling wasm32-unknown-unknown on 64bit debian #1221
Comments
|
@e0gs wow, this is an interesting project! I think you're the first person I'm aware of trying to use pyo3 to put Python on the web 😄 Thanks for investigating the build script. Sorry that there's been some issues with it. If you've got time to put those fixes in a PR I can review and merge them. I have a few general questions about how this works - you're compiling python from source to wasm? And then you're using wasm-pack to link in against the Python library? I have to confess I have absolutely no knowledge about the current state of linking C libraries from Rust code in the wasm world... |
|
Thought I'd see how far I can get with using youtube-dl via pyo3 and web assembly for a browser extension :) Almost, atm I'm trying with https://github.com/e0gs/cpython-emscripten (added wasm32-wasi cross-compiling) for a python wasm32-wasi build and https://github.com/e0gs/pyo3 and then simply trying to create an example use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
#[no_mangle]
pub extern fn execute() -> i32 {
Python::with_gil(|py| {
main_(py).map_err(|e| {
e.print_and_set_sys_last_vars(py);
})
});
100
}
fn main_(py: Python) -> PyResult<()> {
let sys = py.import("sys")?;
let version: String = sys.get("version")?.extract()?;
let locals = [("os", py.import("os")?)].into_py_dict(py);
let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
let user: String = py.eval(code, None, Some(&locals))?.extract()?;
println!("Hello {}, I'm Python {}", user, version);
Ok(())
}With PYO3_CROSS_LIB_DIR set to cpython-emscripten/installs/python-3.5.2/lib (with a renamed libpython3.5m.a copy in it, cpython-emscripten creates a static libpython3.5.so next to the lib dir) and then simply runnning and execution Though atm ends up with Some helpful resources have been https://stackoverflow.com/questions/44761748/compiling-python-to-webassembly/47739532#47739532 |
|
Wow, very cool 🤤
Both of those contain wasm machine code? I'm totally guessing at random, wondering if that's where the wrong machine type error is coming from. Afraid I'm not going to be much use with that error, though feel free to keep asking questions and I'll try to be any use I can. And naturally as you progress do please consider opening PRs or sending links to blog posts / examples - would be really cool to document this for the community to use! |
|
I have managed to get
The first four lines are PYO3 configuration. The rest of the settings are needed to generate a dynamic wasm library. It currently doesn't work for me due to rust-lang/rust#80685 , but the pyo3 part is working well, and replacing the last two lines with a normal cargo build produces a reasonable-looking I would like to upstream these changes, but I think it is better to gather some input from maintainers first |
|
Thanks for the detailed write-up and for the PRs! It would be nice to get wasm support working - I've put a couple of comments on your PRs. I'm looking forward to trying this myself when I get a chance... |
|
@dalcde I don't know if you mentioned it, but you should be aware that rust doesn't actually use the same ABI as C/C++ on wasm32-unknown-unknown, this can make it crash in weird ways or fail to link, if you want it to work you'd probably want to use wasm32-wasi or wasm32-unknown-emscripten. |
turns out that only wasm32-unknown-emscripten was fixed: |
|
On Wed, Jan 06, 2021 at 09:49:51AM -0800, Jacob Lifshay wrote:
> @dalcde I don't know if you mentioned it, but you should be aware that rust doesn't actually use the same ABI as C/C++ on wasm32-unknown-unknown, this can make it crash in weird ways or fail to link, if you want it to work you'd probably want to use wasm32-wasi or wasm32-unknown-emscripten.
I assumed they would be compatible because both use LLVM, but I guess
I'm wrong. I was running into other difficulties, which is why I marked
this as draft
|
|
See #1522 (comment) |
🐛 Bug Reports
When cross-compiling pyo3s build.rs fails on two places without any pointers on what lines, but rather just "Error: NotPresent",
The guilty lines in build.rs:
https://github.com/PyO3/pyo3/blob/master/build.rs#L157, env::var("CARGO_CFG_TARGET_FAMILY") is not set
https://github.com/PyO3/pyo3/blob/master/build.rs#L433, env::var("CARGO_CFG_TARGET_FAMILY") is not set
After that, a custom python 32bit compilation requires PYO3_CROSS_INCLUDE_DIR which is only set for windows in fn cross_compilings https://github.com/PyO3/pyo3/blob/master/build.rs#L150
And finally, Include/pyconfig.h doesn't exists in https://www.python.org/downloads/release/python-379/
Which is required at https://github.com/PyO3/pyo3/blob/master/build.rs#L414
pyconfig.h gets generated when using ./configure and ends up in the root dir.
Also, when python is configured without --enable-shared and such, options retrieved from config_map crashes instead of defaulting to false, dunno if there's a reason for that. https://github.com/PyO3/pyo3/blob/master/build.rs#L415 but unwrap_or(false) might be better than current solution?
Accidently posted this issue before successfully compiling, will update on progress
update:
Well, seem to have gotten rid of all errors in build.rs, it seems to find all neccessary files and and so on but now I get about 100 errors regarding libc.
Using wasm32-wasi instead pulls the errors down to only 3 regarding wchar_t in libc. Seems to be a problem with libc and wasm32 target from here on out.
🌍 Environment
rustc --version): rustc 1.49.0-nightly (38d911dfc 2020-10-09)version = "0.x.y"withgit = "https://github.com/PyO3/pyo3")?: yupp💥 Reproducing
Please provide a minimal working example. This means both the Rust code and the Python.
Please also write what exact flags are required to reproduce your results.
Compiled Python 3.7.9 with
Setup PYO3_CROSS flags
lib.rs
Cargo.toml
Build command
The text was updated successfully, but these errors were encountered: