Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libdbus-sys: consider a Rust-native implementation #100

Open
lucab opened this issue Aug 24, 2017 · 24 comments
Open

libdbus-sys: consider a Rust-native implementation #100

lucab opened this issue Aug 24, 2017 · 24 comments

Comments

@lucab
Copy link
Contributor

lucab commented Aug 24, 2017

This is a feature request to eventually get rid of the C libdbus part and make this a native Rust crate.

Context and usecase for this is that I have some portable/containerized applications which need to talk over dbus. The goal would be for those applications to be static binaries only depending on rust/musl toolchain to build, and not depending on external libs to run. However this currently requires libdbus library at build- and run-time, which in turn depends on a longer list of other C libraries dependencies (see ldd libdbus.so).

@diwic
Copy link
Owner

diwic commented Aug 24, 2017

I consider it regurlarly, i e, every time I get frustrated over DBus library's internals ;-)

Last time it was discussed was here: #85 (comment)

@lucab
Copy link
Contributor Author

lucab commented Aug 25, 2017

@diwic ah, good to know. Perhaps you may want to write down a short doc about what's the design you would expect for such a library, or perhaps stub down an interface, either here or in some experimental repo ad-hoc. I don't have enough dbus-internals knowledge to design/architect it myself, but perhaps I can contribute to implementation. Not sure if @albel727 is interested too.

@diwic
Copy link
Owner

diwic commented Aug 26, 2017

@lucab You might want to look at https://crates.io/crates/dbus-bytestream

@Arnavion
Copy link

Arnavion commented Jan 2, 2020

I started https://github.com/Arnavion/dbus-pure for my own projects. It has the D-Bus type system, and a raw Connection type with send(header, body) and recv() -> (header, body) that implements the binary protocol using serde.

Let me know if you want to use it.

@diwic
Copy link
Owner

diwic commented Jan 2, 2020

@Arnavion cool!

As for serde, last time I checked, I could not use it because it couldn't handle empty arrays correctly - i e, when you serialize an empty array, there is no way to figure out what type that array should be. Is that not a problem with recent serde versions?

@Arnavion
Copy link

Arnavion commented Jan 2, 2020

@Arnavion
Copy link

Arnavion commented Jan 3, 2020

To clarify, when I say that dbus-pure uses serde, I don't mean that it exposes a Deserializer which the caller can use to deserialize message bodies as arbitrary structs. Similarly, the crate does not expose a Serializer that the caller can use to serialize arbitrary structs as message bodies.

The crate internally uses a serde::Deserializer / serde::Serializer, but the public Connection::send and Connection::recv API are in terms of the MessageHeader type for the message header, and the Variant type for the message body. To send an arbitrary struct as a message body, the user will have to first convert it to Variant (*). Similarly, when they receive a message, they'll have to convert the Variant into their arbitrary struct (**). So it's not a problem if the user has a Vec<T> that they want to serialize but the Vec is empty - they have to convert it to a Variant::Array anyway, and at that point they have to set what the element's signature is.

Eg https://github.com/Arnavion/dbus-pure/blob/395d1d6332ad5883a4be9be0c3dc25d5b46cfd62/examples/mpris_playback_status.rs#L55-L71

Of course, there can be a wrapper on top of raw Variant API where the user sees normal Rust code like:

trait MprisPlayer {
    fn playback_status(&mut self) -> String;
}

Then you can have a code generator macro that emits something like:

struct MprisPlayer<'a> {
    client: &'a Client,
    destination: String,
    object_path: String,
}

impl MprisPlayer<'_> {
    fn playback_status(&mut self) -> Result<String> {
        let (mut request_header, request_body) = /* Build a Variant::Tuple for the message body, like in that example */;
        let (response_header, response_body) = self.client.method_call(&mut request_header, &request_body)?;
        response_body.into_string() // Fallibly parse the response Variant as a Variant::String and return the inner String
    }
}

Similarly for a &[&str] parameter, the impl code would have the compile-time type information to know that it has to construct a Variant::Array { element_signature: Signature::String, elements: ... } regardless of whether the slice is empty at runtime or not.

But this kind of generated code is at a higher level of abstraction than Connection and Variant. What dbus_pure provides are the building blocks that such a macro would expand to.


(*) : Variant uses Cow for everything, so it can be usually be created using borrows of the original Rust type instead of copying.

(**) : I'll probably implement serde::Deserializer on Variant so that it's easier to parse into an arbitrary struct, just like serde_json::Value does.

@diwic
Copy link
Owner

diwic commented Jan 3, 2020

@Arnavion Thanks for the explanation!

So I had some more look at your code. I'm open for optionally depending on dbus-pure, here are my current hesitations/thoughts:

  • dbus-pure looks like a very young project, only a few weeks old and not on crates.io. (The dbus crate is IIRC four years old.) What are your future plans for the crate?
  • There are functions in the C library that you currently don't implement (I think?), e g checking if a string is a valid object path or getting the machine UUID.
  • My own time is currently limited and there are other projects in the pipeline (e g dbus-crossroads). How open are you to helping out with the implementation?

An implementation should be as simple as setting a feature flag on the dbus crate and recompile, no code changes should be necessary (for the people using the dbus crate). At least the "main" stuff should be the same (like, commonly used methods on Connection, Message, the arg module).

@Arnavion
Copy link

Arnavion commented Jan 3, 2020

  • dbus-pure looks like a very young project, only a few weeks old and not on crates.io. (The dbus crate is IIRC four years old.) What are your future plans for the crate?

I need a D-Bus client that doesn't depend on the C library for my own use. So I'm going to develop and maintain it for that purpose, including publishing to crates.io when I've ironed out the obvious bugs - serialization of empty arrays was broken until last night due to missing pre-element padding, connecting to the system bus only got added last night, env var parsing is still just slicing the string instead of properly parsing as a URL, etc.

Currently I'm using the raw Connection interface that I wrote about above, not this crate, and that will definitely hold as long as this crate has a hard dependency on the C library.

  • There are functions in the C library that you currently don't implement (I think?),

Almost certainly. I have no idea what the C library offers. I want to make a clean-room implementation just from the spec so that it doesn't have to be GPL, so I've only looked at the spec, not the C library. So the minimum I will guarantee is that the type system and binary protocol are implemented correctly, and anything else mentioned in the spec.

e g checking if a string is a valid object path

I'm curious why a client needs that. That sounds like something a message bus implementation itself would need. Or is this just for convenience in the ObjectPath newtype's constructor?

Anyway, yes, some validation is missing that I plan to add. The more important one is for validity of signatures and containers. Currently it's possible to construct invalid Variants, like DictEntrys with non-simple keys like Array, and anything involving Tuple that isn't at the top-level like Array(Tuple(...)).

or getting the machine UUID.

That is obtained by calling the org.freedesktop.DBus.Peer.GetMachineId method, so it doesn't look like it needs to be a dedicated function in the library.

  • My own time is currently limited and there are other projects in the pipeline (e g dbus-crossroads). How open are you to helping out with the implementation?

I haven't looked at this crate at all yet - after I saw it depended on the C library I stopped looking at it. I'll go over this crate's docs over the weekend.

An implementation should be as simple as setting a feature flag on the dbus crate and recompile, no code changes should be necessary (for the people using the dbus crate). At least the "main" stuff should be the same (like, commonly used methods on Connection, Message, the arg module).

That is likely to be impossible / more effort than I care to make, but I'll know for sure after the weekend.

I want to emphasize that my personal need is already satisfied by dbus-pure's level of abstraction, so making this crate use dbus-pure as a backend is not a priority for me. For example, from a quick glance the only thing that this crate offers which I could want is the Get trait for deserializing message bodies to Rust types, but as I wrote in the last comment I'm planning to solve that by impling serde::Deserializer on Variant instead.

@diwic
Copy link
Owner

diwic commented Jan 3, 2020

  • dbus-pure looks like a very young project, only a few weeks old and not on crates.io. (The dbus crate is IIRC four years old.) What are your future plans for the crate?

I need a D-Bus client that doesn't depend on the C library for my own use. So I'm going to develop and maintain it for that purpose, including publishing to crates.io when I've ironed out the obvious bugs - serialization of empty arrays was broken until last night due to missing pre-element padding, connecting to the system bus only got added last night, env var parsing is still just slicing the string instead of properly parsing as a URL, etc.

Currently I'm using the raw Connection interface that I wrote about above, not this crate, and that will definitely hold as long as this crate has a hard dependency on the C library.

  • There are functions in the C library that you currently don't implement (I think?),

Almost certainly. I have no idea what the C library offers. I want to make a clean-room implementation just from the spec so that it doesn't have to be GPL, so I've only looked at the spec, not the C library. So the minimum I will guarantee is that the type system and binary protocol are implemented correctly, and anything else mentioned in the spec.

The C library is dual licensed under GPL and AFL, both allow you to study the source code.

e g checking if a string is a valid object path

I'm curious why a client needs that. That sounds like something a message bus implementation itself would need. Or is this just for convenience in the ObjectPath newtype's constructor?

The latter, mostly. It's a matter of being a well behaved client (or server) - I don't want the users of my dbus crate to be able to send invalid ObjectPaths on the bus.

Anyway, yes, some validation is missing that I plan to add. The more important one is for validity of signatures and containers. Currently it's possible to construct invalid Variants, like DictEntrys with non-simple keys like Array, and anything involving Tuple that isn't at the top-level like Array(Tuple(...)).

or getting the machine UUID.

That is obtained by calling the org.freedesktop.DBus.Peer.GetMachineId method, so it doesn't look like it needs to be a dedicated function in the library.

A well behaved server should implement org.freedesktop.DBus.Peer on all its object paths. That's how I understand the spec, at least. And my dbus library supports both clients and servers.

  • My own time is currently limited and there are other projects in the pipeline (e g dbus-crossroads). How open are you to helping out with the implementation?

I haven't looked at this crate at all yet - after I saw it depended on the C library I stopped looking at it. I'll go over this crate's docs over the weekend.

An implementation should be as simple as setting a feature flag on the dbus crate and recompile, no code changes should be necessary (for the people using the dbus crate). At least the "main" stuff should be the same (like, commonly used methods on Connection, Message, the arg module).

That is likely to be impossible / more effort than I care to make, but I'll know for sure after the weekend.

I want to emphasize that my personal need is already satisfied by dbus-pure's level of abstraction, so making this crate use dbus-pure as a backend is not a priority for me. For example, from a quick glance the only thing that this crate offers which I could want is the Get trait for deserializing message bodies to Rust types, but as I wrote in the last comment I'm planning to solve that by impling serde::Deserializer on Variant instead.

Okay, thanks for clarifying. Maybe someone else will carry on the torch and try to write some glue between dbus-pure and the dbus crate, if neither us have the time or effort necessary to complete it right now.

@Arnavion
Copy link

Arnavion commented Jan 3, 2020

The C library is dual licensed under GPL and AFL, both allow you to study the source code.

I'm aware. This is not acceptable reasoning for me. It's non-trivial to argue that writing code based on reading GPL'd code is not a derived work of said GPL'd code. A clean-room implementation is safe(r).

A well behaved server should implement org.freedesktop.DBus.Peer on all its object paths. That's how I understand the spec, at least.

Sure, and it can be implemented by the layer on top of dbus-pure, sourced from the bus's response to org.freedesktop.DBus.Peer.GetMachineId or from another place of the caller's choosing. I don't think it belongs at the dbus-pure layer.

And my dbus library supports both clients and servers.

To be clear, so does dbus-pure, since it does not different between them.

@mathstuf
Copy link
Contributor

mathstuf commented Jan 3, 2020 via email

@diwic
Copy link
Owner

diwic commented Jan 5, 2020

GetMachineId is not a big deal, since it just reads /etc/machine-id anyway. And given that it seems possible to get something up and running in just a few weeks, maybe I would be better off writing something myself? That way I get something that's specifically tailored to the needs of the dbus crate. What do you think?

@mathstuf
Copy link
Contributor

mathstuf commented Jan 6, 2020

That assumes it's using /etc/machine-id (have the non-systemd distros started providing it?). I'd also really like to not have it be Linux-only.

@mathstuf
Copy link
Contributor

mathstuf commented Jan 6, 2020

Also, if you have two crates wanting to work with machine-id numbers and one has to be crafted, it'd be nice if all the internals would agree on the same one at least (even if other processes don't necessarily agree).

@diwic
Copy link
Owner

diwic commented Jan 30, 2020

Oh, and suddenly they just keep popping up like mushrooms after a rainy day!

https://github.com/KillingSpark/rustbus
https://gitlab.freedesktop.org/zeenix/zbus/
https://github.com/Arnavion/dbus-pure
https://github.com/srwalter/dbus-bytestream

...and did I mention I started writing something myself yesterday...

@mathstuf
Copy link
Contributor

dbus-bytestream is considerably older than the rest and is just a marshaller for the bytestream of the protocol itself; it doesn't implement any actual communication logic. I recommend it to any pure Rust impl of D-Bus as a place to start.

@KillingSpark
Copy link

Oh, and suddenly they just keep popping up like mushrooms after a rainy day!

https://github.com/KillingSpark/rustbus
https://gitlab.freedesktop.org/zeenix/zbus/
https://github.com/Arnavion/dbus-pure
https://github.com/srwalter/dbus-bytestream

I started a benchmarking repo for those here

@landhb
Copy link
Contributor

landhb commented Dec 24, 2022

It isn't as ideal as a pure-rust implementation, but I've started a draft PR to add a vendoring option, which will reduce the build time dependencies.

#408

@Altair-Bueno
Copy link

I don't think a Rust native implementation is sensible. See zbus, for example. It struggles with binary size, making it completely unusable in IoT Linux. Specially when the same device might have multiple (different) processes connected through dbus. A Rust rewrite wouldn't allow for libdbus.so memory pages to be shared between them. Shared libraries are still important and actively being used by the industry!

@KillingSpark
Copy link

A Rust rewrite wouldn't allow for libdbus.so memory pages to be shared between them. Shared libraries are still important and actively being used by the industry!

That's not true. Rust libaries can be built to shared libraries. A prominent example is the rewrite of libsvg to librsvg

@Altair-Bueno
Copy link

Altair-Bueno commented Nov 30, 2024

That's not true. Rust libaries can be built to shared libraries. A prominent example is the rewrite of libsvg to librsvg

My comment was about reusing already system-wide installed and available libraries that ship with everyone's Linux. Is not about linkage, is about using what is already there and other programs expect. You need to take into account not everyone is going to switch the existing "libdbus" library to "librdbus" just to use the dbus crate. Heck, they wouldn't even consider that option if there isn't a Yocto recipe or similar.

Of course, you could still ship "librdbus" for Rust programs to share, but everything else (systemd, wpa_supplicant, bluez, avahi,...) wouldn't use it.

That's not true. Rust libaries can be built to shared libraries

quick annotation: You can make shared libraries in rust using C ABI or something like stabby. Rust's ABI is not stable. Your comment does not state which kind of shared library you are referring to.

@diwic
Copy link
Owner

diwic commented Dec 1, 2024

I don't think a Rust native implementation is sensible. See zbus, for example. It dbus2/zbus#304, making it completely unusable in IoT Linux.

Having read through that issue quickly, I don't think the binary size is so much related to whether or not you use libdbus as it is to how you make use of generics when you do the Rust code.

I remember a PR that would significantly increase code size to these bindings some time ago, that had to end up with a different solution. Not sure what the code size is now, but let me know if it increases to a point that it causes a problem for you.

@Altair-Bueno
Copy link

Not sure what the code size is now, but let me know if it increases to a point that it causes a problem for you.

Setting release to the following configuration

[profile.release]
lto = true
debug = false
panic = "abort"
opt-level = "z"
strip = true

With something like this:

use std::error::Error;
use std::thread::sleep;
use std::time::Duration;

use wpa_supplicant1;
use wpa_supplicant1::bsss::FiW1WpaSupplicant1BSS;
use wpa_supplicant1::interfaces::FiW1WpaSupplicant1Interface;
use wpa_supplicant1::FiW1WpaSupplicant1;
use dbus::arg::PropMap;
use dbus::arg::Variant;
use dbus::blocking::Connection;

fn main() -> Result<(), Box<dyn Error>> {
    let interface_name = std::env::args().nth(1).unwrap_or("wlan0".to_string());
    let timeout = Duration::from_secs(1);

    let connection = Connection::new_system()?;
    let wpa_supplicant1_proxy =
        connection.with_proxy(wpa_supplicant1::BUS_NAME, wpa_supplicant1::PATH, timeout);

    let interface_path = wpa_supplicant1_proxy.get_interface(&interface_name)?;
    println!("Wifi interface is at path: {}", interface_path);

    let interface_proxy = connection.with_proxy(wpa_supplicant1::BUS_NAME, interface_path, timeout);

    println!("Scanning for networks");
    let mut map = PropMap::new();
    map.insert("Type".to_string(), Variant(Box::new("passive".to_string())));
    interface_proxy.scan(map)?;

    sleep(Duration::from_secs(5));

    let bsss = interface_proxy.bsss()?;

    println!("Got {} BSSs", bsss.len());
    for bss_path in bsss {
        println!("BSS: {:?}", bss_path);
        let bss_proxy = connection.with_proxy(wpa_supplicant1::BUS_NAME, bss_path, timeout);

        let ssid = bss_proxy.ssid()?;
        let ssid = String::from_utf8_lossy(&ssid);
        println!("SSID: {:?}", ssid);
    }

    Ok(())
}

Creates a binary of size 359KB for x86-64 linux. A hello world in Rust is 318KB. Binary size for this crate is great.

We did some similar tests with zbus and the binary size was at around 1.2MB with the same configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants