UART: Add wrapper around RIOT's UART-interface

[kbarning]

Here is my attempt to create a wrapper for RIOT's UART-interface :)

[chrysn]

Thanks for the PR. I've cleared it for a first CI run, but haven't yet had time to look at it. Given this needs a uart_t at construction, it might be good to look at #37 in parallel.

[chrysn]

Thanks for adding this. It looks good over-all, and very comprehensive. I have several details and questions annotated across the source. Quite a few of them might easily be explained by inspiration from existing code; I'd still hope to not continue on the bad examples I've set in the past.

[chrysn]

This is where we still feel the shock of the leakpocalypse -- sadly, having a &a callback and a Drop implementation isn't good enough. Say I write this code:

fn init() {
    let mut f = || {...};
    let dev = UartDevice::new(0, 8600, &mut f).unwrap();
    core::mem::forget(dev);
}

then when data is received, f will be called even though it has long been deallocated.

There are two (mutually non-exclusive) ways around this:

• Implement .new() on UartDeviceStatus<'static>, thus taking a &'static FnMut.
  • If we wanted to close the door on the second approach, we could remove the PhantomData and the lifetime, but let's not.
• Implement a scoped approach (like scoped threads), where the UART is only active for the duration of a scope (which may easily be a fn(...) -> !), and drop gets called reliably.

I suggest to use the first approach, unless your application needs the second, in which case I'd suggest doing both. (Existing APIs in riot-wrappers often only do the second, but I've come to consider that impractical in many situations).

[kbarning]

I'm not sure if I'm misinformed here, but if I use a &'static mut F as parameter, then it's not possible anymore that the closure captures variables because you have to do something like this:

static mut CB: fn(u8) = |data| println!("New data received: {data}");
let mut uart = UartDevice::new(0, 115200, unsafe { &mut CB }).unwrap();

This is because static can only be function pointers afaik.

I also thought about taking ownership of the closure, but that does't work either because if we take the address of the closure and then return it with the Self, the address of the closure changed. All other wrappers avoid this by putting the closure on the heap e.g. with Box, but we don't have this option here because of the #![no_std] world.

[chrysn]

Yeah, a 'static can not be such a closure -- that's what the scoped API would be for.

It's not only about not being able to Box something away. The thread's initial example would also not be sound if we had Box -- the Box would a Box<dyn FnMut() + 'a>, with the 'a ending when the function returns, but the function would need to demand + 'static to warrant passing it into the unsafe C function. The box would help with the issue of the callback not moving when stored, but as we need the drop issue solved anyway (which is most easily done with the body-runs-in-callback approach), we don't have the moving reference problem.

[kbarning]

I have updated my old solution and have followed the scoped approach. I think this is a good compromise between comfort and safety. As long as the user doesn't get the idea to use something like core::mem::forget, he should be on the safe side. Furthermore I have now implemented an extra new method to inizialize the UART using a static callback.

[chrysn]

Unfortunately, we can't rely on the user to not core::mem::forget when the consequences are unsound (use-after-free).

From the current state (which is good but does not go all the way yet), we'd need to:

• move the safe new function from the impl<'scope> UartDevice<'scope> { ... } to a block that is impl UartDevice<'static> { ... }, because that's the scope-less situation that is sound
• Have a function roughly like this (can be on UartDevice or free, you probably have a test application with which you can try out the ergonomics):

impl UartDevice<'scope> {
    /// ...
    ///
    /// This is the scoped version of [`new()`] that can be used if you want to use short-lived callbacks, such as
    /// closures or anything containing references. The UartDevice is deconfigured when the internal main function
    /// terminates. A common pattern around this kind of scoped functions is that `main` contains the application's
    /// main loop, and never terminates (in which case the clean-up code is eliminated during compilation).
    pub fn new_scoped<CB, M, R>(
        index: usize,
        baud: u32,
        user_callback: &'scope mut F,
        main: M,
    ) -> Result<R, UartDeviceError>
    where
        F: FnMut(u8) + Sync + 'scope,
        M: FnOnce(&mut Self) -> R,
    {
        let self_ = unsafe { ... }?;
        let result = (main)(&mut self_);
        drop(self_);
        result
    }
}

(One could also leave out the drop as it's implied -- what matters is that it happens reliably at this point, and that while the scope can safely forget the &mut, the self_ will be dropped reliably before all the references to 'scope it contains become invalid).

[kbarning]

I thought about the problem for a bit. I still like my version better because in normal rust you can do something like this:

let vec = vec![0; 100];
std::mem::forget(vec);

This creates a memory leak without even using forget. You can do all sorts of things causing bad behavior in safe rust if you really wish. I think atm there is a good compromise between safety and ease of use. Furthermore, the normal use case would be that the user creates the UART once on startup, then it stays forever, and the destructor is never called (at least in my experience). And I also find it a bit awkward and unintuitive, that the main function runs inside the UART-Wrapper.

But that is only my opinion. What do you think?

[chrysn]

(Reminder to never type longer text in browser windows... let's hope this second version is better)

Safe Rust code must never allow unsafe behavior. No matter how exotic or even malicious the safe code is: If it leads to undefined behavior (use-after-free, double-free, unaligned access, read while modify, or using uninitialized memory), it's always the fault of the crate that did the unsafe thing. (There are some corner cases, such as opening /proc/self/mem or overwriting libc symbols, but they're more the fault of the OS or linker).

Leaking memory is safe in Rust -- it's not undefined behavior, but more importantly, as was found out late during Rust's 1.0 phase, there are just too many ways in which one can leak memory that static analysis would not cover. Thus, our code must be safe even when memory is (intentionally or accidentally) leaked.

There are some APIs in riot-wrappers that need to deal with this already; I hope I've caught all by now that I've written before I learned the above. Using them is not too bad, I think, and falls in three categories:

• Use in a main function.

  Yeah, it's wrapping main in possibly several layers of callbacks. But the compiler can inline expand them, and drop the cleanup code if the innermost function is an endless loop (which it often is), so it's not too bad. Indent-wise, a good pattern is to have a main function that sets up all these scopes, and then calls a fn inner_main(thread_scope, ...) -> ! { ... } that gets rid of all the indentation.

• Use in startup functions.

  Startup functions can make use of static_cell as in the stdio_coap startup code to store their forever data. A downside you mentioned is that closures can't be named and thus not be stored in statics -- true, but these can be worked around using functions. Things get convoluted and type names truly horrible, so a viable alternative is:

• Use workarounds:

  • Use with nightly. The naming issues of closures and the horrible type names around them can both be done away using the type_alias_impl_trait (TAIT) nightly feature. Then, the type names become simple, and anything can be stored statically easily.

    Given the current progress on async functions, I hope that TAIT is stabilized this year.

  • Put it in a box and use Box::leak to get a &'static.

    RIOT does support dynamic memory management, and while it's generally frowned upon, some dynamic allocation at startup is sometimes accepted as being a practical compromise.

[kbarning]

Ok that makes sense. I have to say that I think I'm at the end of my knowledge here. What would you say is the best way to implement it?

[chrysn]

The code outline is still at #39 (comment). Implementing either of the two options there would be sufficient to make this PR mergable. (I'll eventually want to have both, but do whichever works better with your use case).

[kbarning]

Fixed :)

[kbarning]

Thanks for adding this. It looks good over-all, and very comprehensive. I have several details and questions annotated across the source. Quite a few of them might easily be explained by inspiration from existing code; I'd still hope to not continue on the bad examples I've set in the past.

First of all, thank you for this comprehensive review. I will try to resolve your issues in the next few days :)

[chrysn]

Unfortunately, we can't rely on the user to not core::mem::forget when the consequences are unsound (use-after-free).

From the current state (which is good but does not go all the way yet), we'd need to:

• move the safe new function from the impl<'scope> UartDevice<'scope> { ... } to a block that is impl UartDevice<'static> { ... }, because that's the scope-less situation that is sound
• Have a function roughly like this (can be on UartDevice or free, you probably have a test application with which you can try out the ergonomics):

impl UartDevice<'scope> {
    /// ...
    ///
    /// This is the scoped version of [`new()`] that can be used if you want to use short-lived callbacks, such as
    /// closures or anything containing references. The UartDevice is deconfigured when the internal main function
    /// terminates. A common pattern around this kind of scoped functions is that `main` contains the application's
    /// main loop, and never terminates (in which case the clean-up code is eliminated during compilation).
    pub fn new_scoped<CB, M, R>(
        index: usize,
        baud: u32,
        user_callback: &'scope mut F,
        main: M,
    ) -> Result<R, UartDeviceError>
    where
        F: FnMut(u8) + Sync + 'scope,
        M: FnOnce(&mut Self) -> R,
    {
        let self_ = unsafe { ... }?;
        let result = (main)(&mut self_);
        drop(self_);
        result
    }
}

(One could also leave out the drop as it's implied -- what matters is that it happens reliably at this point, and that while the scope can safely forget the &mut, the self_ will be dropped reliably before all the references to 'scope it contains become invalid).

[chrysn]

CI tests are failing because RIOT's version of the riot-sys crate wasn't updated to the latest nightly yet -- but that's on the RIOT repository side of things.

[chrysn]

When all open issues are addressed (I think it's only the matter of safety and scoping), as with the DAC PR, I'll want to have a test around to ensure that the code is actually built during CI. Let me know whether you want to give that a try in the style of c9cf3f2, otherwise I'll do it as part of the final review.

[chrysn]

I think this is approaching completion.

Given the many fixes and merges, please rebase onto current master and squash into one or some logically structured commits.

Would you add some small runnable test?

[chrysn]

Which purpose does this serve? Off my head, it doesn't change any properties of UartDevice, so I don't see why it's needed. (The UartDevice can easily have a 'scope without referring to it in its properties.)

[kbarning]

But if I remove the PhantomData, the compiler tells me that the parameter is unused and fails to compile. So should I put the 'scope generic parameter on the new functions?

[chrysn]

Yes, I think that'd be the right way to go. Only the construct_uart and new_scoped would need one -- new_without_rx and new_with_static_cb would not need a generic (although the latter will still need to demand F: ... + 'static as it does now, and may need to call construct_uart::<'static>(index. ...) but just try it out).

That's also based in the type's properties: No callback (which is all for which we'd need the lifetime) is part of the type, the type is just the UART that's usable for writing and power management. The callback just gets set by a few of the constructors, but it's not something the live type interacts with.

[kbarning]

Fixed

[Teufelchen1]

@kbarning hey! I accidentally started a competing PR, I overlooked your work. Do you intend of picking this PR up again?