asyncio.StreamReader similar behaviour to uart.read() possible?

[JohannesNeu]

Hi everyone,

I try to pipe RS323 signals over UART to a client program via websocket. Microdot provides the Websocket and I have passed the data to it in the past by reading directly from uart.read():

@app.route('/echo')
@with_websocket
async def echo(request, ws):
   while True:
      await ws.send(passthrough(uart.read()))

(passthrough is a function that turns Nones into b''

To allow for users to send commands as well, I was told that I need to make the whole process asynchronous .

I looked around and found this tutorial which uses asyncio.StreamReader.

My slimmed down version of it looks like this:

async def receiver():
    sreader = asyncio.StreamReader(uart)
    while True:
        res = await sreader.read(1)
        if res is None:
            return b''
        else:
            return res

@app.route('/echo')
@with_websocket
async def echo(request, ws):
   while True:
      await ws.send(await receiver())

and it works, I can see the output visualized by xterm.js, but there is one problem:
asyncio.StreamReader does not seem to be able to "just pump out the bytes" as uart.read() would do.
I can use readline() or try with read(1), but it is never as smooth as it used to be with uart.read().

Can i tweak StreamReader in any way? Am I missing something?

[jimmo]

@JohannesNeu do you need to use sreader.read(1) -- can you just do sreader.read() to read all available bytes. This would make it more aligned with your original version.

[JohannesNeu]

Hi @jimmo

I tried read(), but it sadly doesnt forward or print any data:

async def receiver():
    sreader = asyncio.StreamReader(uart)
    while True:
        res = await sreader.read()
        if res is None:
            print(b'')
            return b''
        else:
            print(res)
            return res

[peterhinch]

@jimmo From the CPython docs:

If n is not provided or set to -1, read until EOF, then return all read bytes. If EOF was received and the internal buffer is empty, return an empty bytes object.

I'm not sure whether a UART ever issues EOF. It doesn't in normal use, in my experience.

Issuing (say) sreader.read(1000) will return all available bytes.

[PolarGoose]

@peterhinch,

I'm not sure whether a UART ever issues EOF. It doesn't in normal use, in my experience.

Indeed, it seems it doesn't because the await bytes = uart_stream_reader.read() never returns.

[JohannesNeu]

@peterhinch I have increased the value to something big as you suggested. (2048 to be precise)
And now I see some information:

b'  [Disabled]                \xe2\x94\x82                                \xe2\x94\x82\x1b[16;1H\xe2\x94\x82  Boot Priority #6                     [Disabled]                \xe2\x94\x82\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x82\x1b[17;1H\xe2\x94\x82  Boot Priority #7                     [Disabled]                \xe2\x94\x82\xe2\x86\x92\xe2\x86\x90: Select Screen               \xe2\x94\x82\x1b[18;1H\xe2\x94\x82  Boot Priority #8                     [Disabled]                \xe2\x94\x82\xe2\x86\x91\xe2\x86\x93: Select Item                 \xe2\x94\x82\x1b[19;1H\xe2\x94\x82'

Could I run into a situation where only half of the buffer is filled and it would keep on waiting for more data?

[peterhinch]

sreader.read(2048) will return under two conditions:

1. If more than 2048 characters have been received it will return with the first 2048.
2. If fewer characters have been received (including 0) it will return with what it has.

It won't block under any circumstances.

[JohannesNeu]

Thank you for the clarification, I will go with a high number then.

[PolarGoose]

@peterhinch,

If fewer characters have been received (including 0) it will return with what it has.
It won't block under any circumstances.

I have made a small test to check this on a ESP32-D0WD-V3.

import asyncio
import machine


async def uart_receiving_task():
    print("Task started")
    uart = machine.UART(2, 115200, tx=17, rx=16, rxbuf=10 * 1000)
    uart_stream_reader = asyncio.StreamReader(uart)
    while True:
        bytes_from_uart = await uart_stream_reader.read(2000)
        print(f"Received: {bytes_from_uart}")

asyncio.create_task(uart_receiving_task())
asyncio.get_event_loop().run_forever()

The await uart_stream_reader.read(2000) never returns 0 bytes. It returns at least 1 byte. If nothing is in the UART, the coroutine doesn't get woken up.

[peterhinch]

I think you'll find this depends on the timeout arg in the UART constructor. If there is no timeout, it will behave as you describe. I should have made that clear.