Unable to read/write

[senglk]

I am having trouble reading and writing to/from a BLE device. This is follow up analysis from Jakeler/ble-serial#108. Write immediately causes disconnect, and read does not get registered.

Set-up

RPi5 (running minicom) <=USB=> PL2303 <=UART=> BLE-Device

1. BLE-Device is connected to the same RPi5 with bleak_test.py below.
   1. write (via bleak), the connection disconnects after awhile. From btmon and wireshark, I see no corresponding Bluetooth/ dbus-system message with the character being sent out.
   2. no read/ notify (from minicom to bleak) seems to be happening. From btmon and wireshark, There is corresponding bluetooth message but no dbus-system message, and the bleak_test.py script does not seem to register
2. When the BLE-device is connected to an android phone with serial bluetooth terminal instead, I am able to achieve bi-directional read/write.

Bleak version: 0.22.3
Python version: 3.12.3
Operating System: Ubuntu 24.04.1 LTS on RaspberryPi 5
BlueZ version (bluetoothctl -v) in case of Linux: 5.72

Found device FD:05:D3:64:F7:76: BLE_DEVICE (out of 1)
SERVICE 00001801-0000-1000-8000-00805f9b34fb (Handle: 10): Generic Attribute Profile
     CHARACTERISTIC 00002a05-0000-1000-8000-00805f9b34fb (Handle: 11): Service Changed ['indicate']
         DESCRIPTOR 00002902-0000-1000-8000-00805f9b34fb (Handle: 13): Client Characteristic Configuration
SERVICE 0000ffe0-0000-1000-8000-00805f9b34fb (Handle: 14): Vendor specific
     CHARACTERISTIC 0000ffe1-0000-1000-8000-00805f9b34fb (Handle: 15): Vendor specific ['write', 'notify']
         DESCRIPTOR 00002902-0000-1000-8000-00805f9b34fb (Handle: 17): Client Characteristic Configuration
     CHARACTERISTIC 0000ffe2-0000-1000-8000-00805f9b34fb (Handle: 18): Vendor specific ['write']

While the name of the device is HC-05, it is highly unlikely it is HC-05.

Code

"""
UART Service (bleak_test.py)
-------------
Adapted from the Nordic Semiconductor (nRF) UART service example.
"""

import asyncio
import sys
from itertools import count, takewhile
from typing import Iterator

from bleak import BleakClient, BleakScanner
from bleak.backends.characteristic import BleakGATTCharacteristic
from bleak.backends.device import BLEDevice
from bleak.backends.scanner import AdvertisementData

UART_SERVICE_UUID = "0000ffe0-0000-1000-8000-00805f9b34fb".upper()
UART_RX_CHAR_UUID = "0000ffe1-0000-1000-8000-00805f9b34fb".upper()
UART_TX_CHAR_UUID = "0000ffe1-0000-1000-8000-00805f9b34fb".upper()

# TIP: you can get this function and more from the ``more-itertools`` package.
def sliced(data: bytes, n: int) -> Iterator[bytes]:
    """
    Slices *data* into chunks of size *n*. The last slice may be smaller than
    *n*.
    """
    return takewhile(len, (data[i : i + n] for i in count(0, n)))


async def uart_terminal():
    """This is a simple "terminal" program that demonstrates UART on BLE. It reads from stdin and sends each line of data to the
    remote device. Any data received from the device is printed to stdout.
    """

    def match_nus_uuid(device: BLEDevice, adv: AdvertisementData):
        # This assumes that the device includes the UART service UUID in the
        # advertising data. This test may need to be adjusted depending on the
        # actual advertising data supplied by the device.
        if UART_SERVICE_UUID.lower() in adv.service_uuids:
            return True

        return False

    device = await BleakScanner.find_device_by_filter(match_nus_uuid)

    if device is None:
        print("no matching device found, you may need to edit match_nus_uuid().")
        sys.exit(1)

    def handle_disconnect(_: BleakClient):
        print("Device was disconnected, goodbye.")
        # cancelling all tasks effectively ends the program
        for task in asyncio.all_tasks():
            task.cancel()

    def handle_rx(_: BleakGATTCharacteristic, data: bytearray):
        print("received:", data)

    async with BleakClient(device, disconnected_callback=handle_disconnect) as client:
        ### await asyncio.sleep(5) # Initially I thought this might fix the issue but I guess not
        await client.start_notify(UART_TX_CHAR_UUID, handle_rx)

        print("Connected, start typing and press ENTER...")

        loop = asyncio.get_running_loop()
        nus = client.services.get_service(UART_SERVICE_UUID)
        rx_char = nus.get_characteristic(UART_RX_CHAR_UUID)

        while True:
            # This waits until you type a line and press ENTER.
            # A real terminal program might put stdin in raw mode so that things
            # like CTRL+C get passed to the remote device.
            data = await loop.run_in_executor(None, sys.stdin.buffer.readline)

            # data will be empty on EOF (e.g. CTRL+D on *nix)
            if not data:
                break

            # some devices, like devices running MicroPython, expect Windows
            # line endings (uncomment line below if needed)
            # data = data.replace(b"\n", b"\r\n")

            # Writing without response requires that the data can fit in a
            # single BLE packet. We can use the max_write_without_response_size
            # property to split the data into chunks that will fit.

            for s in sliced(data, rx_char.max_write_without_response_size):
                print(f"{rx_char}: {s}")
                await client.write_gatt_char(rx_char, s, response=False)

            print("sent:", data)


if __name__ == "__main__":
    try:
        asyncio.run(uart_terminal())
    except asyncio.CancelledError:
        # task is cancelled on disconnect, so we ignore this error
        pass

Write

Output (trying to send character 'a')

# (ble-venv) user@user-desktop:~/ble-serial/examples$ BLEAK_LOGGING=1 python bleak_test.py
2024-12-04 05:00:59,570 bleak.backends.bluezdbus.manager MainThread DEBUG: initial properties: {'/org/bluez': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.AgentManager1': {}, 'org.bluez.ProfileManager1': {}, 'org.bluez.HealthManager1': {}}, '/org/bluez/hci0': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.Adapter1': {'Address': 'D8:3A:DD:AD:4D:E7', 'AddressType': 'public', 'Name': 'user-desktop', 'Alias': 'user-desktop', 'Class': 8126464, 'Powered': True, 'Discoverable': False, 'DiscoverableTimeout': 180, 'Pairable': True, 'PairableTimeout': 0, 'Discovering': False, 'UUIDs': ['00001133-0000-1000-8000-00805f9b34fb', '0000110e-0000-1000-8000-00805f9b34fb', '00001105-0000-1000-8000-00805f9b34fb', '00001132-0000-1000-8000-00805f9b34fb', '00001200-0000-1000-8000-00805f9b34fb', '00001104-0000-1000-8000-00805f9b34fb', '00005005-0000-1000-8000-0002ee000001', '0000110c-0000-1000-8000-00805f9b34fb', '00001801-0000-1000-8000-00805f9b34fb', '0000112f-0000-1000-8000-00805f9b34fb', '0000110b-0000-1000-8000-00805f9b34fb', '0000180a-0000-1000-8000-00805f9b34fb', '00001800-0000-1000-8000-00805f9b34fb', '0000111f-0000-1000-8000-00805f9b34fb', '0000110a-0000-1000-8000-00805f9b34fb', '00001106-0000-1000-8000-00805f9b34fb', '0000111e-0000-1000-8000-00805f9b34fb'], 'Modalias': 'usb:v1D6Bp0246d0548', 'Roles': ['central', 'peripheral'], 'Manufacturer': 305, 'Version': 9}, 'org.freedesktop.DBus.Properties': {}, 'org.bluez.BatteryProviderManager1': {}, 'org.bluez.GattManager1': {}, 'org.bluez.Media1': {'SupportedUUIDs': ['0000110a-0000-1000-8000-00805f9b34fb', '0000110b-0000-1000-8000-00805f9b34fb']}, 'org.bluez.NetworkServer1': {}, 'org.bluez.LEAdvertisingManager1': {'ActiveInstances': 0, 'SupportedInstances': 5, 'SupportedIncludes': ['tx-power', 'appearance', 'local-name']}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.Device1': {'Address': 'FD:05:D3:64:F7:76', 'AddressType': 'public', 'Name': 'HC-05', 'Alias': 'HC-05', 'Paired': True, 'Bonded': True, 'Trusted': False, 'Blocked': False, 'LegacyPairing': False, 'Connected': False, 'UUIDs': ['00001800-0000-1000-8000-00805f9b34fb', '00001801-0000-1000-8000-00805f9b34fb', '0000ffe0-0000-1000-8000-00805f9b34fb'], 'Adapter': '/org/bluez/hci0', 'ManufacturerData': {1494: bytearray(b'\x05\x08\xfd\x05\xd3dv\xf7')}, 'ServicesResolved': False}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattService1': {'Handle': 14, 'UUID': '0000ffe0-0000-1000-8000-00805f9b34fb', 'Device': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76', 'Primary': True, 'Includes': []}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e/char0012': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattCharacteristic1': {'Handle': 18, 'UUID': '0000ffe2-0000-1000-8000-00805f9b34fb', 'Service': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e', 'Value': bytearray(b''), 'Flags': ['write']}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e/char000f': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattCharacteristic1': {'Handle': 15, 'UUID': '0000ffe1-0000-1000-8000-00805f9b34fb', 'Service': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e', 'Value': bytearray(b''), 'Notifying': False, 'Flags': ['write', 'notify'], 'NotifyAcquired': False}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e/char000f/desc0011': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattDescriptor1': {'Handle': 36, 'UUID': '00002902-0000-1000-8000-00805f9b34fb', 'Characteristic': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e/char000f', 'Value': bytearray(b'')}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000a': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattService1': {'Handle': 10, 'UUID': '00001801-0000-1000-8000-00805f9b34fb', 'Device': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76', 'Primary': True, 'Includes': []}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000a/char000b': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattCharacteristic1': {'Handle': 11, 'UUID': '00002a05-0000-1000-8000-00805f9b34fb', 'Service': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000a', 'Value': bytearray(b''), 'Notifying': False, 'Flags': ['indicate']}, 'org.freedesktop.DBus.Properties': {}}, '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000a/char000b/desc000d': {'org.freedesktop.DBus.Introspectable': {}, 'org.bluez.GattDescriptor1': {'Handle': 36, 'UUID': '00002902-0000-1000-8000-00805f9b34fb', 'Characteristic': '/org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000a/char000b', 'Value': bytearray(b'')}, 'org.freedesktop.DBus.Properties': {}}}
2024-12-04 05:00:59,576 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0): ['org.bluez.Adapter1', {'Discovering': <dbus_fast.signature.Variant ('b', True)>}, []]
2024-12-04 05:01:00,787 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0/dev_FD_05_D3_64_F7_76): ['org.bluez.Device1', {'RSSI': <dbus_fast.signature.Variant ('n', -51)>}, []]
2024-12-04 05:01:00,797 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0/dev_FD_05_D3_64_F7_76): ['org.bluez.Device1', {}, ['RSSI']]
2024-12-04 05:01:00,797 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0): ['org.bluez.Adapter1', {'Discovering': <dbus_fast.signature.Variant ('b', False)>}, []]
2024-12-04 05:01:00,806 bleak.backends.bluezdbus.client MainThread DEBUG: Connecting to device @ FD:05:D3:64:F7:76
2024-12-04 05:01:00,824 bleak.backends.bluezdbus.client MainThread DEBUG: Connecting to BlueZ path /org/bluez/hci0/dev_FD_05_D3_64_F7_76
2024-12-04 05:01:02,305 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0/dev_FD_05_D3_64_F7_76): ['org.bluez.Device1', {'Connected': <dbus_fast.signature.Variant ('b', True)>}, []]
2024-12-04 05:01:02,387 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0/dev_FD_05_D3_64_F7_76): ['org.bluez.Device1', {'ServicesResolved': <dbus_fast.signature.Variant ('b', True)>}, []]
Connected, start typing and press ENTER...
a
0000ffe1-0000-1000-8000-00805f9b34fb (Handle: 15): Vendor specific: b'a\n'
2024-12-04 05:01:16,295 bleak.backends.bluezdbus.client MainThread DEBUG: Disconnecting ({/org/bluez/hci0/dev_FD_05_D3_64_F7_76})
2024-12-04 05:01:18,719 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0/dev_FD_05_D3_64_F7_76): ['org.bluez.Device1', {'ServicesResolved': <dbus_fast.signature.Variant ('b', False)>}, []]
2024-12-04 05:01:18,720 bleak.backends.bluezdbus.manager MainThread DEBUG: received D-Bus signal: org.freedesktop.DBus.Properties.PropertiesChanged (/org/bluez/hci0/dev_FD_05_D3_64_F7_76): ['org.bluez.Device1', {'Connected': <dbus_fast.signature.Variant ('b', False)>}, []]
2024-12-04 05:01:18,720 bleak.backends.bluezdbus.client MainThread DEBUG: Device disconnected (/org/bluez/hci0/dev_FD_05_D3_64_F7_76)
2024-12-04 05:01:18,721 bleak.backends.bluezdbus.client MainThread DEBUG: _cleanup_all(/org/bluez/hci0/dev_FD_05_D3_64_F7_76)
Device was disconnected, goodbye.

Read

# wireshark bluetooth0
47	493.721171	fd:05:d3:64:f7:76 (HC-05)	localhost ()	ATT	13	Rcvd Handle Value Notification, Handle: 0x0010 (Unknown)

# btmon on bluetooth0
> ACL Data RX: Handle 64 flags 0x02 dlen 8                   #1 [hci0] 6.232223
      ATT: Handle Value Notification (0x1b) len 3
        Handle: 0x0010
          Data: 61

Red herring?

Maybe 1 out of 50 resets of bleak_test.py, Read would be working, on dbus-system I will see:

# wireshark dbus-system
2	3.812929	:1.4		D-Bus	260	* PropertiesChanged(sa{sv}as) @ /org/bluez/hci0/dev_FD_05_D3_64_F7_76/service000e/char000f

and on bleak_test.py I would see:

(ble-venv) user@user-desktop:~/ble-serial/examples$ python bleak_test.py
Connected, start typing and press ENTER...
received: bytearray(b'l')
received: bytearray(b'a')
received: bytearray(b'l')

Write still does not appear on dbus-system or minicom. However, the message seems to be captured on btmon:

# btmon, 610x => "a\n"
bluetoothd[1051]: < ACL Data TX:... flags 0x00 dlen 9  #3154 [hci0] 6417.136482
      ATT: Write Command (0x52) len 4
        Handle: 0x0010 Type: Unknown (0xffe1)
          Data: 610a
> HCI Event: Number of Completed Pack.. (0x13) plen 5  #3155 [hci0] 6417.403294
        Num handles: 1
        Handle: 64 Address: FD:05:D3:64:F7:76 (OUI FD-05-D3)
        Count: 1
        #3154: len 9 (0 Kb/s)
        Latency: 266 msec (29-408 msec ~264 msec)

At this point I'm really curious what could be causing why it works on Android (with serial bluetooth terminal) versus bleak (on raspberrypi5). Any ideas?

[dlech]

UART_RX_CHAR_UUID = "0000ffe1-0000-1000-8000-00805f9b34fb".upper()
UART_TX_CHAR_UUID = "0000ffe1-0000-1000-8000-00805f9b34fb".upper()

These are the same UUID. I assume TX is supposed to be ffe2?

[senglk]

Unfortunately using TX as ffe2 does not result in "Connected, start typing and press ENTER..." to print. Instead, it directly prints "Device was disconnected, goodbye".

For what it's worth I've also attempted to use 0000ffe2-0000-1000-8000-00805f9b34fb in the past with ble-serial without success.

[dlech]

CHARACTERISTIC 0000ffe1-0000-1000-8000-00805f9b34fb (Handle: 15): Vendor specific ['write', 'notify']

 await client.write_gatt_char(rx_char, s, response=False)

The characteristic only supports write with response, so it should be response=True.

[senglk]

I don't think changing it to True helped either. I suspect it is a bluez/dbus issue though, but I am none the wiser on how to prove or disprove this.

I've tried to use the write and read example on pygattlib release v20210616, with connection via bluetoothctl (since connect is broken with latest bluez), and I could write and read (notify) single character fine.

[dlech]

The best way to see what is going on under the hood is to log Bluetooth packets with Wireshark. Then you can compare the working and not working cases to see what the difference is.

[senglk]

Post mortem: It is likely that the device requires that the pairing connection is also the read notify/write connection, that is, the device can only be used within the pairing session. It could be a blueZ issue, but I find it more likely that this is a peculiarity with the device I am working with.

I anchored my analysis on getting notifications, but it extends to write as well. Without prior pairing, any connection ends shortly after as it is unable to retrieve services and characteristics. With prior pairing, followed by disconnect and connect, notify was never changed to true. With a fresh pairing connection, notify happens fine.

So for my case, I have to pair without connect first, then start notify and write as part of the script. When terminating, there is a need to unpair so we can pair nicely next connection attempt (cant unpair without connect). #309 would definitely make this easier.