CCC does not get cleared when CONFIG_BT_KEYS_OVERWRITE_OLDEST is enabled

[mfiumara]

Describe the bug
When CONFIG_BT_KEYS_OVERWRITE_OLDEST is enabled, a warning gets thrown by bt_gatt saying no space to store CCC cfg.

To Reproduce
Steps to reproduce the behavior:

1. git apply prj.conf.patch && west build -b nrf52_pca10040 samples/bluetooth/peripheral_hr
2. west flash
3. bond with the peripheral using nrfconnect and subscribe to the CCC notification
4. disconnect and perform step 3. with a different device (bluetoothctl)
5. disconnect and perform step 3 again with the first device. The CCC cannot be written to the device and the error message appears

Expected behavior
Expected is that the CCC can be normally subscribed to without warning messages from bt_gatt

Impact
This leads to all notifications being broken once in this situation. If the device is power cycled the state is returned to normal.

Screenshots or console output

[00:00:55.689,941] <inf> hrs: HRS notifications enabled
Disconnected (reason 0x13)
[00:01:51.021,789] <inf> hrs: HRS notifications disabled
Connected
[00:02:06.110,473] <inf> bas: BAS Notifications enabled
[00:03:09.010,406] <inf> hrs: HRS notifications enabled
[00:03:38.810,302] <wrn> bt_ctlr_hci: type: 0x12
Disconnected (reason 0x13)
[00:03:51.810,699] <inf> bas: BAS Notifications disabled
[00:03:51.810,760] <inf> hrs: HRS notifications disabled
Connected
[00:03:58.308,197] <wrn> bt_ctlr_hci: type: 0x14
[00:04:11.032,073] <wrn> bt_gatt: No space to store CCC cfg

Environment (please complete the following information):

• OS: manjaro
• Toolchain gcc-arm-non-eabi-9-2019-q4-major
• Commit SHA or Version used: commit 9518bd19b7a736bd649fa2fc8d8d52967c0b486f (HEAD, tag: zephyr-v2.2.0, tag: v2.2.0)

Additional context
Short discussion with Joakim revelead this could be an issue in CONFIG_BT_KEYS_OVERWRITE_OLDEST, as it's quite a new feature

patch:

diff --git a/samples/bluetooth/peripheral_hr/prj.conf b/samples/bluetooth/peripheral_hr/prj.conf
index 3dfa8796b8..f65f1a0913 100644
--- a/samples/bluetooth/peripheral_hr/prj.conf
+++ b/samples/bluetooth/peripheral_hr/prj.conf
@@ -8,3 +8,7 @@ CONFIG_BT_GATT_BAS=y
 CONFIG_BT_GATT_HRS=y
 CONFIG_BT_DEVICE_NAME="Zephyr Heartrate Sensor"
 CONFIG_BT_DEVICE_APPEARANCE=833
+
+CONFIG_BT_KEYS_OVERWRITE_OLDEST=y
+CONFIG_BT_WHITELIST=y
+CONFIG_BT_MAX_PAIRED=1
\ No newline at end of file

[mfiumara]

@joerchan I redid the test as follows:
Set-up:

nrfjprog --eraseall && west flash

Connect using smartphone with (nrfconnect).
Bond using nrfconnect.
Subscribe to notifications.
Disconnect
Corresponding output from device:

*** Booting Zephyr OS build zephyr-v2.2.0  ***
Bluetooth initialized
Advertising successfully started
[00:00:00.004,974] <inf> bt_hci_core: HW Platform: Nordic Semiconductor (0x0002)
[00:00:00.004,974] <inf> bt_hci_core: HW Variant: nRF52x (0x0002)
[00:00:00.004,974] <inf> bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Version 2.2 Build 0
[00:00:00.005,615] <inf> bt_hci_core: Identity: db:5c:ca:0c:c9:c2 (random)
[00:00:00.005,615] <inf> bt_hci_core: HCI: version 5.1 (0x0a) revision 0x0000, manufacturer 0x05f1
[00:00:00.005,615] <inf> bt_hci_core: LMP: version 5.1 (0x0a) subver 0xffff
Connected
[00:00:34.359,222] <wrn> bt_ctlr_hci: type: 0x14
Disconnected (reason 0x13)
Connected
[00:00:58.149,353] <wrn> bt_ctlr_hci: type: 0x14
[00:01:15.943,328] <inf> hrs: HRS notifications enabled
Disconnected (reason 0x13)
[00:01:22.963,684] <inf> hrs: HRS notifications disabled

Connect device 2 (bluetoothctl) and perform
Pair
Subscribe
Disconnect
as follows:

[bluetooth]# scan on                                                                                                                                                                                                                                           
Discovery started                                                                                                                                                                                                                                              
[CHG] Controller 00:28:F8:F8:D6:6B Discovering: yes                                                                                                                                                                                                            
[NEW] Device DB:5C:CA:0C:C9:C2 Zephyr Heartrate Sensor                                                                                                                                                                                                         
[bluetooth]# scan off
[bluetooth]# connect DB:5C:CA:0C:C9:C2                                                                                                                                                                                                                         
Attempting to connect to DB:5C:CA:0C:C9:C2                                                                                                                                                                                                                     
[CHG] Device DB:5C:CA:0C:C9:C2 Connected: yes                                                                                                                                                                                                                  
Connection successful
[NEW] Primary Service (Handle 0x81fe)                                                                                                                                                                                                                          
        /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0001                                                                                                                                                                                                      
        00001801-0000-1000-8000-00805f9b34fb                                                                                                                                                                                                                   
        Generic Attribute Profile                                                                                                                                                                                                                              
[NEW] Characteristic (Handle 0x81fe) 
.
.
.
.
[Zephyr Heartrate Sensor]# pair                                
Attempting to pair with DB:5C:CA:0C:C9:C2
[CHG] Device DB:5C:CA:0C:C9:C2 Paired: yes
Pairing successful                                             
[Zephyr Heartrate Sensor]# menu gatt
[Zephyr Heartrate Sensor]# select-attribute 00002a37-0000-1000-8000-00805f9b34fb
[Zephyr Heartrate Sensor:/service0019/char001a]# notify on
[CHG] Attribute /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0019/char001a Notifying: yes
Notify started                                                 
[CHG] Attribute /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0019/char001a Value:
  06 66                                            .f               
[CHG] Attribute /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0019/char001a Value:
  06 67                                            .g               
[CHG] Attribute /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0019/char001a Value:
  06 68                                            .h               
[Zephyr Heartrate Sensor:/service0019/char001a]# back
[Zephyr Heartrate Sensor:/service0019/char001a]# disconnect
Attempting to disconnect from DB:5C:CA:0C:C9:C2
[CHG] Attribute /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0019/char001a Value:
  06 6b                                            .k               
[CHG] Attribute /org/bluez/hci1/dev_DB_5C_CA_0C_C9_C2/service0019/char001a Value:
  06 6c                                            .l               
[CHG] Device DB:5C:CA:0C:C9:C2 ServicesResolved: no
Successful disconnected

Corresponding output from device:

Connected
[00:01:56.086,242] <inf> bas: BAS Notifications enabled
[00:02:32.113,067] <inf> hrs: HRS notifications enabled
Disconnected (reason 0x13)
[00:02:38.937,927] <inf> bas: BAS Notifications disabled
[00:02:38.937,957] <inf> hrs: HRS notifications disabled

Connect again with device 1 (nrfconnect)
Automatically bonded using LTK's
Unable to subscribe to notifications
Output from device:

Connected
[00:02:48.763,641] <wrn> bt_ctlr_hci: type: 0x14
[00:03:08.458,831] <wrn> bt_gatt: No space to store CCC cfg

[johnsmithecon]

I am still seeing this issue occasionally, but I am using bluetooth mesh. I connect to the bearer successfully but the bearer does not open. And my device is outputting the following:

bt_gatt: No space to store CCC cfg

I'm not exactly sure how to reproduce. It seems to be happening pretty randomly and I can't pinpoint exactly when it happens.

[joerchan]

@johnsmithecon Please find out how to reproduce your issue and open a new bug-report if you can