[v1.14] Bluetooth: thread crashes when configuring a non 0 Slave Latency

[rolandz-setec]

Describe the bug
Our nRF52 peripheral device running Zephyr v1.14 accepts up to 4 connections from central devices (CONFIG_BT_MAX_CONN = 4), all go well with the default BT_PERIPHERAL_PREF_SLAVE_LATENCY = 0 config.
If we change BT_PERIPHERAL_PREF_SLAVE_LATENCY to non 0, and make 3 or above connections, the bluetooth thread crashes within minutes.

To Reproduce

1. build and run the peripheral system with the following,
   CONFIG_BT_MAX_CONN = 4
   BT_PERIPHERAL_PREF_SLAVE_LATENCY = 2
2. Connect 3 or more mobile devices to the peripheral.

Expected behavior
No crash of threads.

Impact
Major

Logs and console output
[00:05:33.008,941] bt_ctlr_ctrl: assert: '(ticker_status == 0) || (ticker_status == 2)' failed
***** Kernel OOPS! *****
Current thread ID = 0x200028f4
Faulting instruction address = 0x266e6
Fatal fault in thread 0x200028f4! Aborting.

Environment (please complete the following information):

• OS: Windows 10
• Toolchain :GNU Arm Embedded Toolchain
• v1.14-branch, SHA-1: ffd8a91

Additional context
Reported to @cvinayak on Slack.