Skip to content

[BUG] Seg fault when using expiremember on a large key space with many updates #323

@cvonwallenstein

Description

@cvonwallenstein

This bug generates a segmentation fault due to invalid memory access. My use case involves a large key space of sorted sets with member expirations and frequent refreshing of the expiration times, which surfaces the bug quite quickly.

In expireSubKey, when refreshing an existing member with a new expiration time, the code removes the entry from the vector and index and then re-adds the entry to the vector and index. But the index references the memory allocated by the vector. When the vector is erased first, this causes the removal from the index to fail, and then re-adding to the vector causes the following add to the index to fail, which returns a null pointer, which generates a seg fault when accessed on this line:

https://github.com/EQ-Alpha/KeyDB/blob/0a56a73bd98d4e692ae77683fdb9dd644ecfc2eb/src/expire.cpp#L857

The solution is to make sure you remove the entry from the index before erasing from the vector, since the index is referencing the memory allocated by the vector.

https://github.com/EQ-Alpha/KeyDB/blob/0a56a73bd98d4e692ae77683fdb9dd644ecfc2eb/src/expire.cpp#L824

                m_vecexpireEntries.erase(itr);
                dictDelete(m_dictIndex, szSubkey);

should be

                dictDelete(m_dictIndex, szSubkey);
                m_vecexpireEntries.erase(itr);

Reversing the order fixes the bug.

Here's a snippet of the crash:

=== KEYDB BUG REPORT START: Cut & paste starting from here ===
40348:4675715:M 10 Jun 2021 15:20:51.077 # KeyDB 6.0.16 crashed by signal: 11
40348:4675715:M 10 Jun 2021 15:20:51.077 # Crashed running the instruction at: 0x106dae1f2
40348:4675715:M 10 Jun 2021 15:20:51.077 # Accessing address: 0x8
40348:4675715:M 10 Jun 2021 15:20:51.077 # Failed assertion: (:0)

------ STACK TRACE ------
EIP:
0 keydb-server 0x0000000106dae1f2 _ZN14expireEntryFat12expireSubKeyEPKcx + 744

Backtrace:
0 keydb-server 0x0000000106d75762 _Z13logStackTraceP17__darwin_ucontext + 114
1 keydb-server 0x0000000106d75b65 _Z14sigsegvHandleriP9__siginfoPv + 255
2 libsystem_platform.dylib 0x00007fff68c355fd _sigtramp + 29
3 ??? 0x0000007900000005 0x0 + 519691042821
4 keydb-server 0x0000000106d40fb4 _Z9setExpireP6clientP7redisDbP11redisObjectS4_x + 624
5 keydb-server 0x0000000106dacf07 _Z16expireMemberCoreP6clientP11redisObjectS2_xxi + 283
6 keydb-server 0x0000000106dad007 _Z19expireMemberCommandP6client + 157
7 keydb-server 0x0000000106d23ad8 _Z4callP6clienti + 377
8 keydb-server 0x0000000106d247ca _Z14processCommandP6clienti + 2295
9 keydb-server 0x0000000106d3688c _Z28processCommandAndResetClientP6clienti + 68
10 keydb-server 0x0000000106d36a3f _Z18processInputBufferP6clienti + 348
11 keydb-server 0x0000000106d37007 _Z14processClientsv + 115
12 keydb-server 0x0000000106d212de _Z11beforeSleepP11aeEventLoop + 103
13 keydb-server 0x0000000106d1bf61 aeMain + 104
14 keydb-server 0x0000000106d2762a _Z16workerThreadMainPv + 139
15 libsystem_pthread.dylib 0x00007fff68c41109 _pthread_start + 148
16 libsystem_pthread.dylib 0x00007fff68c3cb8b thread_start + 15

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions