Failed assertion: <no assertion failed> (<no file>:0) #303
Description
Describe the bug
After the Docker Container with the latest stable/unstable Version crashed and i try to restart the KeyDB, im getting
this infinity loop of errors.
** Log Files **
1:1:C 01 Apr 2021 17:32:28.406 # oO0OoO0OoO0Oo KeyDB is starting oO0OoO0OoO0Oo
1:1:C 01 Apr 2021 17:32:28.406 # KeyDB version=255.255.255, bits=64, commit=00000000, modified=0, pid=1, just started
1:1:C 01 Apr 2021 17:32:28.406 # Configuration loaded
_
_-(+)-_
_-- / \ --_
_-- / \ --_ KeyDB 255.255.255 (00000000/0) 64 bit
__-- / \ --__
(+) _ / \ _ (+) Running in standalone mode
| -- / \ -- | Port: 6379
| /--_ _ _--\ | PID: 1
| / -(+)- \ |
| / | \ | https://docs.keydb.dev
| / | \ |
| / | \ |
(+)_ -- -- -- | -- -- -- _(+)
--_ | _--
--_ | _--
-(+)- Join the KeyDB community! https://community.keydb.dev/
1:1:M 01 Apr 2021 17:32:28.662 # WARNING: The TCP backlog setting of 2048 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:1:M 01 Apr 2021 17:32:28.662 # Server initialized
1:1:M 01 Apr 2021 17:32:28.663 * Loading RDB produced by version 255.255.255
1:1:M 01 Apr 2021 17:32:28.663 * RDB age 522 seconds
1:1:M 01 Apr 2021 17:32:28.663 * RDB memory usage when created 731.18 Mb
1:1:M 01 Apr 2021 17:32:38.055 * DB loaded from disk: 9.392 seconds
1:35:M 01 Apr 2021 17:32:38.055 Thread 0 alive.
1:35:M 01 Apr 2021 17:33:30.600 # Lua slow script detected: still in execution after 11699 milliseconds. You can try killing the script using the SCRIPT KILL command. Script SHA1 is: aa102f22e4bd254581ad75f64cb52ea1cc1aa128
=== KEYDB BUG REPORT START: Cut & paste starting from here ===
1:35:M 01 Apr 2021 17:33:41.932 # KeyDB 255.255.255 crashed by signal: 11, si_code: 1
1:35:M 01 Apr 2021 17:33:41.932 # Crashed running the instruction at: 0x55dbb0c2ac58
1:35:M 01 Apr 2021 17:33:41.932 # Accessing address: 0xffffffffffffffff
1:35:M 01 Apr 2021 17:33:41.932 # Failed assertion: <no assertion failed> (<no file>:0)
------ STACK TRACE ------
EIP:
keydb-server *:6379(processInputBuffer(client*, int)+0x98) [0x55dbb0c2ac58]
Backtrace:
keydb-server *:6379(logStackTrace(ucontext_t*)+0x6b) [0x55dbb0c71ecb]
keydb-server *:6379(sigsegvHandler(int, siginfo_t*, void*)+0xed) [0x55dbb0c726ad]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f6ae4282980]
keydb-server *:6379(processInputBuffer(client*, int)+0x98) [0x55dbb0c2ac58]
keydb-server *:6379(processClients()+0xae) [0x55dbb0c2da9e]
keydb-server *:6379(beforeSleep(aeEventLoop*)+0xe4) [0x55dbb0c10424]
keydb-server *:6379(aeMain+0x89) [0x55dbb0c0c5a9]
keydb-server *:6379(workerThreadMain(void*)+0x7a) [0x55dbb0c1386a]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f6ae42776db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f6ae3fa071f]
------ INFO OUTPUT ------
# Server
redis_version:255.255.255
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:f5c4e4b31d2d686d
redis_mode:standalone
os:Linux 4.19.0-16-amd64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:7.5.0
process_id:1
run_id:67478145b164db14d666e61cec3e2780eb632a9e
tcp_port:6379
uptime_in_seconds:73
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:6685653
executable:/data/keydb-server
config_file:/etc/keydb/keydb.conf
# Clients
connected_clients:5
client_recent_max_input_buffer:1024
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
current_client_thread:0
thread_0_clients:5
# Memory
used_memory:688436128
used_memory_human:656.54M
used_memory_rss:719921152
used_memory_rss_human:686.57M
used_memory_peak:702588744
used_memory_peak_human:670.04M
used_memory_peak_perc:97.99%
used_memory_overhead:6339680
used_memory_startup:6228376
used_memory_dataset:682096448
used_memory_dataset_perc:99.98%
allocator_allocated:700005488
allocator_active:703070208
allocator_resident:719642624
total_system_memory:16820826112
total_system_memory_human:15.67G
used_memory_lua:3099648
used_memory_lua_human:2.96M
used_memory_scripts:5800
used_memory_scripts_human:5.66K
number_of_cached_scripts:5
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:3064720
allocator_rss_ratio:1.02
allocator_rss_bytes:16572416
rss_overhead_ratio:1.00
rss_overhead_bytes:278528
mem_fragmentation_ratio:1.03
mem_fragmentation_bytes:20010728
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:103152
mem_aof_buffer:0
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0
# Persistence
loading:0
rdb_changes_since_last_save:166730
rdb_bgsave_in_progress:0
rdb_last_save_time:1617298348
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
# Stats
total_connections_received:241
total_commands_processed:226193
instantaneous_ops_per_sec:5063
total_net_input_bytes:12467194
total_net_output_bytes:58646747
instantaneous_input_kbps:262.94
instantaneous_output_kbps:1525.80
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:2
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:56034
keyspace_misses:181
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_reads_processed:67959
total_writes_processed:225760
# Replication
role:master
connected_slaves:0
master_replid:f6d00f42030e6333b791524913a1efe47ec8bf52
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:13.339255
used_cpu_user:55.249342
used_cpu_sys_children:0.004308
used_cpu_user_children:0.003735
server_threads:1
long_lock_waits:0
# Modules
# Commandstats
cmdstat_del:calls=3,usec=4895,usec_per_call=1631.67
cmdstat_hset:calls=9679,usec=31585,usec_per_call=3.26
cmdstat_multi:calls=10301,usec=4780,usec_per_call=0.46
cmdstat_scard:calls=120,usec=40,usec_per_call=0.33
cmdstat_hexists:calls=2885,usec=6531,usec_per_call=2.26
cmdstat_keys:calls=1,usec=17,usec_per_call=17.00
cmdstat_command:calls=1,usec=2101,usec_per_call=2101.00
cmdstat_exec:calls=7645,usec=15415187,usec_per_call=2016.38
cmdstat_srem:calls=41928,usec=75860,usec_per_call=1.81
cmdstat_zcard:calls=60,usec=78,usec_per_call=1.30
cmdstat_script:calls=310,usec=947,usec_per_call=3.05
cmdstat_llen:calls=120,usec=118,usec_per_call=0.98
cmdstat_hget:calls=52957,usec=124898,usec_per_call=2.36
cmdstat_publish:calls=7585,usec=476338,usec_per_call=62.80
cmdstat_incr:calls=2885,usec=5946,usec_per_call=2.06
cmdstat_evalsha:calls=3067,usec=33996681,usec_per_call=11084.67
cmdstat_brpoplpush:calls=32959,usec=185696,usec_per_call=5.63
cmdstat_info:calls=2182,usec=85471,usec_per_call=39.17
cmdstat_set:calls=182,usec=316,usec_per_call=1.74
cmdstat_subscribe:calls=60,usec=176,usec_per_call=2.93
cmdstat_lrem:calls=40476,usec=48297207,usec_per_call=1193.23
cmdstat_hdel:calls=7585,usec=28626,usec_per_call=3.77
cmdstat_get:calls=60,usec=32,usec_per_call=0.53
cmdstat_lrange:calls=3,usec=4033,usec_per_call=1344.33
cmdstat_lpush:calls=2919,usec=24344,usec_per_call=8.34
cmdstat_hlen:calls=7,usec=9,usec_per_call=1.29
cmdstat_sadd:calls=30,usec=8180,usec_per_call=272.67
cmdstat_smembers:calls=3,usec=6095,usec_per_call=2031.67
cmdstat_select:calls=180,usec=109,usec_per_call=0.61
# Cluster
cluster_enabled:0
# Keyspace
db0:keys=5,expires=1,avg_ttl=0
------ CLIENT LIST OUTPUT ------
id=207 addr=10.0.14.6:54568 fd=217 name= age=60 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=hget user=default
id=184 addr=127.0.0.1:60216 fd=194 name= age=61 idle=49 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=hlen user=default
id=162 addr=10.0.14.6:47954 fd=173 name= age=67 idle=32 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default
id=163 addr=10.0.14.6:47956 fd=174 name= age=67 idle=32 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 argv-mem=0 obl=0 oll=17 omem=328112 tot-mem=389616 events=r cmd=evalsha user=default
id=164 addr=10.0.14.6:47958 fd=175 name= age=67 idle=32 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=brpoplpush user=default
------ REGISTERS ------
1:35:M 01 Apr 2021 17:33:41.933 #
RAX:000055dbb0fe1e20 RBX:00007f6aa6785c40
RCX:0000000000000000 RDX:000055dbb0fe1e48
RDI:0000000000000000 RSI:00007f6ade42f100
RBP:ffffffffffffffe0 RSP:00007f6aa7ffd4a0
R8 :0000000000000000 R9 :0000000000001000
R10:0000000000000001 R11:0000000000000010
R12:000000000000000f R13:000055dbb0d74f0c
R14:0000000000000000 R15:00007ffccde0c2c0
RIP:000055dbb0c2ac58 EFL:0000000000010246
CSGSFS:002b000000000033
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4af) -> 000055dbb0c10424
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4ae) -> 0000000000000000
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4ad) -> ffffffffffffffff
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4ac) -> 00007f6ade42f100
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4ab) -> 0000000000000000
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4aa) -> 000055dbb0fe1b20
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a9) -> 4dc657ff53727300
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a8) -> 0000000000000000
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a7) -> 000055dbb0c0c101
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a6) -> 00007f6aa6789edc
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a5) -> 000055dbb0c2da9e
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a4) -> 0000000000000000
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a3) -> ffffffffffffffe0
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a2) -> 00007f6aa72051d8
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a1) -> 00007f6aa6785c40
1:35:M 01 Apr 2021 17:33:41.933 # (00007f6aa7ffd4a0) -> 00007f6aa7205058
------ MODULES INFO OUTPUT ------
------ FAST MEMORY TEST ------
1:35:M 01 Apr 2021 17:33:41.934 # main thread terminated
1:35:M 01 Apr 2021 17:33:41.934 # Bio thread for job type #0 terminated
1:35:M 01 Apr 2021 17:33:41.934 # Bio thread for job type #1 terminated
1:35:M 01 Apr 2021 17:33:41.934 # Bio thread for job type #2 terminated
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: _Z18processInputBufferP6clienti (base: 0x55dbb0c2abc0)
Module: keydb-server *:6379 (base 0x55dbb0bbe000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x55dbb0c2abc0 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
1:35:M 01 Apr 2021 17:33:41.934 # dump of function (hexdump of 280 bytes):
415641554154554189f4488b2d6fd33a00534889fb8b8f9842000064488b55004885d2741f4863c1488d3480488d04f0488b3248c1e004480305d2d73a0048397028743d85c9743048837b08007432488d3d4e451400488d35b79c1400ba04090000e8c9590400bf01000000e82f87fdff0f1f8000000000e8b30d080085c074c74c8d2dc4a214000f1f840000000000488b7b28488b4b300fb647ff4889ce89c283e2070f8556010000c1f80348984839c10f83d8000000488b83c00000004c8db3c0000000a801751364488b45008bb09000000085f60f853b010000488b83c0000000a8100f852c010000488b83c0000000a9000000400f851a010000488b050bd73a008b90ec35000085d2740f488b83c0000000a802
Function at 0x55dbb0c705f0 is _serverAssert
Function at 0x55dbb0cab9f0 is _Z25moduleGILAcquiredByModulev
=== KEYDB BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
https://github.com/JohnSully/KeyDB/issues
Suspect RAM error? Use keydb-server --test-memory to verify it.
To Reproduce
no sorry :/