KeyDB crashes when active-replication enabled

[sathish-a]

While I'm trying to reproduce this. I faced the below issue.

`1:24:S 13 May 2022 11:08:40.785 # === ASSERTION FAILED ===
1:24:S 13 May 2022 11:08:40.785 # ==> db.cpp:2863 'o->FExpires() == (m_setexpire->find(sdsKey) != m_setexpire->end())' is not true

------ STACK TRACE ------

Backtrace:
keydb-server 0.0.0.0:6379(redisDbPersistentData::ensure(char const*, dictEntry**)+0x68c) [0x55b666a6e7ac]
keydb-server 0.0.0.0:6379(redisDbPersistentData::ensure(char const*)+0x41) [0x55b666a6edb1]
keydb-server 0.0.0.0:6379(redisDbPersistentData::insert(char*, redisObject*, bool, dict_iter*)+0x4a) [0x55b666a6ee2a]
keydb-server 0.0.0.0:6379(dbAddCore(redisDb*, char*, redisObject*, bool, bool, dict_iter*)+0x62) [0x55b666ac2a22]
keydb-server 0.0.0.0:6379(genericSetKey(client*, redisDb*, redisObject*, redisObject*, int, int)+0x8b) [0x55b666ac821b]
keydb-server 0.0.0.0:6379(setGenericCommand(client*, int, redisObject*, redisObject*, redisObject*, int, redisObject*, redisObject*)+0x241) [0x55b666ad82e1]
keydb-server 0.0.0.0:6379(setCommand(client*)+0xad) [0x55b666ad863d]
keydb-server 0.0.0.0:6379(call(client*, int)+0xb1) [0x55b666a951b1]
keydb-server 0.0.0.0:6379(processCommand(client*, int)+0x954) [0x55b666a96334]
keydb-server 0.0.0.0:6379(processCommandAndResetClient(client*, int)+0x66) [0x55b666b1bdb6]
keydb-server 0.0.0.0:6379(processInputBuffer(client*, bool, int)+0x174) [0x55b666b21b84]
keydb-server 0.0.0.0:6379(processClients()+0xd9) [0x55b666b21d19]
keydb-server 0.0.0.0:6379(+0x8e7f2) [0x55b666a8e7f2]
keydb-server 0.0.0.0:6379(beforeSleep(aeEventLoop*)+0x160) [0x55b666a66650]
keydb-server 0.0.0.0:6379(aeProcessEvents+0x10b) [0x55b666a5e01b]
keydb-server 0.0.0.0:6379(aeMain+0x47) [0x55b666a685d7]
keydb-server 0.0.0.0:6379(workerThreadMain(void*)+0x73) [0x55b666a8f483]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fd0b6da66db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fd0b6acf61f]

------ INFO OUTPUT ------

Server

redis_version:6.3.0
redis_git_sha1:cbc7a895
redis_git_dirty:1
redis_build_id:b187f94b9b8209e4
redis_mode:standalone
os:Linux 5.13.0-1022-aws x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:7.5.0
process_id:1
process_supervised:no
run_id:fec6cf753e62d3f699f32f16e2e593907e24b1bb
tcp_port:6379
server_time_usec:1652440120789998
uptime_in_seconds:2150
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:8272952
executable:/data/keydb-server
config_file:/etc/keydb/keydb.conf

Clients

connected_clients:176
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:48
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
current_client_thread:1
thread_0_clients:88
thread_1_clients:89

Memory

used_memory:14527504
used_memory_human:13.85M
used_memory_rss:20393984
used_memory_rss_human:19.45M
used_memory_peak:24119680
used_memory_peak_human:23.00M
used_memory_peak_perc:60.23%
used_memory_overhead:3688580
used_memory_startup:2071192
used_memory_dataset:10838924
used_memory_dataset_perc:87.02%
allocator_allocated:14971616
allocator_active:17174528
allocator_resident:20279296
total_system_memory:8115908608
total_system_memory_human:7.56G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.15
allocator_frag_bytes:2202912
allocator_rss_ratio:1.18
allocator_rss_bytes:3104768
rss_overhead_ratio:1.01
rss_overhead_bytes:114688
mem_fragmentation_ratio:1.43
mem_fragmentation_bytes:6114432
mem_not_counted_for_evict:1048576
mem_replication_backlog:1048576
mem_clients_slaves:20512
mem_clients_normal:471596
mem_aof_buffer:0
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
storage_provider:none

Persistence

loading:0
current_cow_size:6373376
current_cow_size_age:318
current_fork_perc:0.00
current_save_keys_processed:1
current_save_keys_total:0
rdb_changes_since_last_save:608
rdb_bgsave_in_progress:1
rdb_last_save_time:1652439802
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:0
rdb_last_cow_size:6410240
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

Stats

total_connections_received:2605
total_commands_processed:17510612
instantaneous_ops_per_sec:1783
total_net_input_bytes:673327824
total_net_output_bytes:88605463
instantaneous_input_kbps:102.90
instantaneous_output_kbps:63.22
rejected_connections:0
sync_full:1
sync_partial_ok:0
sync_partial_err:1
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:15870258
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:5
dump_payload_sanitizations:0
total_reads_processed:17509854
total_writes_processed:17507985
instantaneous_lock_contention:1
avg_lock_contention:0.046875
storage_provider_read_hits:0
storage_provider_read_misses:0

Replication

role:active-replica
master_global_link_status:up
connected_masters:1
master_host:172.31.27.147
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_read_repl_offset:408514
slave_repl_offset:408514
slave_priority:100
slave_read_only:0
replica_announced:1
connected_slaves:1
slave0:ip=172.31.27.147,port=6379,state=online,offset=372389,lag=1
master_failover_state:no-failover
master_replid:86152b0b3ee5b0e8b4f929520b54ee4ca79bed6a
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:504992
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:504992

CPU

used_cpu_sys:365.939394
used_cpu_user:144.930870
used_cpu_sys_children:0.001688
used_cpu_user_children:0.001929
server_threads:2
long_lock_waits:214
used_cpu_sys_main_thread:168.062849
used_cpu_user_main_thread:54.839638

Modules

Commandstats

cmdstat_set:calls=1593186,usec=2465090,usec_per_call=1.55,rejected_calls=0,failed_calls=0
cmdstat_replconf:calls=619,usec=609,usec_per_call=0.98,rejected_calls=0,failed_calls=0
cmdstat_ping:calls=662,usec=400,usec_per_call=0.60,rejected_calls=0,failed_calls=0
cmdstat_psync:calls=1,usec=350,usec_per_call=350.00,rejected_calls=0,failed_calls=0
cmdstat_info:calls=586,usec=10577,usec_per_call=18.05,rejected_calls=4,failed_calls=0
cmdstat_replping:calls=0,usec=0,usec_per_call=0.00,rejected_calls=1,failed_calls=0
cmdstat_KEYDB.MVCCRESTORE:calls=910,usec=1246,usec_per_call=1.37,rejected_calls=0,failed_calls=0
cmdstat_replicaof:calls=1,usec=59,usec_per_call=59.00,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=2600,usec=8833,usec_per_call=3.40,rejected_calls=0,failed_calls=0
cmdstat_get:calls=15909965,usec=9930893,usec_per_call=0.62,rejected_calls=0,failed_calls=0
cmdstat_rreplay:calls=2079,usec=9067,usec_per_call=4.36,rejected_calls=0,failed_calls=0
cmdstat_command:calls=3,usec=1950,usec_per_call=650.00,rejected_calls=0,failed_calls=0

Errorstats

errorstat_NOAUTH:count=5

Cluster

cluster_enabled:0

Keyspace

db0:keys=910,expires=908,avg_ttl=9158520556606729216,cached_keys=910

KeyDB

mvcc_depth:1

------ CLIENT LIST OUTPUT ------
id=2595 addr=172.31.24.79:45576 laddr=172.17.0.2:6379 fd=202 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2596 addr=172.31.24.79:45584 laddr=172.17.0.2:6379 fd=204 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2597 addr=172.31.24.79:45590 laddr=172.17.0.2:6379 fd=206 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2598 addr=172.31.24.79:45502 laddr=172.17.0.2:6379 fd=157 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2599 addr=172.31.24.79:45516 laddr=172.17.0.2:6379 fd=159 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2600 addr=172.31.24.79:45520 laddr=172.17.0.2:6379 fd=161 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2601 addr=172.31.24.79:45524 laddr=172.17.0.2:6379 fd=163 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2602 addr=172.31.24.79:45530 laddr=172.17.0.2:6379 fd=167 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2603 addr=172.31.24.79:45534 laddr=172.17.0.2:6379 fd=168 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2604 addr=172.31.24.79:45544 laddr=172.17.0.2:6379 fd=171 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2605 addr=172.31.24.79:45548 laddr=172.17.0.2:6379 fd=175 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2606 addr=172.31.24.79:45558 laddr=172.17.0.2:6379 fd=176 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2607 addr=172.31.24.79:45560 laddr=172.17.0.2:6379 fd=180 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2608 addr=172.31.24.79:45566 laddr=172.17.0.2:6379 fd=184 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2609 addr=172.31.24.79:45580 laddr=172.17.0.2:6379 fd=189 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2610 addr=172.31.24.79:45586 laddr=172.17.0.2:6379 fd=190 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2611 addr=172.31.24.79:45596 laddr=172.17.0.2:6379 fd=193 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2566 addr=172.31.24.79:45482 laddr=172.17.0.2:6379 fd=166 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2567 addr=172.31.24.79:45536 laddr=172.17.0.2:6379 fd=169 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2568 addr=172.31.24.79:45538 laddr=172.17.0.2:6379 fd=170 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2569 addr=172.31.24.79:45546 laddr=172.17.0.2:6379 fd=172 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2570 addr=172.31.24.79:45498 laddr=172.17.0.2:6379 fd=174 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2571 addr=172.31.24.79:45562 laddr=172.17.0.2:6379 fd=177 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2572 addr=172.31.24.79:45506 laddr=172.17.0.2:6379 fd=179 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2573 addr=172.31.24.79:45564 laddr=172.17.0.2:6379 fd=183 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2574 addr=172.31.24.79:45510 laddr=172.17.0.2:6379 fd=182 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2575 addr=172.31.24.79:45578 laddr=172.17.0.2:6379 fd=185 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2576 addr=172.31.24.79:45528 laddr=172.17.0.2:6379 fd=188 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2577 addr=172.31.24.79:45592 laddr=172.17.0.2:6379 fd=191 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2578 addr=172.31.24.79:45598 laddr=172.17.0.2:6379 fd=195 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2579 addr=172.31.24.79:45552 laddr=172.17.0.2:6379 fd=196 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=1776 addr=172.31.27.147:6379 laddr=172.17.0.2:45299 fd=27 name= age=645 idle=0 flags=M db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61480 events=r cmd=rreplay user=(superuser) redir=-1
id=1778 addr=172.31.27.147:39519 laddr=172.17.0.2:6379 fd=26 name= age=619 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=3980 tot-mem=24492 events=r cmd=replconf user=default redir=-1
id=2437 addr=172.31.24.79:45228 laddr=172.17.0.2:6379 fd=33 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2438 addr=172.31.24.79:45230 laddr=172.17.0.2:6379 fd=32 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2439 addr=172.31.24.79:45226 laddr=172.17.0.2:6379 fd=34 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2440 addr=172.31.24.79:45232 laddr=172.17.0.2:6379 fd=36 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2441 addr=172.31.24.79:45236 laddr=172.17.0.2:6379 fd=37 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2442 addr=172.31.24.79:45238 laddr=172.17.0.2:6379 fd=38 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2443 addr=172.31.24.79:45234 laddr=172.17.0.2:6379 fd=35 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2444 addr=172.31.24.79:45240 laddr=172.17.0.2:6379 fd=39 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2445 addr=172.31.24.79:45242 laddr=172.17.0.2:6379 fd=40 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2446 addr=172.31.24.79:45246 laddr=172.17.0.2:6379 fd=41 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2447 addr=172.31.24.79:45244 laddr=172.17.0.2:6379 fd=42 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2448 addr=172.31.24.79:45250 laddr=172.17.0.2:6379 fd=43 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2449 addr=172.31.24.79:45252 laddr=172.17.0.2:6379 fd=44 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2450 addr=172.31.24.79:45254 laddr=172.17.0.2:6379 fd=45 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2451 addr=172.31.24.79:45256 laddr=172.17.0.2:6379 fd=46 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2452 addr=172.31.24.79:45258 laddr=172.17.0.2:6379 fd=47 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2453 addr=172.31.24.79:45260 laddr=172.17.0.2:6379 fd=48 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2454 addr=172.31.24.79:45262 laddr=172.17.0.2:6379 fd=49 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2455 addr=172.31.24.79:45264 laddr=172.17.0.2:6379 fd=50 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2456 addr=172.31.24.79:45268 laddr=172.17.0.2:6379 fd=51 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2457 addr=172.31.24.79:45266 laddr=172.17.0.2:6379 fd=54 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2458 addr=172.31.24.79:45270 laddr=172.17.0.2:6379 fd=55 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2459 addr=172.31.24.79:45276 laddr=172.17.0.2:6379 fd=56 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2460 addr=172.31.24.79:45272 laddr=172.17.0.2:6379 fd=52 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2461 addr=172.31.24.79:45274 laddr=172.17.0.2:6379 fd=53 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2462 addr=172.31.24.79:45280 laddr=172.17.0.2:6379 fd=57 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2463 addr=172.31.24.79:45282 laddr=172.17.0.2:6379 fd=58 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2464 addr=172.31.24.79:45284 laddr=172.17.0.2:6379 fd=59 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2465 addr=172.31.24.79:45278 laddr=172.17.0.2:6379 fd=60 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=18 obl=0 oll=0 omem=0 tot-mem=61474 events=r cmd=get user=default redir=-1
id=2466 addr=172.31.24.79:45286 laddr=172.17.0.2:6379 fd=61 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2467 addr=172.31.24.79:45290 laddr=172.17.0.2:6379 fd=62 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2468 addr=172.31.24.79:45292 laddr=172.17.0.2:6379 fd=63 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2469 addr=172.31.24.79:45294 laddr=172.17.0.2:6379 fd=64 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2470 addr=172.31.24.79:45296 laddr=172.17.0.2:6379 fd=65 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2471 addr=172.31.24.79:45304 laddr=172.17.0.2:6379 fd=66 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2472 addr=172.31.24.79:45300 laddr=172.17.0.2:6379 fd=68 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2473 addr=172.31.24.79:45298 laddr=172.17.0.2:6379 fd=69 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2474 addr=172.31.24.79:45306 laddr=172.17.0.2:6379 fd=70 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2475 addr=172.31.24.79:45308 laddr=172.17.0.2:6379 fd=71 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2476 addr=172.31.24.79:45302 laddr=172.17.0.2:6379 fd=67 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2477 addr=172.31.24.79:45312 laddr=172.17.0.2:6379 fd=72 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2478 addr=172.31.24.79:45314 laddr=172.17.0.2:6379 fd=73 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2479 addr=172.31.24.79:45316 laddr=172.17.0.2:6379 fd=74 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2480 addr=172.31.24.79:45310 laddr=172.17.0.2:6379 fd=75 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2481 addr=172.31.24.79:45318 laddr=172.17.0.2:6379 fd=76 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2482 addr=172.31.24.79:45320 laddr=172.17.0.2:6379 fd=77 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2483 addr=172.31.24.79:45322 laddr=172.17.0.2:6379 fd=78 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2484 addr=172.31.24.79:45324 laddr=172.17.0.2:6379 fd=79 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2485 addr=172.31.24.79:45326 laddr=172.17.0.2:6379 fd=80 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2486 addr=172.31.24.79:45328 laddr=172.17.0.2:6379 fd=81 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2487 addr=172.31.24.79:45330 laddr=172.17.0.2:6379 fd=82 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2488 addr=172.31.24.79:45332 laddr=172.17.0.2:6379 fd=84 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2489 addr=172.31.24.79:45334 laddr=172.17.0.2:6379 fd=83 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2490 addr=172.31.24.79:45336 laddr=172.17.0.2:6379 fd=86 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2491 addr=172.31.24.79:45340 laddr=172.17.0.2:6379 fd=85 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2492 addr=172.31.24.79:45338 laddr=172.17.0.2:6379 fd=87 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2493 addr=172.31.24.79:45344 laddr=172.17.0.2:6379 fd=88 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2494 addr=172.31.24.79:45346 laddr=172.17.0.2:6379 fd=89 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2495 addr=172.31.24.79:45342 laddr=172.17.0.2:6379 fd=91 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2496 addr=172.31.24.79:45352 laddr=172.17.0.2:6379 fd=90 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2497 addr=172.31.24.79:45354 laddr=172.17.0.2:6379 fd=92 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2498 addr=172.31.24.79:45348 laddr=172.17.0.2:6379 fd=93 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2499 addr=172.31.24.79:45350 laddr=172.17.0.2:6379 fd=94 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2500 addr=172.31.24.79:45356 laddr=172.17.0.2:6379 fd=95 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2501 addr=172.31.24.79:45358 laddr=172.17.0.2:6379 fd=97 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2502 addr=172.31.24.79:45372 laddr=172.17.0.2:6379 fd=98 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2503 addr=172.31.24.79:45370 laddr=172.17.0.2:6379 fd=96 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2504 addr=172.31.24.79:45374 laddr=172.17.0.2:6379 fd=99 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2505 addr=172.31.24.79:45362 laddr=172.17.0.2:6379 fd=100 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2506 addr=172.31.24.79:45360 laddr=172.17.0.2:6379 fd=101 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2507 addr=172.31.24.79:45376 laddr=172.17.0.2:6379 fd=102 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2508 addr=172.31.24.79:45366 laddr=172.17.0.2:6379 fd=103 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2509 addr=172.31.24.79:45380 laddr=172.17.0.2:6379 fd=104 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2510 addr=172.31.24.79:45382 laddr=172.17.0.2:6379 fd=105 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2511 addr=172.31.24.79:45384 laddr=172.17.0.2:6379 fd=106 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2512 addr=172.31.24.79:45390 laddr=172.17.0.2:6379 fd=109 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2513 addr=172.31.24.79:45386 laddr=172.17.0.2:6379 fd=108 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2514 addr=172.31.24.79:45388 laddr=172.17.0.2:6379 fd=107 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2515 addr=172.31.24.79:45392 laddr=172.17.0.2:6379 fd=110 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=50 obl=0 oll=0 omem=0 tot-mem=61514 events=r cmd=set user=default redir=-1
id=2516 addr=172.31.24.79:45394 laddr=172.17.0.2:6379 fd=111 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2517 addr=172.31.24.79:45398 laddr=172.17.0.2:6379 fd=113 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2518 addr=172.31.24.79:45396 laddr=172.17.0.2:6379 fd=112 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2519 addr=172.31.24.79:45402 laddr=172.17.0.2:6379 fd=114 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2520 addr=172.31.24.79:45404 laddr=172.17.0.2:6379 fd=116 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2521 addr=172.31.24.79:45400 laddr=172.17.0.2:6379 fd=115 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2522 addr=172.31.24.79:45406 laddr=172.17.0.2:6379 fd=117 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2523 addr=172.31.24.79:45408 laddr=172.17.0.2:6379 fd=118 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2524 addr=172.31.24.79:45410 laddr=172.17.0.2:6379 fd=119 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2525 addr=172.31.24.79:45412 laddr=172.17.0.2:6379 fd=120 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2526 addr=172.31.24.79:45414 laddr=172.17.0.2:6379 fd=121 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2527 addr=172.31.24.79:45416 laddr=172.17.0.2:6379 fd=122 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2528 addr=172.31.24.79:45418 laddr=172.17.0.2:6379 fd=123 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2529 addr=172.31.24.79:45420 laddr=172.17.0.2:6379 fd=124 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2530 addr=172.31.24.79:45424 laddr=172.17.0.2:6379 fd=126 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2531 addr=172.31.24.79:45426 laddr=172.17.0.2:6379 fd=127 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2532 addr=172.31.24.79:45422 laddr=172.17.0.2:6379 fd=125 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2533 addr=172.31.24.79:45428 laddr=172.17.0.2:6379 fd=128 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2534 addr=172.31.24.79:45430 laddr=172.17.0.2:6379 fd=129 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2535 addr=172.31.24.79:45434 laddr=172.17.0.2:6379 fd=131 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2536 addr=172.31.24.79:45436 laddr=172.17.0.2:6379 fd=132 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2537 addr=172.31.24.79:45432 laddr=172.17.0.2:6379 fd=130 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2538 addr=172.31.24.79:45440 laddr=172.17.0.2:6379 fd=133 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2539 addr=172.31.24.79:45442 laddr=172.17.0.2:6379 fd=134 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2540 addr=172.31.24.79:45438 laddr=172.17.0.2:6379 fd=135 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2541 addr=172.31.24.79:45446 laddr=172.17.0.2:6379 fd=137 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2542 addr=172.31.24.79:45444 laddr=172.17.0.2:6379 fd=136 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2543 addr=172.31.24.79:45456 laddr=172.17.0.2:6379 fd=138 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2544 addr=172.31.24.79:45450 laddr=172.17.0.2:6379 fd=140 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2545 addr=172.31.24.79:45448 laddr=172.17.0.2:6379 fd=141 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2546 addr=172.31.24.79:45458 laddr=172.17.0.2:6379 fd=139 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2547 addr=172.31.24.79:45460 laddr=172.17.0.2:6379 fd=142 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2548 addr=172.31.24.79:45454 laddr=172.17.0.2:6379 fd=143 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2549 addr=172.31.24.79:45462 laddr=172.17.0.2:6379 fd=144 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2550 addr=172.31.24.79:45464 laddr=172.17.0.2:6379 fd=145 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2551 addr=172.31.24.79:45466 laddr=172.17.0.2:6379 fd=146 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2552 addr=172.31.24.79:45470 laddr=172.17.0.2:6379 fd=148 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2553 addr=172.31.24.79:45474 laddr=172.17.0.2:6379 fd=150 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2554 addr=172.31.24.79:45468 laddr=172.17.0.2:6379 fd=147 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2555 addr=172.31.24.79:45472 laddr=172.17.0.2:6379 fd=149 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2556 addr=172.31.24.79:45478 laddr=172.17.0.2:6379 fd=151 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2557 addr=172.31.24.79:45480 laddr=172.17.0.2:6379 fd=152 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2558 addr=172.31.24.79:45486 laddr=172.17.0.2:6379 fd=154 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2559 addr=172.31.24.79:45484 laddr=172.17.0.2:6379 fd=153 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2560 addr=172.31.24.79:45494 laddr=172.17.0.2:6379 fd=155 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2561 addr=172.31.24.79:45496 laddr=172.17.0.2:6379 fd=156 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2562 addr=172.31.24.79:45504 laddr=172.17.0.2:6379 fd=158 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2563 addr=172.31.24.79:45514 laddr=172.17.0.2:6379 fd=160 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2564 addr=172.31.24.79:45522 laddr=172.17.0.2:6379 fd=162 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2565 addr=172.31.24.79:45518 laddr=172.17.0.2:6379 fd=164 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2580 addr=172.31.24.79:45550 laddr=172.17.0.2:6379 fd=197 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2581 addr=172.31.24.79:45568 laddr=172.17.0.2:6379 fd=199 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2582 addr=172.31.24.79:45574 laddr=172.17.0.2:6379 fd=201 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2583 addr=172.31.24.79:45582 laddr=172.17.0.2:6379 fd=203 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61464 events=r cmd=set user=default redir=-1
id=2584 addr=172.31.24.79:45588 laddr=172.17.0.2:6379 fd=205 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2585 addr=172.31.24.79:45490 laddr=172.17.0.2:6379 fd=165 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2586 addr=172.31.24.79:45492 laddr=172.17.0.2:6379 fd=173 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2587 addr=172.31.24.79:45500 laddr=172.17.0.2:6379 fd=178 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2588 addr=172.31.24.79:45508 laddr=172.17.0.2:6379 fd=181 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2589 addr=172.31.24.79:45512 laddr=172.17.0.2:6379 fd=186 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2590 addr=172.31.24.79:45526 laddr=172.17.0.2:6379 fd=187 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2591 addr=172.31.24.79:45532 laddr=172.17.0.2:6379 fd=192 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2592 addr=172.31.24.79:45542 laddr=172.17.0.2:6379 fd=194 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2593 addr=172.31.24.79:45556 laddr=172.17.0.2:6379 fd=198 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=5 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1
id=2594 addr=172.31.24.79:45570 laddr=172.17.0.2:6379 fd=200 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=61456 events=r cmd=get user=default redir=-1

------ CURRENT CLIENT INFO ------
id=2515 addr=172.31.24.79:45392 laddr=172.17.0.2:6379 fd=110 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=50 obl=0 oll=0 omem=0 tot-mem=61514 events=r cmd=set user=default redir=-1
argv[0]: 'SET'
argv[1]: 'memtier-6294639'
argv[2]: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
1:24:S 13 May 2022 11:08:40.793 # key 'memtier-6294639' found in DB containing the following object:
1:24:S 13 May 2022 11:08:40.793 # Object type: 0
1:24:S 13 May 2022 11:08:40.793 # Object encoding: 8
1:24:S 13 May 2022 11:08:40.793 # Object refcount: 1

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
1:24:S 13 May 2022 11:08:40.793 # main thread terminated
1:24:S 13 May 2022 11:08:40.794 # Bio thread for job type #0 terminated
1:24:S 13 May 2022 11:08:40.794 # Bio thread for job type #1 terminated
1:24:S 13 May 2022 11:08:40.794 # Bio thread for job type #2 terminated

Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
`

[VivekSainiEQ]

Hi @sathish-a,

What did your config files look like?

[solracsf]

Seems like a duplicate of #425

[sathish-a]

Hi @VivekSainiEQ

I'm using default config however, I've changed a few options in runtime. The below command is used to run the keydb server.

docker run -itd \ --name $1 \ --sysctl net.core.somaxconn=1024 \ -p 6380:6379 \ -d eqalpha/keydb:latest \ keydb-server /etc/keydb/keydb.conf \ --server-threads 4 \ --requirepass $REDIS_AUTH \ --active-replica yes \ --masterauth $REDIS_AUTH \ --bind 0.0.0.0
And my haproxy config goes below.

`
defaults REDIS
mode tcp
timeout connect 4s
timeout server 30s
timeout client 30s

frontend stats
mode http
bind *:8404
stats enable
stats uri /stats
stats refresh 10s
stats admin if LOCALHOST

listen keydb_cluster
bind *:6379 name keydb
maxconn 40000
balance roundrobin
option tcp-check
tcp-check send AUTH\ hello-world\r\n
tcp-check expect string +OK
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send "info replication\r\n"
tcp-check expect string role:active-replica
tcp-check send QUIT\r\n
tcp-check expect string +OK
server keydb-0 192.168.14.33:6380 maxconn 20000 check inter 1s
server keydb-1 192.168.14.33:6381 maxconn 20000 check inter 1s

`

[JohnSully]

This is confirmed the same issue as #425

Fixed PR is: #431

Note that this can be mitigated by setting "enable-async-commands" to "no" in your configuration file.

[JohnSully]

closing: Using #425 to track this issue