can't upload db by redis-cli --pipe

[rnz]

Describe the bug

time redis-cli -2 --pipe < db0-31073609.resp

real	0m4.453s
user	0m0.355s
sys	0m2.143s

dragonfly_connection.cc:1030] Pipeline buffer over limit: pipeline_bytes 78768428 queue_size 100001, consider increasing pipeline_buffer_limit/pipeline_queue_limit

but in flagfile:

--pipeline_buffer_limit=7008822000
--pipeline_queue_limit=100000

Environment (please complete the following information):

• OS: Debian 12
• Kernel: Linux n02-test-podman 6.8.12-4-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-4 (2024-11-06T15:04Z) x86_64 GNU/Linux
• Containerized: podman
• "CreateCommand": [
  "podman",
  "run",
  "-dt",
  "--pod",
  "new:dfly",
  "--name",
  "df01",
  "-p",
  "6379:6379",
  "-v",
  "/var/lib/dragonfly:/data",
  "-v",
  "/opt/dfly/etc:/etc/dfly",
  "docker.dragonflydb.io/dragonflydb/dragonfly:latest",
  "--flagfile=/etc/dfly/flagfile.conf"
  ],
• Dragonfly Version: 1.25.3

[romange]

it's just a warning. have you actually saw it does not upload?

[rnz]

@romange

it's just a warning. have you actually saw it does not upload?

# Keyspace
db0:keys=2293351,expires=0,avg_ttl=-1

resp file have 31073609 keys

[romange]

Maybe it has duplicates? Can you share the file with me?
this is what I did:

printf "set KEY_%d value\n" $(seq 0 31073609) > 1.resp
redis-cli -2 --pipe < 1.resp
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 31073610

> dbsize
(integer) 31073610

[romange]

have not heard from you. so far i have not seen any evidence that there is a bug. closing, please reopen if you have more info

[rnz]

@romange

haven't heard from you. I haven't seen any evidence that there is a bug yet. closing, please reopen if you have more info

Well, I'm not a fast Gonzalez, rather a slow Rodriguez )

printf generates a non resp file. But yes, this file is fully loaded
However, the file dumped from keydb is not loaded in dragonfly. But it can be loaded in keydb, redis and garnet.

Unfortunately, I can't provide the file itself.

I also noticed that keys are scanned slowly (generated by printf):

# time redis-cli --scan | (pv -p --timer --rate --bytes )| wc -l
 374MiB 0:06:05 [1.02MiB/s] [                                                    <=>                         ]
31073610

real	6m5.862s
user	0m14.719s
sys	3m28.920s

[romange]

@romange

haven't heard from you. I haven't seen any evidence that there is a bug yet. closing, please reopen if you have more info

Well, I'm not a fast Gonzalez, rather a slow Rodriguez )

Had to google these references to understand :)

printf generates a non resp file. But yes, this file is fully loaded However, the file dumped from keydb is not loaded in dragonfly. But it can be loaded in keydb, redis and garnet.

Unfortunately, I can't provide the file itself.

Can you please run dragonfly with --vmodule=dragonfly_connection=1 and provide the info log after the the resp file has finished uploading?

I also noticed that keys are scanned slowly (generated by printf):

# time redis-cli --scan | (pv -p --timer --rate --bytes )| wc -l
 374MiB 0:06:05 [1.02MiB/s] [                                                    <=>                         ]
31073610

real	6m5.862s
user	0m14.719s
sys	3m28.920s

[rnz]

@romange

Can you please run dragonfly with --vmodule=dragonfly_connection=1 and provide the info log after the the resp file has finished uploading?

--vmodule=dragonfly_connection=1:

I20241205 15:46:12.598796    10 dragonfly_connection.cc:662] [5] HandleRequests
I20241205 15:46:12.598882    10 dragonfly_connection.cc:1668] [5] LaunchDispatchFiberIfNeeded 
W20241205 15:46:12.702705    10 dragonfly_connection.cc:1030] Pipeline buffer over limit: pipeline_bytes 78768428 queue_size 100001, consider increasing pipeline_buffer_limit/pipeline_queue_limit
I20241205 15:46:16.900218    10 dragonfly_connection.cc:984] Error parser status 2
I20241205 15:46:16.900285    10 dragonfly_connection.cc:754] Closed connection for peer id=5 addr=10.88.0.1:59094 laddr=10.88.0.39:6379 fd=676 name= tid=4 irqmatch=1 age=4 idle=0 db=0 flags=t phase=preclose lib-name= lib-ver=

--vmodule=dragonfly_connection=2:

I20241205 15:49:09.352258     6 dragonfly_connection.cc:662] [1] HandleRequests
I20241205 15:49:09.352349     6 dragonfly_connection.cc:1668] [1] LaunchDispatchFiberIfNeeded 
I20241205 15:49:09.352470     6 dragonfly_connection.cc:121] Grown io_buf to 512
I20241205 15:49:09.352504     6 dragonfly_connection.cc:121] Grown io_buf to 1024
I20241205 15:49:09.353669     6 dragonfly_connection.cc:121] Grown io_buf to 2048
I20241205 15:49:09.353709     6 dragonfly_connection.cc:121] Grown io_buf to 4096
W20241205 15:49:09.454347     6 dragonfly_connection.cc:1030] Pipeline buffer over limit: pipeline_bytes 78768428 queue_size 100001, consider increasing pipeline_buffer_limit/pipeline_queue_limit
I20241205 15:49:09.801020     6 dragonfly_connection.cc:121] Grown io_buf to 8192
I20241205 15:49:11.235170     6 dragonfly_connection.cc:121] Grown io_buf to 32768
I20241205 15:49:11.620589     6 dragonfly_connection.cc:121] Grown io_buf to 65536
I20241205 15:49:13.765785     6 dragonfly_connection.cc:969] Before dispatch_fb.join()
I20241205 15:49:13.765923     6 dragonfly_connection.cc:971] After dispatch_fb.join()
I20241205 15:49:13.768098     6 dragonfly_connection.cc:984] Error parser status 2
I20241205 15:49:13.768144     6 dragonfly_connection.cc:754] Closed connection for peer id=1 addr=10.88.0.1:34728 laddr=10.88.0.40:6379 fd=676 name= tid=0 irqmatch=1 age=4 idle=0 db=0 flags=t phase=preclose lib-name= lib-ver=

[romange]

yeah, please run dragonfly with --max_multi_bulk_len=1000000 and tell me if it helps.
I will make the warning more user friendly

[rnz]

@romange --max_multi_bulk_len=1000000 doesn't help too:

I20241205 18:26:37.683950     6 dragonfly_connection.cc:662] [1] HandleRequests
I20241205 18:26:37.684032     6 dragonfly_connection.cc:1668] [1] LaunchDispatchFiberIfNeeded 
I20241205 18:26:37.684060     6 dragonfly_connection.cc:121] Grown io_buf to 512
I20241205 18:26:37.684079     6 dragonfly_connection.cc:121] Grown io_buf to 1024
I20241205 18:26:37.685305     6 dragonfly_connection.cc:121] Grown io_buf to 2048
I20241205 18:26:37.685393     6 dragonfly_connection.cc:121] Grown io_buf to 4096
W20241205 18:26:37.790148     6 dragonfly_connection.cc:1030] Pipeline buffer over limit: pipeline_bytes 78768428 queue_size 100001, consider increasing pipeline_buffer_limit/pipeline_queue_limit
I20241205 18:26:38.147810     6 dragonfly_connection.cc:121] Grown io_buf to 8192
I20241205 18:26:39.554211     6 dragonfly_connection.cc:121] Grown io_buf to 32768
I20241205 18:26:39.922112     6 dragonfly_connection.cc:121] Grown io_buf to 65536
I20241205 18:26:42.053949     6 dragonfly_connection.cc:969] Before dispatch_fb.join()
I20241205 18:26:42.054078     6 dragonfly_connection.cc:971] After dispatch_fb.join()
I20241205 18:26:42.056334     6 dragonfly_connection.cc:984] Error parser status 2
I20241205 18:26:42.056417     6 dragonfly_connection.cc:754] Closed connection for peer id=1 addr=10.88.0.1:45752 laddr=10.88.0.43:6379 fd=676 name= tid=0 irqmatch=1 age=5 idle=0 db=0 flags=t phase=preclose lib-name= lib-ver=