fix panic when chunk size exceeds merge buffer size #2068
Conversation
| // Check if chunk size exceeds merge buffer size to prevent slice bounds out of range panic | ||
| // when mergeBufferSize is smaller than chunkSize, report error to suggest user to | ||
| // update config or adjust image chunk size | ||
| if chunkSize > mergeBufferSize { |
There was a problem hiding this comment.
Dynamic allocation also carries risks. For example, in environments with limited memory, if the chunk size is large and multiple chunks are being downloaded concurrently, it could lead to OOM (out-of-memory) issues—similar to the problem addressed in this PR.
The purpose of making mergeBufferSize configurable is to allow users to set an acceptable buffer size based on their own environment’s resources and the characteristics of their images.
There was a problem hiding this comment.
allow users to set an acceptable buffer size based on their own environment’s resources and the characteristics of their images.
It's better to automatically control (reduce or disable) the concurrency not to exceed the buffer limit. Snapshotter should not fail depending on the image size.
There was a problem hiding this comment.
done， when chunkSize > mergeBufferSize, it will automatically fall back to sequential processing using the legacy approach.
wswsmao
force-pushed
the
main
branch
2 times, most recently
from
498ed55 to
df76ff2
Compare
Signed-off-by: abushwang <[email protected]>
Add a check to ensure that chunkSize does not exceed mergeBufferSize.
If chunkSize is greater than mergeBufferSize, return an error to prompt the user to update the config or adjust the image chunk size.
This prevents potential slice bounds out of range panics during merging.