forked from apache/pulsar
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit a664ed1
[Java Client] Fix wrong behavior of deduplication for key based batching (apache#15413)
### Motivation
Currently message deduplication doesn't work well for key based
batching. First, the key based batch container doesn't update the
`lastSequenceIdPushed`. So a batch could contain both duplicated and not
duplicated messages. Second, when `createOpSendMsgs` is called, the
`OpSendMsg` objects are sorted by the lowest sequence ids, and the
highest sequence id is not set. If a batch contains sequence id 0,1,2,
then the message with sequence id 1 or 2 won't be dropped.
### Modifications
- Refactor the key based batch container that the
`BatchMessageContainerImpl` is reused instead of maintaining a
`KeyedBatch` class.
- When `createOpSendMsgs` is called, clear the highest sequence id field
and configure the sequence id field with the highest sequence id to fix
the second issue described before.
- Add `testKeyBasedBatchingOrder` to show and verify the current
behavior.
- Add test for key based batching into
`testProducerDeduplicationWithDiscontinuousSequenceId` to verify
`lastSlastSequenceIdPushed` is updated correctly.1 parent 59d1756 commit a664ed1Copy full SHA for a664ed1
File tree
Expand file treeCollapse file tree
0 file changed
+0
-0
lines changedFilter options
Expand file treeCollapse file tree
0 file changed
+0
-0
lines changed
0 commit comments