Skip to content

Commit a664ed1

Browse files
BewareMyPowergavingaozhangmin
authored andcommitted
[Java Client] Fix wrong behavior of deduplication for key based batching (apache#15413)
### Motivation Currently message deduplication doesn't work well for key based batching. First, the key based batch container doesn't update the `lastSequenceIdPushed`. So a batch could contain both duplicated and not duplicated messages. Second, when `createOpSendMsgs` is called, the `OpSendMsg` objects are sorted by the lowest sequence ids, and the highest sequence id is not set. If a batch contains sequence id 0,1,2, then the message with sequence id 1 or 2 won't be dropped. ### Modifications - Refactor the key based batch container that the `BatchMessageContainerImpl` is reused instead of maintaining a `KeyedBatch` class. - When `createOpSendMsgs` is called, clear the highest sequence id field and configure the sequence id field with the highest sequence id to fix the second issue described before. - Add `testKeyBasedBatchingOrder` to show and verify the current behavior. - Add test for key based batching into `testProducerDeduplicationWithDiscontinuousSequenceId` to verify `lastSlastSequenceIdPushed` is updated correctly.
1 parent 59d1756 commit a664ed1

File tree

0 file changed

+0
-0
lines changed

    0 file changed

    +0
    -0
    lines changed

    0 commit comments

    Comments
     (0)