Skip to content

Conversation

@akrambek
Copy link
Contributor

@akrambek akrambek commented Dec 29, 2023

Description

This is a current understanding of the issue

zilla: offset <= stable <= latest
which means offset <= stable, stable <= latest, offset <= latest
kafka: offset < stable <= high water mark
which means offset < stable, stable <= high water mark, offset < high water mark

Due to this invariant sometimes we may get into a state where the fix to send a flush when stableOffset or latestOffset has advanced (with no records) after records have been sent or caught up to the latest offset so at least one FLUSH was already sent.

Fixes #666

@akrambek akrambek marked this pull request as ready for review December 29, 2023 19:33
@jfallows jfallows merged commit aeeb302 into aklivity:develop Dec 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Kafka Merge is getting stall because of intermediate partition offset state

2 participants