Skip to content

Conversation

@travisp
Copy link

@travisp travisp commented Aug 16, 2017

This is my proposal for issue #728. Basically it gives us the ability to use an async or buffering doc manager (such as kafka) and ensure that writes are sent prior to writing oplog progress.

In our Kafka Doc Manager, the implementation essentially looks like:

def flush(self):
  self.producer.flush()

It is a different purpose than commit(), because with an async or buffering doc manager data can be lost without the flush()

@makhdumi
Copy link
Contributor

We also implemented something like this in our own branch. It's critical that the oplog progress is only updated after all docmanagers have committed. I'm not sure why it already isn't. It caused us significant data loss.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants