Advice on taking backup?

[trygve-lie]

How do you guys and girls take backup of your level db data?

Do you (A) just copy out the files level create on disc or do you (b) implement a export feature which dumps all data in the database to file for later import?

If A: How does this work with having the db open by the running node program? Do you close down, backup and then open the database again or do you just copy out files wile running?

If B: Do you export all data every time or do you do diffs? If the database is large, how much would a full export slow down the other operations?

Or do you simply do more advanced stuff where you write to multiple databases on different machines (in different data centers) and look at that as backup?

Maybe we are missing some backup modules for level. At least there should be some advice on how to backup / avoid data loss in the documentation I think.

[dominictarr]

I havn't really done this my self because the stuff that I happen to have done could all be rebuilt from scratch, but I think copying all the files would work, or you could use a data replication module.

[dominictarr]

there could be some edge cases around copying the files if a compaction is currently underway...
but I think this wouldn't be a problem if you snapshot the files (fs.readdir) and then copy those.
especially if you copy the manifest file, the sst files, and then the log files.

[ralphtheninja]

@trygve-lie You can use level-hyper and do liveBackup(). Currently this is poorly documented, but I'm working on fixing that. Check level-test for an example of what this might look like. So the idea with liveBackup() is to be able to create backups without needing to close the underlying db.

[ralphtheninja]

Maybe we are missing some backup modules for level. At least there should be some advice on how to backup / avoid data loss in the documentation I think.

Amen.