Use IndexedDB as cache storage on Chromium platform

[gorhill]

Currently, IndexedDB is used only on Firefox platform to implement cache storage. Chromium uses extension API storage (chrome.storage.local) as a cache storage, i.e. cache items are sharing the same physical storage as user setting ones.

A benefit of using IndexedDB is that the items stored in it are LZ4-compressable (which is enabled by default for that storage). LZ4 compression is not trivially possible with extension API storage, as this requires the storage to support the saving of Blob resources. (An approach would be to create a special encoding/decoding step, but that is just not practical.)

There is no technical obstacle to also use IndexedDB on Chromium. The main obstacle of doing so is that there is no code to automatically transfer cached items from one storage to the other, which would cause stale cache items to linger in the old storage should there be a switch to a new one for cache purpose.

Provide code to move cache items stored in chrome.storage.local to IndexedDB when uBO launches and detect that IndexedDB is being used as cache storage.

Isn't extension storage already compressed natively in Chromium? Files in Local Extension Settings/cjpalhdlnbpafiamejdnhcphjbkeiagm/ are already small and not compressible.

---

Snappy https://en.wikipedia.org/wiki/LevelDB#Features
and "LevelDB is widely noted for being unreliable and databases it manages are prone to corruption."

[gorhill]

Could be.

However I found that with IndexedDB (seems to be at IndexedDB/chrome-extension_[id]_0.indexeddb.leveldb on my side) I could definitely see less issues with garbage collection after loading filter lists or selfie.

With a valid selfie, when using chrome.storage.local, the baseline memory seems stuck at a higher level than expected. I found that with IndexedDB, the baseline memory usage at launch quickly snaps well under 40 MB on my side, something I am not able to get with chrome.storage.local. With a valid selfie (default lists), uBO's baseline memory usage at launch is now lower than it ever has been I am pretty sure.

I can only speculate but it seems the browser will keep entries from chrome.storage.local in memory more aggressively -- even forcing garbage collection in dev tools does not help, while this does not happen with IndexedDB. There is no point to keep cache-related entries in memory once they are loaded.

[uBlock-user]

My storage usage on Chromium on 1.17.3rc5 was 27 MB and now with the upgrade to 1.17.5b0 it went down to 10.3 MB, WOW! Is this because of IndexedDB being used now ?

Measured by uBO?

[uBlock-user]

Yes, the one listed in settings.html ?

On disk for all non-regional list both use ~17MB (needs some time to stabilize and restart to remove overgrown 000*.log)

[uBlock-user]

what do you mean ?

[uBlock-user]

@gorhill The storage value seems to be jumping back and forth on my end. Once I see 10 MB in use and then after few minutes/hours, it's 17 MB in use, is this to be expected ?

[gorhill]

Probably uBO created a selfie, used for fast loading next time uBO is launched. The selfie is destroyed whenever a filter list is updated, and it will re-create itself after ~17 minutes following the reload of all filter lists.

[uBlock-user]

next time uBO is launched.

So for future use ? Thats odd as I don't close Chromium for hours together(not disabling uBO either) and this happens randomly.