core/rawdb: integrate eradb backend for RPC #31604
Conversation
s1na
changed the title
cmd/geth/chaincmd.go
Outdated
There was a problem hiding this comment.
We should probably make this a config struct
|
Twist the era path resolution a bit:
|
core/rawdb/chain_freezer.go
Outdated
There was a problem hiding this comment.
TODO (sina and gary)
We should support AncientRange as well
There was a problem hiding this comment.
Created a ticket for it for a future PR
common/lru/basiclru.go
Outdated
There was a problem hiding this comment.
Can we at least use Add2 for name? :))
cmd/utils/flags.go
Outdated
There was a problem hiding this comment.
Should we also check Era flag isn't used with --datadir="" since it will silently disregard the era directory?
There was a problem hiding this comment.
Overall this looks really great. I was able to prune my mainnet node, download the era files with the downloader, then pass the workload tests.
--
One question is this PR is trying to load Era files from ancient/chain/era, but the downloaded stores them in ancient/era. Which is best?
I also pushed one small commit to remove some old uses of the non-pass through accessors on ancients.
cmd/utils/flags.go
Outdated
There was a problem hiding this comment.
Also, do you think in the future this flag will also work for full era or e2hs files? If not, maybe we should be more specific with the name as "era1"? Otherwise Era is good.
cmd/utils/flags.go
Outdated
There was a problem hiding this comment.
| Usage: "Root directory for era1 history (default = inside ancient/chain)", | |
| Usage: "Root directory for era1 history (default = inside ancient)", |
It looks like the root directory actually ends up being ancient when you run geth download-era, is this okay or should we actually fix it put the era files in ancient/chain/era ?
core/rawdb/eradb/eradb.go
Outdated
core/rawdb/eradb/eradb.go
Outdated
It's best to avoid callbacks, especially in the locked variant of the LRU, since it can lead to weird bugs.
This changes the file cache to avoid some issues around file closing. With the new logic, a file will never be closed while it is still in use on any goroutine accessing the era store. To do this, we maintain a reference counter for each file. The new cache also explicitly tracks files which are being opened. This ensures that 'replacing' a file in the cache cannot happen.
This removes the use of atomic reference counters in favor of using a scheme where all changes to reference counters are done while holding the Store mutex. Store.Close now synchronizes with all running accesses and waits until all files have been closed before returning.
We don't use it anywhere.
This adds a new constructor for the database+freezer that takes named parameters via a struct. Calls to NewDatabaseWithFreezer are replaced across the codebase.
This implements a backing store for chain history based on era1 files. The new store is integrated with the freezer. Queries for blocks and receipts below the current freezer tail are handled by the era store. --------- Co-authored-by: Gary Rong <[email protected]> Co-authored-by: Felix Lange <[email protected]> Co-authored-by: lightclient <[email protected]>
This implements a backing store for chain history based on era1 files. The new store is integrated with the freezer. Queries for blocks and receipts below the current freezer tail are handled by the era store. --------- Co-authored-by: Gary Rong <[email protected]> Co-authored-by: Felix Lange <[email protected]> Co-authored-by: lightclient <[email protected]>
5 participants
Implements #31279