Skip to content

Commit 85fb661

Browse files
committed
ci: enable 'librocksdb-sys' to be able to be properly cached
Today we leverage the 'Swatinem/rust-cache' github action in order to cache rust third-party dependencies and avoid needing to rebuild them every time CI is run. Unfortunately it was identified that despite a cache hit the 'librocksdb-sys' crate was always retriggering a very costly build. The crux of the issue is mostly highlighted by this issue (rust-rocksdb/rust-rocksdb#574) on the rust-rocksdb repository. The 'librocksdb-sys' crate uses a build script to be able to build the c++ rocksdb project (tracked via a submodule in the rust-rocksdb repo) and makes use of of the cargo directive "cargo:rerun-if-changed=rocksdb/" to ensure that anytime the submodule is updated that the c++ code should be recompiled. In particular this cargo directive only uses the filesystem last-modified "mtime" timestamp to determine if the file has changed (https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorerun-if-changedpath). The 'Swatinem/rust-cache' action explicitly avoids caching the '.cargo/registry/src' directory since its faster for cargo to repopulate the unpacked crate src from their archives in '.cargo/registry/cache' (which is cached by the github action). This leads to the unintended consiquency that when cargo goes to unpack the 'librocksdb-sys' crate src, the mtime of the "rocksdb/" directory no longer matches the cached mtime that was restored from a previous build of the 'librocksdb-sys' dependency resulting in 'librocksdb-sys' being rebuilt. In order to fix this issue we can additionally cache the '.cargo/registry/src/*/librocksdb-sys-*' directory (since the github caching infrastructure preserves mtimes of cached files) and ensure that the mtime matches what cargo expects, avoiding a rebuild. In order to do this I've forked the 'Swatinem/rust-cache' action to 'bmwill/rust-cache' add added the ability to additionally specify other paths that should be cached and specified the '.cargo/registry/src/*/lib-rocksdb-sys-*' directory.
1 parent e513293 commit 85fb661

File tree

1 file changed

+15
-3
lines changed

1 file changed

+15
-3
lines changed

.github/workflows/rust.yml

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,11 @@ jobs:
6060
steps:
6161
- uses: actions/checkout@v2
6262
- uses: actions-rs/toolchain@v1
63-
- uses: Swatinem/rust-cache@v1
63+
# Enable caching of the 'librocksdb-sys' crate by additionally caching the
64+
# 'librocksdb-sys' src directory which is managed by cargo
65+
- uses: bmwill/rust-cache@v1 # Fork of 'Swatinem/rust-cache' which allows caching additional paths
66+
with:
67+
path: ~/.cargo/registry/src/**/librocksdb-sys-*
6468
- name: cargo test
6569
uses: actions-rs/cargo@v1
6670
with:
@@ -76,7 +80,11 @@ jobs:
7680
- uses: actions-rs/toolchain@v1
7781
with:
7882
components: clippy
79-
- uses: Swatinem/rust-cache@v1
83+
# Enable caching of the 'librocksdb-sys' crate by additionally caching the
84+
# 'librocksdb-sys' src directory which is managed by cargo
85+
- uses: bmwill/rust-cache@v1 # Fork of 'Swatinem/rust-cache' which allows caching additional paths
86+
with:
87+
path: ~/.cargo/registry/src/**/librocksdb-sys-*
8088
- name: cargo clippy
8189
uses: actions-rs/cargo@v1
8290
with:
@@ -112,7 +120,11 @@ jobs:
112120
steps:
113121
- uses: actions/checkout@v2
114122
- uses: actions-rs/toolchain@v1
115-
- uses: Swatinem/rust-cache@v1
123+
# Enable caching of the 'librocksdb-sys' crate by additionally caching the
124+
# 'librocksdb-sys' src directory which is managed by cargo
125+
- uses: bmwill/rust-cache@v1 # Fork of 'Swatinem/rust-cache' which allows caching additional paths
126+
with:
127+
path: ~/.cargo/registry/src/**/librocksdb-sys-*
116128
- name: Install cargo-udeps, and cache the binary
117129
uses: baptiste0928/cargo-install@v1
118130
with:

0 commit comments

Comments
 (0)