Skip to content

Conversation

@MrCroxx
Copy link
Member

@MrCroxx MrCroxx commented Jun 30, 2023

Signed-off-by: MrCroxx [email protected]

  • enable direct i/o on linux target
  • refine flusher and reclaimed

close #26

- enable direct i/o on linux target
- refine flusher and reclaimer

Signed-off-by: MrCroxx <[email protected]>
@MrCroxx MrCroxx added the feature New feature or request label Jun 30, 2023
@MrCroxx MrCroxx self-assigned this Jun 30, 2023
Signed-off-by: MrCroxx <[email protected]>
@codecov-commenter
Copy link

Codecov Report

Merging #29 (d8bb7ad) into main (64b72b4) will decrease coverage by 0.36%.
The diff coverage is 4.54%.

@@            Coverage Diff             @@
##             main      #29      +/-   ##
==========================================
- Coverage   52.37%   52.01%   -0.36%     
==========================================
  Files          45       45              
  Lines        5253     5306      +53     
==========================================
+ Hits         2751     2760       +9     
- Misses       2502     2546      +44     
Impacted Files Coverage Δ
foyer-intrusive/src/eviction/mod.rs 0.00% <0.00%> (ø)
foyer-storage-bench/src/main.rs 0.36% <0.00%> (-0.01%) ⬇️
foyer-storage/src/error.rs 0.00% <0.00%> (ø)
foyer-storage/src/flusher.rs 0.00% <0.00%> (ø)
foyer-storage/src/lib.rs 100.00% <ø> (ø)
foyer-storage/src/reclaimer.rs 0.00% <0.00%> (ø)
foyer-storage/src/region.rs 0.00% <0.00%> (ø)
foyer-storage/src/region_manager.rs 0.00% <0.00%> (ø)
foyer-storage/src/store.rs 0.00% <0.00%> (ø)
foyer-storage/src/device/fs.rs 77.77% <100.00%> (+4.44%) ⬆️

@MrCroxx
Copy link
Member Author

MrCroxx commented Jun 30, 2023

Benchmark (on p44 pro)

foyer-storage-bench

args:

Args {
    dir: "/data/foyer",
    capacity: 10240,
    time: 60,
    concurrency: 16,
    pools: 16,
    report_interval: 2,
    iostat_dev: "",
    w_rate: 0.0,
    r_rate: 0.0,
    entry_size: 65536,
    lookup_range: 100000,
    region_size: 64,
    buffer_pool_size: 1024,
    flushers: 4,
    reclaimers: 4,
    align: 4096,
    io_size: 16384,
}
StoreConfig {
    eviction_config: LfuConfig {
        window_to_cache_size_ratio: 1,
        tiny_lru_capacity_ratio: 0.01,
    },
    device_config: FsDeviceConfig {
        dir: "/data/foyer",
        capacity: 10737418240,
        file_capacity: 67108864,
        align: 4096,
        io_size: 16384,
    },
    admission: AdmitAll(
        PhantomData<(u64, alloc::vec::Vec<u8>)>,
    ),
    reinsertion: ReinsertNone(
        PhantomData<(u64, alloc::vec::Vec<u8>)>,
    ),
    buffer_pool_size: 1073741824,
    flushers: 4,
    reclaimers: 4,
}

result:

Total:
disk total iops: 62274.0
disk total throughput: 938.5 MiB/s
disk read iops: 14686.8
disk read throughput: 194.9 MiB/s
disk write iops: 47587.3
disk write throughput: 743.6 MiB/s
insert iops: 11191.9/s
insert throughput: 699.5 MiB/s
insert lat p50: 165us
insert lat p90: 215us
insert lat p99: 86015us
get iops: 7330.6/s
get miss: 59.32%
get hit lat p50: 623us
get hit lat p90: 875us
get hit lat p99: 116223us
get miss lat p50: 0us
get miss lat p90: 0us
get miss lat p99: 0us

@MrCroxx MrCroxx merged commit 8388de6 into main Jun 30, 2023
@MrCroxx MrCroxx deleted the xx/dio branch June 30, 2023 07:40
MrCroxx added a commit that referenced this pull request Apr 17, 2024
* feat: enable direct i/o on linux target

- enable direct i/o on linux target
- refine flusher and reclaimer

Signed-off-by: MrCroxx <[email protected]>

* fix unit test

Signed-off-by: MrCroxx <[email protected]>

---------

Signed-off-by: MrCroxx <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

feat: direct i/o

3 participants