Implement cache eviction

Introduce a `CacheLimit` argument for the `DiskDataCache` to configure the eviction policy. If a limit is set, the cache will track usage of the stored blocks. On put, it will check if the limit has been exceeded and start removing the least recently used blocks if required. Signed-off-by: Alessandro Passaro <alexpax@amazon.co.uk>
awslabs · Nov 16, 2023 · 2e7bdba · 2e7bdba
1 parent 68e36c9
commit 2e7bdba
Show file tree

Hide file tree

Showing 5 changed files with 382 additions and 87 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/mountpoint-s3/Cargo.toml b/mountpoint-s3/Cargo.toml
@@ -40,6 +40,8 @@ serde = { version = "1.0.190", features = ["derive"] }
 bincode = "1.3.3"
 sha2 = "0.10.6"
 hex = "0.4.3"
+linked-hash-map = "0.5.6"
+fs2 = "0.4.3"
 
 [target.'cfg(target_os = "linux")'.dependencies]
 procfs = { version = "0.15.1", default-features = false }

diff --git a/mountpoint-s3/src/data_cache.rs b/mountpoint-s3/src/data_cache.rs
@@ -11,7 +11,7 @@ use mountpoint_s3_client::types::ETag;
 use thiserror::Error;
 
 pub use crate::checksums::ChecksummedBytes;
-pub use crate::data_cache::disk_data_cache::DiskDataCache;
+pub use crate::data_cache::disk_data_cache::{CacheLimit, DiskDataCache};
 pub use crate::data_cache::in_memory_data_cache::InMemoryDataCache;
 
 /// Struct representing a key for accessing an entry in a [DataCache].