Skip to content

Commit 00482c9

Browse files
AlexSutilahaochengxiagemini-code-assist[bot]
authored
Bindings for cache admission policies (#54)
* export admissioner struct * Add bindings for existing admission algorithms * Proof of concept implementation of PluginAdmissioner (still uncertain about naming) * Add custom deleter for plugin admissioner * Add examples * Add tests for admission policies * Fix type hints and support admissioner argument for all cache implementations * Fix arg naming and typing for admissioner bindings * Run each admission test with various cache types * include documentation for admissioner bindings and PluginAdmissioner * Fix incorrect argument passing of 'admissioner' * Comment on type for argument for plugin system hook functions * Apply suggestion from @gemini-code-assist[bot] Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Percy <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent 1cce5b2 commit 00482c9

File tree

13 files changed

+1084
-63
lines changed

13 files changed

+1084
-63
lines changed

‎CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,7 @@ include_directories(${MAIN_PROJECT_SOURCE_DIR}/libCacheSim/bin)
252252
set(PYTHON_MODULE_SOURCES
253253
src/export.cpp
254254
src/export_cache.cpp
255+
src/export_admissioner.cpp
255256
src/export_reader.cpp
256257
src/export_analyzer.cpp
257258
src/export_misc.cpp

‎docs/src/en/examples/plugins.md‎

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
# Plugin System
22

3-
We enable user add any customized cache via libCacheSim's plugin system.
3+
## PluginCache
44

5-
With user-defined sive python hook functions,
5+
We enable users to add any customized cache via libCacheSim's plugin system.
6+
7+
With user-defined python hook functions,
68

79
```c++
810
py::function cache_init_hook;
@@ -15,8 +17,7 @@ With user-defined sive python hook functions,
1517

1618
We can simulate and determine the cache eviction behavior from the python side.
1719

18-
Here is the signature requirement for these hook functions.
19-
20+
Here are the signature requirements for these hook functions.
2021
```python
2122
def cache_init_hook(ccparams: CommonCacheParams) -> CustomizedCacheData: ...
2223
def cache_hit_hook(data: CustomizedCacheData, req: Request) -> None: ...
@@ -25,3 +26,32 @@ def cache_eviction_hook(data: CustomizedCacheData, req: Request) -> int | str: .
2526
def cache_remove_hook(data: CustomizedCacheData, obj_id: int | str) ->: ...
2627
def cache_free_hook(data: CustomizedCacheData) ->: ...
2728
```
29+
30+
- **Note:** `CustomizedCacheData` is not a type provided by the library. It simply represents what ever object the user decides to return from `cache_init_hook` and pass to the other hook functions as `data`.
31+
32+
## PluginAdmissioner
33+
34+
We enable users to define their own admission policies via libCacheSim's plugin system, which can be used in conjunction with existing cache implementations (e.g., `LRU`, `S3FIFO`).
35+
36+
With user-defined python hook functions:
37+
38+
```c++
39+
py::function admissioner_init_hook;
40+
py::function admissioner_admit_hook;
41+
py::function admissioner_update_hook;
42+
py::function admissioner_clone_hook;
43+
py::function admissioner_free_hook;
44+
```
45+
46+
We have complete control over which objects are admitted into the underlying cache conveniently from Python.
47+
48+
Here are the signature requirements for these hook functions.
49+
```python
50+
def admissioner_init_hook() -> CustomizedAdmissionerData: ...
51+
def admissioner_admit_hook(data: CustomizedAdmissionerData, req: Request) -> bool: ...
52+
def admissioner_update_hook(data: CustomizedAdmissionerData, req: Request, cache_size: int) -> None: ...
53+
def admissioner_clone_hook(data: CustomizedAdmissionerData) -> AdmissionerBase: ...
54+
def admissioner_free_hook(data: CustomizedAdmissionerData) -> None: ...
55+
```
56+
57+
- **Note:** `CustomizedAdmissionerData` is not a type provided by the library. It simply represents what ever object the user decides to return from `admissioner_init_hook` and pass to the other hook functions as `data`.

‎docs/src/en/examples/simulation.md‎

Lines changed: 174 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,176 @@
11
# Cache Simulation
22

3-
[TBD]
3+
## Basic Usage
4+
5+
The cache classes are the core of cache simulation. When an instance of a cache is creates (e.g., `LRU`, `S3FIFO`), we can configure the cache size and any cache-specific parameters such as promotion thresholds.
6+
7+
```py
8+
import libcachesim as lcs
9+
10+
# Initialize cache
11+
cache = lcs.S3FIFO(
12+
cache_size=1024 * 1024,
13+
# Cache specific parameters
14+
small_size_ratio=0.2,
15+
ghost_size_ratio=0.8,
16+
move_to_main_threshold=2,
17+
)
18+
```
19+
20+
Admission policies are optional - if none is provided, the cache will simply admit all objects according to the replacement policy. An admissioner (e.g., `BloomFilterAdmissioner`) can be placed infront of the cache by specifying the `admissioner` argument.
21+
22+
```py
23+
import libcachesim as lcs
24+
25+
# Initialize admissioner
26+
admissioner = lcs.BloomFilterAdmissioner()
27+
28+
# Step 2: Initialize cache
29+
cache = lcs.S3FIFO(
30+
cache_size=1024 * 1024,
31+
# Cache specific parameters
32+
small_size_ratio=0.2,
33+
ghost_size_ratio=0.8,
34+
move_to_main_threshold=2,
35+
# Optionally provide admissioner
36+
admissioner=admissioner,
37+
)
38+
```
39+
40+
Then we can run cache simulations using real world workloads leveraging trace readers (see [Trace Reader](reader.md) for more on using `TraceReader`):
41+
42+
```py
43+
# Process entire trace efficiently (C++ backend)
44+
req_miss_ratio, byte_miss_ratio = cache.process_trace(reader)
45+
print(f"Request miss ratio: {req_miss_ratio:.4f}, Byte miss ratio: {byte_miss_ratio:.4f}")
46+
```
47+
48+
## Caches
49+
The following cache classes all inherit from `CacheBase` and share a common interface, sharing the following arguments in all cache classes unless otherwise specified:
50+
51+
- `cache_size: int`
52+
- `default_ttl: int` (optional)
53+
- `hashpower: int` (optional)
54+
- `consider_obj_metadata: bool` (optional)
55+
- `admissioner: AdmissionerBase` (optional)
56+
57+
### LHD
58+
**Lest Hit Density** evicts objects based on each objects expected hits-per-space-consumed (hit density).
59+
60+
- *No additional parameters beyond the common arguments*
61+
62+
### LRU
63+
**Least Recently Used** evicts the object that has not been accessed for the longest time.
64+
65+
- *No additional parameters beyond the common arguments*
66+
67+
### FIFO
68+
**First-In, First-Out** evicts objects in order regardless of frequency or recency.
69+
70+
- *No additional parameters beyond the common arguments*
71+
72+
### LFU
73+
**Least Frequently Used** evicts the object with the lowest access frequency.
74+
75+
- *No additional parameters beyond the common arguments*
76+
77+
### Arc
78+
**Adaptive Replacement Cache** a hybrid algorithm which balances recency and frequency.
79+
80+
- *No additional parameters beyond the common arguments*
81+
82+
### Clock
83+
**Clock** is an low-complexity approximation of `LRU`.
84+
85+
- `int_freq: int` - Initial frequency counter value which is used for new objects (default: `0`)
86+
- `n_bit_counter: int` - Number of bits used for the frequency counter (default: `1`)
87+
88+
### Random
89+
**Random** evicts objects at random.
90+
91+
- *No additional parameters beyond the common arguments*
92+
93+
### S3FIFO
94+
[TBD]
95+
96+
### Sieve
97+
[TBD]
98+
99+
### LIRS
100+
[TBD]
101+
102+
### TwoQ
103+
[TBD]
104+
105+
### SLRU
106+
[TBD]
107+
108+
### WTinyLFU
109+
[TBD]
110+
111+
### LeCaR
112+
[TBD]
113+
114+
### LFUDA
115+
[TBD]
116+
117+
### ClockPro
118+
[TBD]
119+
120+
### Cacheus
121+
[TBD]
122+
123+
### Belady
124+
[TBD]
125+
126+
### BeladySize
127+
[TBD]
128+
129+
### LRUProb
130+
[TBD]
131+
132+
### FlashProb
133+
[TBD]
134+
135+
### GDSF
136+
[TBD]
137+
138+
### Hyperbolic
139+
[TBD]
140+
141+
### ThreeLCache
142+
[TBD]
143+
144+
### GLCache
145+
[TBD]
146+
147+
### LRB
148+
[TBD]
149+
150+
## Admission Policies
151+
152+
### BloomFilterAdmissioner
153+
Uses a Bloom filter to decide admissions based on how many times an object has been seen.
154+
155+
- *No parameters*
156+
157+
### ProbAdmissioner
158+
Admits objects with a fixed probability.
159+
160+
- `prob: float` (optional) - Probability of admitting an object (default: `0.5`)
161+
162+
### SizeAdmissioner
163+
Admits objects only if they are below a specified size threshold.
164+
165+
- `size_threshold: int` (optional) - Maximum allowed object size (in bytes) for admission (default: `9_223_372_036_854_775_807`, or `INT64_MAX`)
166+
167+
### SizeProbabilisticAdmissioner
168+
Admits objects with a probability that decreases with object size, favoring smaller objects over large.
169+
170+
- `exponent: float` (optional) - Exponent controlling how aggressively larger objects are filtered out (default: `1e-6`)
171+
172+
### AdaptSizeAdmissioner
173+
Implements **AdaptSize**, a feedback-driven policy that periodically adjusts its size threshold.
174+
175+
- `max_iteration: int` (optional) - Maximum number of iterators for parameter tuning (default: `15`)
176+
- `reconf_interval: int` (optional) - Interval (with respect to request count) at which the threshold is re-evaluated (default: `30_000`)

‎examples/admission/bloomfilter.py‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
from libcachesim import BloomFilterAdmissioner, SyntheticReader, LRU
2+
3+
BloomFilter = BloomFilterAdmissioner()
4+
lru_without_admission = LRU(
5+
cache_size=1024,
6+
# admissioner=BloomFilter
7+
)
8+
lru_with_admission = LRU(
9+
cache_size=1024,
10+
admissioner=BloomFilter
11+
)
12+
13+
reader = SyntheticReader(
14+
num_of_req=100_000,
15+
num_objects=10_000,
16+
obj_size=100,
17+
alpha=0.8,
18+
dist="zipf",
19+
)
20+
21+
without_admission_hits = 0
22+
with_admission_hits = 0
23+
24+
for req in reader:
25+
if lru_without_admission.get(req):
26+
without_admission_hits += 1
27+
if lru_with_admission.get(req):
28+
with_admission_hits += 1
29+
30+
print(f'Obtained {without_admission_hits} without using cache admission')
31+
print(f'Obtained {with_admission_hits} using cache admission')

‎examples/admission/plugin_admissioner.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
from libcachesim import PluginAdmissioner, SyntheticReader, LRU
2+
import random
3+
4+
'''
5+
A toy example where we admit ten percent of all requests
6+
at random. The admit rate is tracked and printed in the
7+
free hook to serve as a final sanity check.
8+
'''
9+
10+
11+
class AdmissionerStats:
12+
admitted_requests: int = 0
13+
total_requests: int = 0
14+
15+
16+
def init_hook():
17+
return AdmissionerStats()
18+
19+
20+
def admit_hook(data, request):
21+
admit = random.randint(1, 10) == 5
22+
if admit:
23+
data.admitted_requests += 1
24+
data.total_requests += 1
25+
return admit
26+
27+
28+
def clone_hook():
29+
raise NotImplementedError("Cloning for this plugin admissioner is not supported.")
30+
31+
32+
def update_hook(data, request, cs):
33+
pass
34+
35+
36+
def free_hook(data):
37+
print(f'Admit rate: {100 * data.admitted_requests / data.total_requests}%')
38+
39+
40+
custom_admissioner = PluginAdmissioner(
41+
"AdmitTenPercent",
42+
init_hook,
43+
admit_hook,
44+
clone_hook,
45+
update_hook,
46+
free_hook,
47+
)
48+
lru_cache = LRU(
49+
cache_size=1024,
50+
admissioner=custom_admissioner
51+
)
52+
53+
reader = SyntheticReader(
54+
num_of_req=100_000,
55+
num_objects=10_000,
56+
obj_size=100,
57+
alpha=0.8,
58+
dist="zipf",
59+
)
60+
61+
for req in reader:
62+
lru_cache.get(req)
63+
64+
# Invokes free_hook, percentage should be ~10%
65+
del lru_cache

‎libcachesim/__init__.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,16 @@
5656
PluginCache,
5757
)
5858

59+
from .admissioner import (
60+
BloomFilterAdmissioner,
61+
ProbAdmissioner,
62+
SizeAdmissioner,
63+
SizeProbabilisticAdmissioner,
64+
AdaptSizeAdmissioner,
65+
PluginAdmissioner,
66+
AdmissionerBase,
67+
)
68+
5969
from .trace_reader import TraceReader
6070
from .trace_analyzer import TraceAnalyzer
6171
from .synthetic_reader import SyntheticReader, create_zipf_requests, create_uniform_requests
@@ -110,6 +120,15 @@
110120
"LRB",
111121
# Plugin cache
112122
"PluginCache",
123+
# Admission algorithms
124+
"BloomFilterAdmissioner",
125+
"ProbAdmissioner",
126+
"SizeAdmissioner",
127+
"SizeProbabilisticAdmissioner",
128+
"AdaptSizeAdmissioner",
129+
"PluginAdmissioner",
130+
# Admissioner base class
131+
"AdmissionerBase",
113132
# Readers and analyzers
114133
"TraceReader",
115134
"TraceAnalyzer",

0 commit comments

Comments
 (0)