Skip to content

Commit 3dc89a1

Browse files
authored
[Fix] Clean up all explict DataLoader usage (#55)
* Clean up all explict DataLoader usage * Combine translator * Fix badge link of MKDocs
1 parent 00482c9 commit 3dc89a1

File tree

6 files changed

+32
-35
lines changed

6 files changed

+32
-35
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# libCacheSim Python Binding
22

33
[![Build](https://github.com/cacheMon/libCacheSim-python/actions/workflows/build.yml/badge.svg)](https://github.com/cacheMon/libCacheSim-python/actions/workflows/build.yml)
4-
[![Documentation](https://github.com/cacheMon/libCacheSim-python/actions/workflows/docs.yml/badge.svg)](docs.libcachesim.com/python)
4+
[![Documentation](https://github.com/cacheMon/libCacheSim-python/actions/workflows/docs.yml/badge.svg)]([docs.libcachesim.com/python](https://github.com/cacheMon/libCacheSim-python/actions/workflows/docs.yml))
55

66

77
libCacheSim is fast with the features from [underlying libCacheSim lib](https://github.com/1a1a11a/libCacheSim):

docs/src/en/getting_started/quickstart.md

Lines changed: 11 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -91,11 +91,10 @@ With libcachesim installed, you can start cache simulation for some eviction alg
9191

9292
The above example demonstrates the basic workflow of using `libcachesim` for cache simulation:
9393

94-
1. Use `DataLoader` to download a cache trace file from an S3 bucket.
95-
2. Open and efficiently process the trace file with `TraceReader`.
96-
3. Initialize a cache object (here, `S3FIFO`) with a specified cache size (e.g., 1MB).
97-
4. Run the simulation on the entire trace using `process_trace` to obtain object and byte miss ratios.
98-
5. Optionally, process only a portion of the trace by specifying `start_req` and `max_req` for partial simulation.
94+
1. Open and efficiently process the trace file with `TraceReader`.
95+
2. Initialize a cache object (here, `S3FIFO`) with a specified cache size (e.g., 1MB).
96+
3. Run the simulation on the entire trace using `process_trace` to obtain object and byte miss ratios.
97+
4. Optionally, process only a portion of the trace by specifying `start_req` and `max_req` for partial simulation.
9998

10099
This workflow applies to most cache algorithms and trace types, making it easy to get started and customize your experiments.
101100

@@ -108,12 +107,9 @@ Here is an example demonstrating how to use `TraceAnalyzer`.
108107
import libcachesim as lcs
109108

110109
# Step 1: Get one trace from S3 bucket
111-
URI = "cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst"
112-
dl = lcs.DataLoader()
113-
dl.load(URI)
114-
110+
URI = "s3://cache-datasets/cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst"
115111
reader = lcs.TraceReader(
116-
trace = dl.get_cache_path(URI),
112+
trace = URI,
117113
trace_type = lcs.TraceType.ORACLE_GENERAL_TRACE,
118114
reader_init_params = lcs.ReaderInitParam(ignore_obj_size=False)
119115
)
@@ -143,12 +139,11 @@ Here is an example demonstrating how to use `TraceAnalyzer`.
143139

144140
The above code demonstrates how to perform trace analysis using `libcachesim`. The workflow is as follows:
145141

146-
1. Download a trace file from an S3 bucket using `DataLoader`.
147-
2. Open the trace file with `TraceReader`, specifying the trace type and any reader initialization parameters.
148-
3. Configure the analysis options with `AnalysisOption` to enable or disable specific analyses (such as request rate, size, etc.).
149-
4. Optionally, set additional analysis parameters with `AnalysisParam`.
150-
5. Create a `TraceAnalyzer` object with the reader, output directory, and the chosen options and parameters.
151-
6. Run the analysis with `analyzer.run()`.
142+
1. Open the trace file with `TraceReader`, specifying the trace type and any reader initialization parameters. The URI starting with `s3://`, will download a trace file from an S3 bucket.
143+
2. Configure the analysis options with `AnalysisOption` to enable or disable specific analyses (such as request rate, size, etc.).
144+
3. Optionally, set additional analysis parameters with `AnalysisParam`.
145+
4. Create a `TraceAnalyzer` object with the reader, output directory, and the chosen options and parameters.
146+
5. Run the analysis with `analyzer.run()`.
152147

153148
After running, you can access the analysis results, such as summary statistics (`stat`) or detailed results (e.g., `example_analysis.size`).
154149

examples/plugin_cache/s3fifo.py

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -193,17 +193,16 @@ def cache_free_hook(cache):
193193
cache_name="S3FIFO",
194194
)
195195

196-
URI = "cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst"
197-
dl = lcs.DataLoader()
198-
dl.load(URI)
196+
URI = "s3://cache-datasets/cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst"
199197

200-
# Step 2: Open trace and process efficiently
198+
# Open trace
201199
reader = lcs.TraceReader(
202-
trace=dl.get_cache_path(URI),
200+
trace=URI,
203201
trace_type=lcs.TraceType.ORACLE_GENERAL_TRACE,
204202
reader_init_params=lcs.ReaderInitParam(ignore_obj_size=True),
205203
)
206204

205+
# Use native S3FIFO for reference
207206
ref_s3fifo = S3FIFO(cache_size=1024, small_size_ratio=0.1, ghost_size_ratio=0.9, move_to_main_threshold=2)
208207

209208
# for req in reader:

examples/trace_analysis.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
import libcachesim as lcs
22

33
# Step 1: Get one trace from S3 bucket
4-
URI = "cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst"
5-
dl = lcs.DataLoader()
6-
dl.load(URI)
4+
URI = "s3://cache-datasets/cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst"
75

86
reader = lcs.TraceReader(
9-
trace=dl.get_cache_path(URI),
7+
trace=URI,
108
trace_type=lcs.TraceType.ORACLE_GENERAL_TRACE,
119
reader_init_params=lcs.ReaderInitParam(ignore_obj_size=False),
1210
)

src/exception.cpp

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,36 +17,41 @@ void register_exception(py::module& m) {
1717
static py::exception<CacheException> exc_cache(m, "CacheException");
1818
static py::exception<ReaderException> exc_reader(m, "ReaderException");
1919

20+
// Single exception translator with catch blocks ordered from most-specific to least-specific
2021
py::register_exception_translator([](std::exception_ptr p) {
2122
try {
2223
if (p) std::rethrow_exception(p);
2324
} catch (const CacheException& e) {
24-
exc_cache(e.what());
25+
// Custom exception: CacheException
26+
py::set_error(exc_cache, e.what());
2527
} catch (const ReaderException& e) {
26-
exc_reader(e.what());
27-
}
28-
});
29-
30-
py::register_exception_translator([](std::exception_ptr p) {
31-
try {
32-
if (p) std::rethrow_exception(p);
28+
// Custom exception: ReaderException
29+
py::set_error(exc_reader, e.what());
3330
} catch (const std::bad_alloc& e) {
31+
// Memory allocation error
3432
PyErr_SetString(PyExc_MemoryError, e.what());
3533
} catch (const std::invalid_argument& e) {
34+
// Invalid argument error
3635
PyErr_SetString(PyExc_ValueError, e.what());
3736
} catch (const std::out_of_range& e) {
37+
// Out of range error
3838
PyErr_SetString(PyExc_IndexError, e.what());
3939
} catch (const std::domain_error& e) {
40+
// Domain error
4041
PyErr_SetString(PyExc_ValueError,
4142
("Domain error: " + std::string(e.what())).c_str());
4243
} catch (const std::overflow_error& e) {
44+
// Overflow error
4345
PyErr_SetString(PyExc_OverflowError, e.what());
4446
} catch (const std::range_error& e) {
47+
// Range error
4548
PyErr_SetString(PyExc_ValueError,
4649
("Range error: " + std::string(e.what())).c_str());
4750
} catch (const std::runtime_error& e) {
51+
// Generic runtime error
4852
PyErr_SetString(PyExc_RuntimeError, e.what());
4953
} catch (const std::exception& e) {
54+
// Catch-all for any other std::exception
5055
PyErr_SetString(PyExc_RuntimeError,
5156
("C++ exception: " + std::string(e.what())).c_str());
5257
}

0 commit comments

Comments
 (0)