Skip to content

Commit a581585

Browse files
chore: suggestions from code review
Co-authored-by: Ara Adkins <[email protected]>
1 parent 2458cc1 commit a581585

File tree

2 files changed

+111
-93
lines changed

2 files changed

+111
-93
lines changed

VisitedPcs_Trait_Benchmark.md

+111
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Adding the `VisitedPcs` Trait
2+
3+
The state of the blockifier as of commit
4+
`14e6a87722c1d0c757b1aa2756ffabe3f248fd7d` doesn't store the complete vector of
5+
visited program counters for each entry-point in an invoke transaction. Instead,
6+
visited program counters are pushed into a `HashSet`. Unfortunately this limits
7+
the ability to perform profiling operations, as many require a record of the
8+
full trace returned from the `cairo-vm`.
9+
10+
In order to enable more in-depth tracing use-cases, we have introduced the
11+
`VisitedPcs` trait which allows the user to process the visited program counters
12+
as they see fit.
13+
14+
## Before Changes
15+
16+
Visited program counters are kept in the `CachedState` structure as shown below:
17+
18+
```rust
19+
#[derive(Debug)]
20+
pub struct CachedState<S: StateReader> {
21+
pub state: S,
22+
// Invariant: read/write access is managed by CachedState.
23+
// Using interior mutability to update caches during `State`'s immutable getters.
24+
pub(crate) cache: RefCell<StateCache>,
25+
pub(crate) class_hash_to_class: RefCell<ContractClassMapping>,
26+
/// A map from class hash to the set of PC values that were visited in the class.
27+
pub visited_pcs: HashMap<ClassHash, HashSet<usize>>,
28+
}
29+
```
30+
31+
This snipped has been extracted from commit
32+
[14e6a87722c1d0c757b1aa2756ffabe3f248fd7d](https://github.com/reilabs/blockifier/blob/14e6a87722c1d0c757b1aa2756ffabe3f248fd7d/crates/blockifier/src/state/cached_state.rs#L36)
33+
34+
## After Changes
35+
36+
> [!NOTE]
37+
> The new code is developed in the branch `visited_pcs_trait` and the
38+
> current head of the branch is at commit
39+
> [`bdb1b49331aad91d445ac2155baa40fa783bcf7f`](https://github.com/reilabs/blockifier/blob/visited_pcs_trait/crates/blockifier/src/state/cached_state.rs#L37).
40+
> This will change once these changes are merged in the main branch.
41+
42+
`VisitedPcs` is added as an additional generic parameter of `CachedState`.
43+
44+
```rust
45+
#[derive(Debug)]
46+
pub struct CachedState<S: StateReader, V: VisitedPcs> {
47+
pub state: S,
48+
// Invariant: read/write access is managed by CachedState.
49+
// Using interior mutability to update caches during `State`'s immutable getters.
50+
pub(crate) cache: RefCell<StateCache>,
51+
pub(crate) class_hash_to_class: RefCell<ContractClassMapping>,
52+
/// A map from class hash to the set of PC values that were visited in the class.
53+
pub visited_pcs: V,
54+
}
55+
```
56+
57+
An implementation of the trait `VisitedPcs` is included in the blockifier with
58+
the name `VisitedPcsSet`. This mimics the existing `HashSet<usize>` usage of
59+
this field. For test purposes, `CachedState` is instantiated using
60+
`VisitedPcsSet`.
61+
62+
## Performance Considerations
63+
64+
Given the importance of the blockifier's performance in the Starknet ecosystem,
65+
measuring the impact of adding the aforementioned `VisitedPcs` trait is very
66+
important. The existing bechmark `transfers` doesn't cover operations that use
67+
the `CachedState`, and therefore we have designed new ones as follows:
68+
69+
- `cached_state`: this benchmark tests the performance impact of populating
70+
`visited_pcs` (implemented using `VisitedPcsSet`) with a realistic amount of
71+
visited program counters. The size of the sets is taken from transaction
72+
`0x0177C9365875CAA840EA8F03F97B0E3A8EE8851A8B952BF157B5DBD4FECCB060` on
73+
mainnet. This transaction has been chosen randomly but there is no assurance
74+
that it's representative of the most common Starknet invoke transaction. This
75+
benchmark tests the write performance of visited program counters in the state
76+
struct.
77+
- `execution`: this benchmark simulates a whole invoke transaction using a dummy
78+
contract.
79+
80+
## Performance Impact
81+
82+
The `bench.sh` script has been added to benchmark the performance impact of
83+
these changes.
84+
85+
The benchmark results presented below were conducted under the following
86+
conditions:
87+
88+
- **Operating System:** Debian 12 (Bookworm) running in a VMWare Workstation 17
89+
VM on Windows 10 22H2
90+
- **Hardware:** i9-9900K @ 5.0 GHz, 64GB of RAM, Samsung 990 Pro NVMe SSD.
91+
- **Rust Toolchain:** 1.78-x86_64-unknown-linux-gnu / rust 1.78.0 (9b00956e5
92+
2024-04-29).
93+
94+
The script was called as follows, but you may need to [adjust the commit
95+
hashes](#after-changes) in question to reproduce these results:
96+
97+
`bash scripts/bench.sh 14e6a87722c1d0c757b1aa2756ffabe3f248fd7d e39ae0be4cec31938399199e0a1070279b4a78ed`
98+
99+
The noise threshold and confidence intervals are kept as per default
100+
Criterion.rs configuration.
101+
102+
The results are as follows:
103+
104+
| Benchmark | Time (ms) | Time change (%) | Criterion.rs report |
105+
| ------------ | --------- | --------------- | ----------------------------- |
106+
| transfers | 94.448 | +0.1080 | No change in performance |
107+
| execution | 1.2882 | -1.7216 | Change within noise threshold |
108+
| cached_state | 5.2330 | -0.8703 | No change in performance |
109+
110+
Criterion's inbuilt confidence analysis suggests that these results have no
111+
statistical significant and do not represent real-world performance changes.

docs/BENCHMARK.md

-93
This file was deleted.

0 commit comments

Comments
 (0)