|
| 1 | +# Adding the `VisitedPcs` Trait |
| 2 | + |
| 3 | +The state of the blockifier as of commit |
| 4 | +`14e6a87722c1d0c757b1aa2756ffabe3f248fd7d` doesn't store the complete vector of |
| 5 | +visited program counters for each entry-point in an invoke transaction. Instead, |
| 6 | +visited program counters are pushed into a `HashSet`. Unfortunately this limits |
| 7 | +the ability to perform profiling operations, as many require a record of the |
| 8 | +full trace returned from the `cairo-vm`. |
| 9 | + |
| 10 | +In order to enable more in-depth tracing use-cases, we have introduced the |
| 11 | +`VisitedPcs` trait which allows the user to process the visited program counters |
| 12 | +as they see fit. |
| 13 | + |
| 14 | +## Before Changes |
| 15 | + |
| 16 | +Visited program counters are kept in the `CachedState` structure as shown below: |
| 17 | + |
| 18 | +```rust |
| 19 | +#[derive(Debug)] |
| 20 | +pub struct CachedState<S: StateReader> { |
| 21 | + pub state: S, |
| 22 | + // Invariant: read/write access is managed by CachedState. |
| 23 | + // Using interior mutability to update caches during `State`'s immutable getters. |
| 24 | + pub(crate) cache: RefCell<StateCache>, |
| 25 | + pub(crate) class_hash_to_class: RefCell<ContractClassMapping>, |
| 26 | + /// A map from class hash to the set of PC values that were visited in the class. |
| 27 | + pub visited_pcs: HashMap<ClassHash, HashSet<usize>>, |
| 28 | +} |
| 29 | +``` |
| 30 | + |
| 31 | +This snipped has been extracted from commit |
| 32 | +[14e6a87722c1d0c757b1aa2756ffabe3f248fd7d](https://github.com/reilabs/blockifier/blob/14e6a87722c1d0c757b1aa2756ffabe3f248fd7d/crates/blockifier/src/state/cached_state.rs#L36) |
| 33 | + |
| 34 | +## After Changes |
| 35 | + |
| 36 | +> [!NOTE] |
| 37 | +> The new code is developed in the branch `visited_pcs_trait` and the |
| 38 | +> current head of the branch is at commit |
| 39 | +> [`bdb1b49331aad91d445ac2155baa40fa783bcf7f`](https://github.com/reilabs/blockifier/blob/visited_pcs_trait/crates/blockifier/src/state/cached_state.rs#L37). |
| 40 | +> This will change once these changes are merged in the main branch. |
| 41 | +
|
| 42 | +`VisitedPcs` is added as an additional generic parameter of `CachedState`. |
| 43 | + |
| 44 | +```rust |
| 45 | +#[derive(Debug)] |
| 46 | +pub struct CachedState<S: StateReader, V: VisitedPcs> { |
| 47 | + pub state: S, |
| 48 | + // Invariant: read/write access is managed by CachedState. |
| 49 | + // Using interior mutability to update caches during `State`'s immutable getters. |
| 50 | + pub(crate) cache: RefCell<StateCache>, |
| 51 | + pub(crate) class_hash_to_class: RefCell<ContractClassMapping>, |
| 52 | + /// A map from class hash to the set of PC values that were visited in the class. |
| 53 | + pub visited_pcs: V, |
| 54 | +} |
| 55 | +``` |
| 56 | + |
| 57 | +An implementation of the trait `VisitedPcs` is included in the blockifier with |
| 58 | +the name `VisitedPcsSet`. This mimics the existing `HashSet<usize>` usage of |
| 59 | +this field. For test purposes, `CachedState` is instantiated using |
| 60 | +`VisitedPcsSet`. |
| 61 | + |
| 62 | +## Performance Considerations |
| 63 | + |
| 64 | +Given the importance of the blockifier's performance in the Starknet ecosystem, |
| 65 | +measuring the impact of adding the aforementioned `VisitedPcs` trait is very |
| 66 | +important. The existing bechmark `transfers` doesn't cover operations that use |
| 67 | +the `CachedState`, and therefore we have designed new ones as follows: |
| 68 | + |
| 69 | +- `cached_state`: this benchmark tests the performance impact of populating |
| 70 | + `visited_pcs` (implemented using `VisitedPcsSet`) with a realistic amount of |
| 71 | + visited program counters. The size of the sets is taken from transaction |
| 72 | + `0x0177C9365875CAA840EA8F03F97B0E3A8EE8851A8B952BF157B5DBD4FECCB060` on |
| 73 | + mainnet. This transaction has been chosen randomly but there is no assurance |
| 74 | + that it's representative of the most common Starknet invoke transaction. This |
| 75 | + benchmark tests the write performance of visited program counters in the state |
| 76 | + struct. |
| 77 | +- `execution`: this benchmark simulates a whole invoke transaction using a dummy |
| 78 | + contract. |
| 79 | + |
| 80 | +## Performance Impact |
| 81 | + |
| 82 | +The `bench.sh` script has been added to benchmark the performance impact of |
| 83 | +these changes. |
| 84 | + |
| 85 | +The benchmark results presented below were conducted under the following |
| 86 | +conditions: |
| 87 | + |
| 88 | +- **Operating System:** Debian 12 (Bookworm) running in a VMWare Workstation 17 |
| 89 | + VM on Windows 10 22H2 |
| 90 | +- **Hardware:** i9-9900K @ 5.0 GHz, 64GB of RAM, Samsung 990 Pro NVMe SSD. |
| 91 | +- **Rust Toolchain:** 1.78-x86_64-unknown-linux-gnu / rust 1.78.0 (9b00956e5 |
| 92 | + 2024-04-29). |
| 93 | + |
| 94 | +The script was called as follows, but you may need to [adjust the commit |
| 95 | +hashes](#after-changes) in question to reproduce these results: |
| 96 | + |
| 97 | +`bash scripts/bench.sh 14e6a87722c1d0c757b1aa2756ffabe3f248fd7d e39ae0be4cec31938399199e0a1070279b4a78ed` |
| 98 | + |
| 99 | +The noise threshold and confidence intervals are kept as per default |
| 100 | +Criterion.rs configuration. |
| 101 | + |
| 102 | +The results are as follows: |
| 103 | + |
| 104 | +| Benchmark | Time (ms) | Time change (%) | Criterion.rs report | |
| 105 | +| ------------ | --------- | --------------- | ----------------------------- | |
| 106 | +| transfers | 94.448 | +0.1080 | No change in performance | |
| 107 | +| execution | 1.2882 | -1.7216 | Change within noise threshold | |
| 108 | +| cached_state | 5.2330 | -0.8703 | No change in performance | |
| 109 | + |
| 110 | +Criterion's inbuilt confidence analysis suggests that these results have no |
| 111 | +statistical significant and do not represent real-world performance changes. |
0 commit comments