-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add benchmarks comparing to Cairo VM #692
Conversation
Benchmark results Main vs HEAD.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #692 +/- ##
=======================================
Coverage 82.54% 82.54%
=======================================
Files 117 117
Lines 33516 33516
=======================================
+ Hits 27665 27666 +1
+ Misses 5851 5850 -1 ☔ View full report in Codecov by Sentry. |
benches/benches.rs
Outdated
|
||
factorial_group.bench_function("VM", |b| { | ||
b.iter(|| { | ||
factorial_runner.run_function_with_starknet_context( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to find another way to execute the VM.
See the fn run_function_with_starknet_context
let assembled_program = self.casm_program.clone().assemble_ex(&entry_code, &footer);
It clone the casm_program adding lot of overhead to the execution
We need to discuss how we do this, maybe we have to add a new method to the cairo-vm/cairo1-run
crate
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that particular issue has been fixed: https://github.com/starkware-libs/cairo/blob/226c3646be247828c08b737a73f6da10de76e0fc/crates/cairo-lang-runner/src/lib.rs#L191
let assembled_cairo_program = self.casm_program.assemble_ex(&header, &footer);
Other thing to take into account, maybe is easier to benchmark a contract execution |
284872c
to
f21ffda
Compare
159bbca
to
dbb6fc7
Compare
This PR benchmarks cairo native against cairo vm with criterion:
Compile time is not taken into account for cairo native runs.
Benchmarking Server
Factorial:
Fibonacci:
Logistic Map:
Macbook Air M1