Add daily block execution

[JulianGCalderon]

Closes #850

You can find an example on how the workflow looks like here

It should work better once lambdaclass/starknet-replay#78 is merged

[github-actions]

Benchmark results Main vs HEAD.

Command	Mean [s]	Min [s]	Max [s]	Relative
head factorial_2M.cairo (JIT)	3.319 ± 0.028	3.269	3.346	1.00 ± 0.01
base factorial_2M.cairo (JIT)	3.310 ± 0.019	3.284	3.344	1.00
head factorial_2M.cairo (AOT)	3.314 ± 0.019	3.282	3.349	1.00 ± 0.01
base factorial_2M.cairo (AOT)	3.321 ± 0.035	3.276	3.368	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
head fib_2M.cairo (JIT)	2.846 ± 0.023	2.809	2.890	1.01 ± 0.01
base fib_2M.cairo (JIT)	2.812 ± 0.028	2.783	2.857	1.00 ± 0.01
head fib_2M.cairo (AOT)	2.811 ± 0.025	2.780	2.860	1.00
base fib_2M.cairo (AOT)	2.813 ± 0.013	2.799	2.834	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
head logistic_map.cairo (JIT)	3.138 ± 0.009	3.121	3.150	1.05 ± 0.01
base logistic_map.cairo (JIT)	3.151 ± 0.013	3.129	3.168	1.06 ± 0.01
head logistic_map.cairo (AOT)	2.977 ± 0.017	2.955	3.016	1.00
base logistic_map.cairo (AOT)	2.978 ± 0.018	2.959	3.014	1.00 ± 0.01

[github-actions]

Benchmarking results

Benchmark for program factorial_2M

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	10.849 ± 0.276	10.519	11.268	23.62 ± 0.60
cairo-native (embedded AOT)	3.140 ± 0.018	3.116	3.171	6.84 ± 0.04
cairo-native (embedded JIT using LLVM's ORC Engine)	3.172 ± 0.022	3.144	3.215	6.91 ± 0.05
cairo-native (standalone AOT)	0.662 ± 0.002	0.659	0.666	1.44 ± 0.01
cairo-native (standalone AOT with -march=native)	0.459 ± 0.001	0.458	0.460	1.00

Benchmark for program fib_2M

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	10.906 ± 0.230	10.435	11.179	1475.78 ± 32.46
cairo-native (embedded AOT)	2.678 ± 0.015	2.659	2.704	362.42 ± 3.01
cairo-native (embedded JIT using LLVM's ORC Engine)	2.721 ± 0.012	2.705	2.738	368.13 ± 2.85
cairo-native (standalone AOT)	0.007 ± 0.000	0.007	0.008	1.00
cairo-native (standalone AOT with -march=native)	0.008 ± 0.000	0.008	0.009	1.09 ± 0.01

Benchmark for program logistic_map

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	4.292 ± 0.019	4.256	4.327	59.99 ± 0.31
cairo-native (embedded AOT)	2.857 ± 0.013	2.841	2.879	39.93 ± 0.21
cairo-native (embedded JIT using LLVM's ORC Engine)	3.008 ± 0.022	2.978	3.044	42.04 ± 0.33
cairo-native (standalone AOT)	0.116 ± 0.000	0.115	0.117	1.62 ± 0.01
cairo-native (standalone AOT with -march=native)	0.072 ± 0.000	0.071	0.072	1.00

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.91%. Comparing base (ab47832) to head (6bfa6c5).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #888   +/-   ##
=======================================
  Coverage   82.91%   82.91%           
=======================================
  Files         120      120           
  Lines       33866    33866           
=======================================
  Hits        28081    28081           
  Misses       5785     5785           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[FrancoGiachetta]

Since this is meant to be run during night, I don't really know if this could make a significant impact on the time it takes to complete the ci once we increased the amount of blocks to execute. But it might be a good idea to run both native and the vm at the same time to reduce the execution time, right?.

[JulianGCalderon]

I considered it, but I think is not that simple. As I understand it, we have two possible approaches:

The first approach (used in the starknet-blocks workflow) is to have two separate jobs, one uses a matrix to execute different blocks with native and VM, then uploads an artifact with the state dumps. Another job depends on the first one, retreives the artifact, and compares the states.

The downside with this approach is that if one of the executions fails, then the compare job is never ran. This is not desirable, as an RPC error is pretty common when running so many blocks, and would cancel the whole daily execution. There might be a way to work around this, but I don't know how.

This is why I opted for a different approach. Use a matrix, but each job in the matrix is complete and independent from the other ones. A failure in one execution would not cancel the other ones, as they are completely separate. The comparison would not have to wait for the whole execution, as each subcomparison could start as soon as the corresponding execution finishes.

The downside of this is that we cannot run VM and Native at the same time, but I don't think that it matters to much, as we can always increase the amount of jobs in the matrix if we want more parallelism.