mpcs e2e batch in one proof

[hero78119]

Build on top of #901.

Design rationale

zkvm + mpcs api: use batch_commit/batch_opening to commit/open ALL opcode together. In more detail, we commit fixed_commit and witin_commit separately with just one opening.
basefold: batch opcodes with rmm with different "height", in other words, different "num_vars".

For basefold to batch opcodes with different variables, for sumcheck part we apply suffix alignment techniques based on
#870. For FRI part, we also apply suffix alignment techniques. So the smaller codeword will "involved" into the folding process when current length match.

During implementation, there is key principle in mind: verifier only rely on one untrusted information "num_instances" from prover, and other information should derived from verifier key.

Working items

• mpcs batch api + batch basefold prover
• batch basefold verifier

Fibonacci Benchmark Results

Size	Branch	Proof Size	E2E Latency Change	Prover Speed (kHz)
2²⁰	Master	14.68 MB	-	363
2²⁰	Batched + conjecture	1.16 MB	-14.195%	422
2²⁰	Batched	2.02 MB	-11.599%	411
2²¹	Master	15.53 MB	-	411
2²¹	Batched + conjecture	1.24 MB	-11.178%	451
2²¹	Batched	2.16 MB	-6.4525%	428
2²²	Master	16.42 MB	-	406
2²²	Batched + conjecture	1.31 MB	-11.178%	433
2²²	Batched	2.31 MB	-13.691%	448

[kunxian-xia]

The batch open feature is very flexible, as we open multilinear polynomials of different number of variable (nv) at different points. I'm afraid it's hard to prove the soundness of our current batch open protocol.

Therefore I propose to add one more additional step to reduce opening at multiple points to opening at one single point. #904

We may want to divide this procedure into 3 steps:

1. for each nv, run a sumcheck to reduce the evaluations of MLPs (stored in matrices) on different points $z_i$ to the evaluation on single point $\alpha$ (check the proposal introduced in HyperPlonk paper reduce evaluations of 2^k MLPs that open at points (possibly distinct) to evaluations on one single point #904).
2. for each nv, randomly combine MLPs into one MLP, and then open it on single point $\alpha$, we get an iterator of tuple $(p(x), \alpha)$;
3. In the commit phase, in each step, pop one tuple from the above iterator, do the folding around challenge $r$, and add the next item in the iterator if its nv + 1 equals to p(x)'s nv.

[hero78119]

Updated: to make verifier more robust, in latest 2 commits f1c9977
50e390e all BTreeMap related are removed and convert to pure vector, as all ordering can deterministic derived from global information num_instances

[kunxian-xia]

soundness: we need to return error if proof.query_opening_proof.len() != queries.len())

[kunxian-xia]

Great work!