Skip to content

mpcs e2e batch in one proof #894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 64 commits into from
Apr 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
5aced4a
improve batching sumcheck parallism dominated by max num var
hero78119 Apr 3, 2025
6f7c695
pass on normal poly logic
hero78119 Apr 3, 2025
3a83908
all compile & test pass
hero78119 Apr 3, 2025
93b38f6
refactor unittest
hero78119 Apr 3, 2025
73c2396
rename api
hero78119 Apr 3, 2025
46fdbf2
sumcheck refactor and rename
hero78119 Apr 4, 2025
3f07940
all test passed
hero78119 Apr 4, 2025
61a3bf8
fix mpcs sumcheck usage
hero78119 Apr 4, 2025
c6aa233
more comment
hero78119 Apr 4, 2025
fcba46e
rewrite unittest to fit into arbitrary cores
hero78119 Apr 4, 2025
2138b52
fmt
hero78119 Apr 4, 2025
8eb778a
take reference
hero78119 Apr 4, 2025
83a6b6b
mpcs e2e batch in one proof
hero78119 Apr 2, 2025
3103777
minor comment
hero78119 Apr 2, 2025
dcf5713
wip
hero78119 Apr 4, 2025
7f760b3
build pass
hero78119 Apr 4, 2025
f649327
fixed batch commit
hero78119 Apr 5, 2025
1ea218c
transcript sample bits
hero78119 Apr 7, 2025
125d092
wip
hero78119 Apr 7, 2025
9840fdf
simplify design without transcript fork
hero78119 Apr 7, 2025
9fcd535
e2e build pass
hero78119 Apr 7, 2025
b2ea8fc
fix various issue in e2e test
hero78119 Apr 7, 2025
2b3c7ab
e2e prover run through
hero78119 Apr 8, 2025
c7224a8
debug log
hero78119 Apr 8, 2025
533f5d8
use circuit index to trace fixed and witin poly & codeword
hero78119 Apr 8, 2025
91027cc
support trivial proof and more span
hero78119 Apr 8, 2025
f3c5b22
recover parallism
hero78119 Apr 8, 2025
57ec903
code cleanup
hero78119 Apr 9, 2025
0670ebb
adjust verifier batch api
hero78119 Apr 9, 2025
e712182
move num_instances to root proof level
hero78119 Apr 9, 2025
005efdc
unify non-uniform witness in proof
hero78119 Apr 9, 2025
999d549
batched verifier pass til query
hero78119 Apr 9, 2025
78db664
query base query proof verify pass
hero78119 Apr 10, 2025
cee71bc
verifier batch oracle query preparation
hero78119 Apr 10, 2025
ca8945d
almost finish query logic
hero78119 Apr 10, 2025
a212782
final message check pass
hero78119 Apr 11, 2025
3a1f931
fix idx shifting bug
hero78119 Apr 11, 2025
c7a1456
cleanu
hero78119 Apr 11, 2025
16e2084
cleanup commitment meta_info
hero78119 Apr 11, 2025
f614188
write trivial commit to transcripts
hero78119 Apr 11, 2025
717d43e
all check pass on batch verifier
hero78119 Apr 12, 2025
0a58d5c
proof.bin, vk.bin to gitignore
hero78119 Apr 12, 2025
6bb35a5
clippy fix
hero78119 Apr 12, 2025
dbb4c8c
merge with master
hero78119 Apr 12, 2025
e5cfeba
support optional fixed commit
hero78119 Apr 12, 2025
f4eea8b
code cosmetics
hero78119 Apr 12, 2025
fa3c7d6
chores: remove bh wordings and only keep evals
hero78119 Apr 15, 2025
36c1665
chores: rename oracle -> codeword
hero78119 Apr 15, 2025
efbcb07
sorted by codeword length in dec then aggregate
hero78119 Apr 15, 2025
69c2ba0
codeword dequeue pass by ownership
hero78119 Apr 15, 2025
3692bb2
merge with master
hero78119 Apr 16, 2025
339b2ba
merge with master
hero78119 Apr 16, 2025
99247b7
chores: cleanup invalid comments
hero78119 Apr 16, 2025
d957832
chores: comments for trivial_proofdata
hero78119 Apr 16, 2025
8edd0cc
Revert "chores: cleanup invalid comments"
hero78119 Apr 16, 2025
f91fa86
chores: fix comment
hero78119 Apr 16, 2025
f1c9977
cleanup btreemap from verifier to avoid soundness due to inconsistency
hero78119 Apr 16, 2025
50e390e
remove circuit_index from mpcs commitment
hero78119 Apr 16, 2025
30ff51a
address review feedback: function reused and better styling
hero78119 Apr 17, 2025
88ebfbb
optimize codeword fold with aggregation first
hero78119 Apr 17, 2025
1e403e8
unify sumcheck message with same struct
hero78119 Apr 17, 2025
9f74164
robust length check
hero78119 Apr 17, 2025
da7d142
named struct in favour of complex type
hero78119 Apr 17, 2025
3c20a04
more comments
hero78119 Apr 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ p3-challenger = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9
p3-commit = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
p3-dft = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
p3-field = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
p3-fri = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
p3-goldilocks = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
p3-matrix = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
p3-maybe-rayon = { git = "https://github.com/scroll-tech/plonky3", rev = "20c3cb9" }
Expand Down
12 changes: 6 additions & 6 deletions ceno_zkvm/benches/riscv_add.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use std::time::Duration;
use std::{collections::BTreeMap, time::Duration};

use ceno_zkvm::{
self,
Expand Down Expand Up @@ -73,14 +73,16 @@ fn bench_add(c: &mut Criterion) {
for _ in 0..iters {
// generate mock witness
let num_instances = 1 << instance_num_vars;
let rmm =
RowMajorMatrix::rand(&mut OsRng, num_instances, num_witin as usize);
let rmms = BTreeMap::from([(
0,
RowMajorMatrix::rand(&mut OsRng, num_instances, num_witin as usize),
)]);

let instant = std::time::Instant::now();
let num_instances = 1 << instance_num_vars;
let mut transcript = BasicTranscript::new(b"riscv");
let commit =
Pcs::batch_commit_and_write(&prover.pk.pp, rmm, &mut transcript)
Pcs::batch_commit_and_write(&prover.pk.pp, rmms, &mut transcript)
.unwrap();
let polys = Pcs::get_arc_mle_witness_from_commitment(&commit);
let challenges = [
Expand All @@ -91,10 +93,8 @@ fn bench_add(c: &mut Criterion) {
let _ = prover
.create_opcode_proof(
"ADD",
&prover.pk.pp,
circuit_pk,
polys,
commit,
&[],
num_instances,
&mut transcript,
Expand Down
21 changes: 2 additions & 19 deletions ceno_zkvm/src/circuit_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ use serde::de::DeserializeOwned;
use std::{collections::HashMap, iter::once, marker::PhantomData};

use ff_ext::ExtensionField;
use mpcs::PolynomialCommitmentScheme;

use crate::{
ROMType,
Expand All @@ -13,7 +12,6 @@ use crate::{
structs::{ProgramParams, ProvingKey, RAMType, VerifyingKey, WitnessId},
};
use p3::field::PrimeCharacteristicRing;
use witness::RowMajorMatrix;

/// namespace used for annotation, preserve meta info during circuit construction
#[derive(Clone, Debug, Default, serde::Serialize, serde::Deserialize)]
Expand Down Expand Up @@ -178,24 +176,9 @@ impl<E: ExtensionField> ConstraintSystem<E> {
}
}

pub fn key_gen<PCS: PolynomialCommitmentScheme<E>>(
self,
pp: &PCS::ProverParam,
fixed_traces: Option<RowMajorMatrix<E::BaseField>>,
) -> ProvingKey<E, PCS> {
// transpose from row-major to column-major
let fixed_traces_polys = fixed_traces.as_ref().map(|rmm| rmm.to_mles());

let fixed_commit_wd = fixed_traces.map(|traces| PCS::batch_commit(pp, traces).unwrap());
let fixed_commit = fixed_commit_wd.as_ref().map(PCS::get_pure_commitment);

pub fn key_gen(self) -> ProvingKey<E> {
ProvingKey {
fixed_traces: fixed_traces_polys,
fixed_commit_wd,
vk: VerifyingKey {
cs: self,
fixed_commit,
},
vk: VerifyingKey { cs: self },
}
}

Expand Down
6 changes: 5 additions & 1 deletion ceno_zkvm/src/e2e.rs
Original file line number Diff line number Diff line change
Expand Up @@ -675,7 +675,11 @@ pub fn verify(
// print verification statistics like proof size and hash count
let stat_recorder = StatisticRecorder::default();
let transcript = BasicTranscriptWithStat::new(&stat_recorder, b"riscv");
verifier.verify_proof_halt(zkvm_proof.clone(), transcript, zkvm_proof.has_halt())?;
verifier.verify_proof_halt(
zkvm_proof.clone(),
transcript,
zkvm_proof.has_halt(&verifier.vk),
)?;
info!("e2e proof stat: {}", zkvm_proof);
info!(
"hashes count = {}",
Expand Down
4 changes: 2 additions & 2 deletions ceno_zkvm/src/instructions/riscv/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ fn test_multiple_opcode() {
|cs| SubInstruction::construct_circuit(&mut CircuitBuilder::<E>::new(cs)),
);
let param = Pcs::setup(1 << 10).unwrap();
let (pp, _) = Pcs::trim(param, 1 << 10).unwrap();
cs.key_gen::<Pcs>(&pp, None);
let (_, _) = Pcs::trim(param, 1 << 10).unwrap();
cs.key_gen();
}
21 changes: 13 additions & 8 deletions ceno_zkvm/src/keygen.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use std::collections::BTreeMap;

use crate::{
error::ZKVMError,
structs::{ZKVMConstraintSystem, ZKVMFixedTraces, ZKVMProvingKey},
Expand All @@ -12,24 +14,27 @@ impl<E: ExtensionField> ZKVMConstraintSystem<E> {
vp: PCS::VerifierParam,
mut vm_fixed_traces: ZKVMFixedTraces<E>,
) -> Result<ZKVMProvingKey<E, PCS>, ZKVMError> {
let mut vm_pk = ZKVMProvingKey::new(pp, vp);
let mut vm_pk = ZKVMProvingKey::new(pp.clone(), vp);
let mut fixed_traces = BTreeMap::new();

for (c_name, cs) in self.circuit_css {
for (circuit_index, (c_name, cs)) in self.circuit_css.into_iter().enumerate() {
// fixed_traces is optional
// verifier will check it existent if cs.num_fixed > 0
let fixed_traces = if cs.num_fixed > 0 {
vm_fixed_traces
if cs.num_fixed > 0 {
let fixed_trace_rmm = vm_fixed_traces
.circuit_fixed_traces
.remove(&c_name)
.ok_or(ZKVMError::FixedTraceNotFound(c_name.clone()))?
} else {
None
.flatten()
.ok_or(ZKVMError::FixedTraceNotFound(c_name.clone()))?;
fixed_traces.insert(circuit_index, fixed_trace_rmm);
};

let circuit_pk = cs.key_gen(&vm_pk.pp, fixed_traces);
let circuit_pk = cs.key_gen();
assert!(vm_pk.circuit_pks.insert(c_name, circuit_pk).is_none());
}

vm_pk.commit_fixed(fixed_traces)?;

vm_pk.initial_global_state_expr = self.initial_global_state_expr;
vm_pk.finalize_global_state_expr = self.finalize_global_state_expr;

Expand Down
Loading