[Storehouse] 007 Payloadless Checkpoint (v7) by zhangchiqing · Pull Request #8578 · onflow/flow-go

zhangchiqing · 2026-06-09T15:44:30Z

Add Version7 Checkpoint file format (Payloadless).
Compactor will still use V6 checkpointer. V7 support will be added in the future.

coderabbitai · 2026-06-09T15:44:39Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fa95d2dc-ff20-4ea0-9f21-712422a37873

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch leo/payloadless-checkpoint-v7

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

zhangchiqing · 2026-06-09T15:49:27Z

+	// Read the V6 checkpoint fully — the V6 reader already reads the 16 subtrie
+	// part files concurrently. The resulting tries share sub-tries via Go pointer
+	// identity, which lets FromV6Tries memoize and avoid redundant conversion.
+	v6Tries, err := LoadCheckpoint(v6Header, logger)


This loads the entire checkpoint in memory, which requires a lot of memory. An improvement would be streaming the file, iterate each node, if it's a leaf node, then convert into payload less. This could save lots of memory. Skipped for now, because that requires additional code, can be implemented later.

zhangchiqing · 2026-06-09T16:41:28Z

+		return nil, nil
+	}
+	if index > totalSubTrieNodeCount {
+		nodePos := index - totalSubTrieNodeCount


This part is same logic as v6 (getNodeByIndex) but slightly different in style, where v6 extracted it into getTopNodeByIndex, which adds little value. Instead, the logic here for getPayloadlessTopNodeByIndex is inlined.

zhangchiqing · 2026-06-09T16:43:50Z

@@ -0,0 +1,421 @@
+package wal


vimdiff ledger/complete/wal/checkpoint_v6_reader.go ledger/complete/wal/checkpoint_v7_reader.go

This file is similar to v6_reader, except some minimal coding style changes. Some read functions are not copied over but reused, such as readCheckpointHeader, allPartFileExist, readSubTriesFooter etc

zhangchiqing · 2026-06-09T17:33:32Z

@@ -0,0 +1,484 @@
+package wal


vimdiff ledger/complete/wal/checkpoint_v6_writer.go ledger/complete/wal/checkpoint_v7_writer.go

v7 is similar to v6 except some comments changes.

zhangchiqing · 2026-06-09T19:14:16Z

@@ -0,0 +1,372 @@
+package payloadless


mirrored from ledger/complete/mtrie/flattener/encoding.go and ledger/complete/mtrie/flattener/iterator.go

zhangchiqing · 2026-06-09T19:16:47Z

+	encNodeIndexSize = 8
+	encRegCountSize  = 8
+
+	encLeafHashFlagSize = 1


encRegSizeSize and encPayloadLengthSize in v6 checkpoint node (see ledger/complete/mtrie/flattener/encoding.go) is replaced by encLeafHashFlagSize in payloadless

zhangchiqing · 2026-06-09T20:28:12Z

+		return nil, fmt.Errorf("cannot read leaf hash flag: %w", err)
+	}
+
+	flag := scratch[0]


this part is different from readPayloadFromReader

zhangchiqing · 2026-06-09T20:29:32Z

+
+type EncodedTrie struct {
+	RootIndex uint64
+	RegCount  uint64


RegSize is removed

zhangchiqing · 2026-06-09T20:30:54Z

+		encLeafHashFlagSize +
+		encLeafHashSize


encPayloadLengthSize + encPayloadSize is replaced by encLeafHashFlagSize + encLeafHashSize in payloadless

zhangchiqing · 2026-06-09T20:32:01Z

+	copy(buf[pos:], path[:])
+	pos += encPathSize
+
+	// Encode leaf hash flag (1 byte) and optional leaf hash (0 or 32 bytes)


this part is different from full trie node's encodeLeafNode

zhangchiqing · 2026-06-09T21:49:16Z

 	}

-	lastCheckpointNum, err := c.checkpointer.LatestCheckpoint()
+	lastCheckpointNum, err := c.checkpointer.LatestCheckpointV6()


The compactor will eventually need to support both V6 and V7, but this PR only supports V6. To avoid ambiguity, I renamed the function to LatestCheckpointV6.

payloadless compactor

zhangchiqing commented Jun 9, 2026

View reviewed changes

zhangchiqing force-pushed the leo/payloadless-checkpoint-v7 branch from 7b14692 to 2625248 Compare June 9, 2026 21:05

zhangchiqing commented Jun 9, 2026

View reviewed changes

zhangchiqing marked this pull request as ready for review June 9, 2026 21:52

zhangchiqing requested a review from a team as a code owner June 9, 2026 21:52

zhangchiqing force-pushed the leo/payloadless-execution-builder branch from d7dea94 to fcc22e5 Compare June 18, 2026 23:18

zhangchiqing force-pushed the leo/payloadless-checkpoint-v7 branch from c25cf48 to 267d209 Compare June 18, 2026 23:19

zhangchiqing force-pushed the leo/payloadless-execution-builder branch from fcc22e5 to f138980 Compare June 19, 2026 00:00

zhangchiqing force-pushed the leo/payloadless-checkpoint-v7 branch from 267d209 to 8d341b5 Compare June 19, 2026 00:00

zhangchiqing force-pushed the leo/payloadless-execution-builder branch from f138980 to fc173be Compare July 2, 2026 18:01

zhangchiqing force-pushed the leo/payloadless-checkpoint-v7 branch 2 times, most recently from c77e8e9 to 1ea0be7 Compare July 2, 2026 18:06

zhangchiqing added 8 commits July 2, 2026 11:23

add v7 checkpointer

256cd6a

add checkpoint v7

9f417af

add checkpoint v6->v7 converter

2480076

reduce diff between checkpoint v7 and checkpoint v6

df0b657

update checkpoint cmd util

16687f9

refactor to minimize the change

4915e48

rename LatestCheckpoint to LatestCheckpointV6

70e2e6c

add payloadless WAL

e7300df

zhangchiqing added 3 commits July 2, 2026 11:23

rename v6 checkpointer related functions

832ae63

update logger

47d0f42

Automatic V6→V7 root-checkpoint conversion at bootstrap, and refactor

ea43efc

payloadless compactor

zhangchiqing force-pushed the leo/payloadless-checkpoint-v7 branch from 1ea0be7 to ea43efc Compare July 2, 2026 18:23

Uh oh!

Conversation

zhangchiqing commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhangchiqing Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhangchiqing Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

zhangchiqing commented Jun 9, 2026 •

edited

Loading

coderabbitai Bot commented Jun 9, 2026 •

edited

Loading

zhangchiqing Jun 9, 2026 •

edited

Loading

zhangchiqing Jun 9, 2026 •

edited

Loading