chore: Added a process document for Block Stream cutover#2193
chore: Added a process document for Block Stream cutover#2193jsync-swirlds wants to merge 2 commits intomainfrom
Conversation
jsync-swirlds
force-pushed
the
create-cutover-documentation
branch
4 times, most recently
from
9781360 to
f5f25fc
Compare
jsync-swirlds
requested review from
AlfredoG87,
Nana-EC,
ata-nas,
berryware and
ivannov
jsync-swirlds
force-pushed
the
create-cutover-documentation
branch
from
f5f25fc to
47795c6
Compare
Nana-EC
left a comment
Nana-EC
left a comment
There was a problem hiding this comment.
Thanks for putting this together.
Left a few comments for consideration
docs/block-node/Cutover-Process.md
Outdated
| ## Goals | ||
| * Establish new WRAPS keys and parameters via a "Powers of Tau" ceremony | ||
| executed on existing trusted network nodes and produce publicly verifiable | ||
| outputs. |
There was a problem hiding this comment.
Instead of "outputs" can we list the outputs here so the goal is more clear.
There was a problem hiding this comment.
I don't have an exhaustive list.
There is a HIP that defines the whole ceremony process.
| outputs. | ||
| * Provide a clear structure to migrate an existing public network to the | ||
| HinTS TSS signature process and WRAPS address book proofs. | ||
| * Provide a clear structure to migrate an existing public network from access |
There was a problem hiding this comment.
Q: Instead of "access" should we instead say "retrieving" or "requesting" to highlight that previously consumers pulled record streams but will now pull block streams?
| ### Definitions and Acronyms | ||
| <dl> | ||
| <dt>Wrapped Record Block (WRB)</dt><dd> | ||
| A block generated by wrapping record stream content in a Block Item, |
There was a problem hiding this comment.
| A block generated by wrapping record stream content in a Block Item, | |
| A block generated by wrapping record file contents in a Block Item, |
There was a problem hiding this comment.
I tried to consistently use record stream to refer to contents; mostly because we extract the record stream data from the file rather than wrapping the file exactly as-is (necessary to have a consistent hash for the blocks).
| adding Header and Footer details, and generating a Block Proof based | ||
| on the individual node signatures produced for that Record File.</dd> | ||
| <dt>Jumpstart File</dt><dd> | ||
| A file containing the Block Hash for a specific WRB, the state of the |
There was a problem hiding this comment.
Can you clarify "state" I believe it's only the necessary rightmost hashes or some other subset of the tree
There was a problem hiding this comment.
Added a second sentence to clarify what the historical block hash tree state is.
docs/block-node/Cutover-Process.md
Outdated
| of that block.</dd> | ||
| <dt>WRB Catch Up</dt><dd> | ||
| A process where the Consensus Node Software combines a Jumpstart file with | ||
| the Timestamp and root hashes for subtrees 3-8 to produce a valid WRB hash. |
There was a problem hiding this comment.
Something like this to note that the process must continue off of the block noted in the jumpstart file
| the Timestamp and root hashes for subtrees 3-8 to produce a valid WRB hash. | |
| the Timestamp and root hashes for subtrees 3-8 to produce a valid WRB hash for the block after the block noted in the jumpstart file. |
There was a problem hiding this comment.
Added, slightly reworded to for the next block following the jumpstart block.
docs/block-node/Cutover-Process.md
Outdated
| * Release N | ||
| * Consensus Node generates WRB hashes | ||
| * Consensus Node stores WRB hash data on disk | ||
| * WRAPS proving key and parameters downloaded by Consensus Node |
There was a problem hiding this comment.
Should we highlight where the parameters are downloaded from?
There was a problem hiding this comment.
We're hoping to have multiple options, not just the current Google Registry location.
| * Release K (<= N) | ||
| * Include the "Powers of Tau" ceremony program with the Consensus Node | ||
| release package. | ||
| * Ensure the "Powers of Tau" ceremony executes during the duration of the |
There was a problem hiding this comment.
Remind readers of the output of the ceremony
docs/block-node/Cutover-Process.md
Outdated
| * "Offline" process downloading record files from S3 | ||
| * Generate WRB blocks from record files | ||
| * WRB files added to "RBH" block node | ||
| * Produce Jumpstart file every 1000 blocks |
There was a problem hiding this comment.
1000 could change no? I imagine it's configureable so maybe just note every configureable batch of record files.
Unless there's an importance to the choice of 1000
There was a problem hiding this comment.
Clarified that it's occasional and configurable.
docs/block-node/Cutover-Process.md
Outdated
| * Generate WRB blocks from record files | ||
| * WRB files added to "RBH" block node | ||
| * Produce Jumpstart file every 1000 blocks | ||
| * Block Nodes _test_ backfill from RBH, but do not store backfilled WRBs |
There was a problem hiding this comment.
I like the note of test. Maybe pull it out to a section under the Release.
So you'd have "Operations" & "Tests" to highlight what must occur and what tests can be used to validate them
There was a problem hiding this comment.
This document does not try to specify exact technical detail and process (as that would require constant updates), and this line does not specify any particular test or testing process.
This item is noting that Block Nodes during this release can backfill from RBH for testing purposes, but do not store the blocks backfilled from that source.
docs/block-node/Cutover-Process.md
Outdated
| * Consensus Node stores the correct WRB hash in BlockInfo in state at the | ||
| end of each block. | ||
| * WRB hash and historical tree hashes are used when creating the block | ||
| for the preview block stream blocks. Effectively, this makes each |
There was a problem hiding this comment.
Can you elaborate. What's the detail you're stressing beyond the block is more complete. Is it that each new block stream styles block relates to the last created record file? Are you highlighting for information only or potential action by the term "preview cutover block"?
There was a problem hiding this comment.
This lacks context so I'm not sure the question.
The term preview cutover block is not used anywhere in the document.
There won't be any connection between preview blocks and the WRB's; it was determined that this is not helpful and creates more risk than it removes.
jsync-swirlds
force-pushed
the
create-cutover-documentation
branch
4 times, most recently
from
bca0756 to
2b9755a
Compare
docs/block-node/Cutover-Process.md
Outdated
| section Block Streams | ||
| Preview Block Streams with incomplete hashes | ||
| : Preview Block Streams with incomplete hashes continue | ||
| : Preview Block Streams with latest wrapped block hash as "previous block hash". Each preview block, then, is effectively a preview of the cutover block. |
There was a problem hiding this comment.
What advantage does this give us? This PR https://github.com/hiero-ledger/hiero-consensus-node/pull/23674/changes plans to add wrapped record information to the BlockInfo singleton to not disrupt the production of the preview block stream.
There was a problem hiding this comment.
This is leftover from before a late change.
I'll remove it.
jsync-swirlds
force-pushed
the
create-cutover-documentation
branch
2 times, most recently
from
75fdd16 to
05b82e7
Compare
Signed-off-by: Joseph S. <[email protected]>
* Added clarifying items * Adjusted some wording * Fixed a line break issue * Moved timeline before the process description Signed-off-by: Joseph S. <[email protected]>
jsync-swirlds
force-pushed
the
create-cutover-documentation
branch
from
05b82e7 to
2a657bb
Compare
5 participants
Reviewer Notes
There are technical details not directly covered. The primary purpose is to describe the process and timing, without excessive technical detail.
Critical Note
Spotless is, apparently, incapable of processing mixed-mode lists, and therefore demands changes that break the list structure.
Related Issue(s)
Resolves #2178.