spec discussion: commitment of shard segments to global history #267
Conversation
nazar-pc
left a comment
nazar-pc
left a comment
There was a problem hiding this comment.
Like in other specs, I think this is a start, but I don't see enough depth to answer all the questions I have. Some I left inline, but more globally I'd like to see a rationale behind every field that exists in every data structure.
For example why do we need something like SuperSegmentHeader at all, literally. And if we do in fact need it, then how we would use its contents.
Similarly, I don't see the full workflow of how farmer would pick pieces from the global history and how that choice can be verified on chain with a compact proof. I tried to infer it from the available details, but didn't succeed so far.
On more stylistic note, I may have not slept well, but I find the long-form sentences with repetition quite hard to read. I need to re-read each several times to distill the essence. Tried to provide a few specific examples of how I would phrase it based on my current understanding, hopefully it is useful.
| > However, to avoid referencing blocks that are too recent (and might not be visible to all nodes, | ||
| > increasing the risk of rejection), the reference implementation will use a fixed. Specifically, | ||
| > all nodes will aim to propose the most recent valid block they see in their view of the beacon | ||
| > chain `1 / SLOT_PROBABILITY` in the past (which is currently set to 1/6, i.e. 6 slots which is ~6 |
There was a problem hiding this comment.
I took this number from the spec and it says ~6 seconds: https://subspace.github.io/protocol-specs/docs/consensus/proof_of_archival_storage#protocol-constants
(maybe I misunderstood something there)
There was a problem hiding this comment.
Yes, my intuition what that 6 is the amount of time given, so maybe it needs to be that +1 for the beacon chain root inclusion to make sure it is strictly outside of this 6 second interval. No strong opinion though and maybe I'm just confusing myself with off-by-one here somehow.
| > all nodes will aim to propose the most recent valid block they see in their view of the beacon | ||
| > chain `1 / SLOT_PROBABILITY` in the past (which is currently set to 1/6, i.e. 6 slots which is ~6 | ||
| > seconds). With this, farmers in a shard will try to set their blockchain reference to the most | ||
| > recent valid beacon chain block. Failing to propose a block reference from the beacon chain that | ||
| > is 6 slots behind would mean that the farmer has fell out of sync. This forces shard farmers to be | ||
| > as up-to-date as possible with the beacon chain, while keeping as recent as possible references in | ||
| > shard block (which will help with the linking and verification of information between shards). |
There was a problem hiding this comment.
I would rewrite this by simply defining what a valid reference is. "aiming" is not the right terminology here. Essentially for every slot produced on lower-level shard there is one and only one valid beacon chain block that must be included. Failure to include it makes block invalid.
There was a problem hiding this comment.
There is a danger with this though: an attacker may withhold blocks for extra seconds, causing invalid blocks to be produced on lower-level shards. This probably means we should tweak the depth of included block from 6-7 to something larger. The idea being that if block production is 6 seconds on average on the beacon chain, delaying block announcement by one slot might actually not make that much of a difference (statistically speaking).
Or maybe I'm just hallucinating here, someone with a better grasp of probabilities should chime in.
Co-authored-by: Nazar Mokrynskyi <[email protected]>
Note: As with all spec discussions, this PR is not meant to be merged
This PR shares the draft of the end-to-end commitment of segments from shards to the global history of the beacon chain. The goal of the discussion is to agree on the low-level details of this protocol mechanism and surface any blind posts or things that need to be defined more clearly in order to be able to prototype it.
Coming next
As described in the last section of the discussion, designing the membership allocation protocol (or at least having a basic one) will be key to address the issue of verifying the availability and correctness of segments before they are included to the global history and archived. I will focus on detailing that mechanism first before I come back to that section.