diff --git a/SWIPs/swip-pullsync.md b/SWIPs/swip-pullsync.md new file mode 100644 index 0000000..c52b608 --- /dev/null +++ b/SWIPs/swip-pullsync.md @@ -0,0 +1,84 @@ +--- +SWIP: 25 +title: More efficient pull syncing within neighbourhood +author: Viktor Tron <@zelig>, Viktor Tóth <@nugaon> +discussions-to: https://discord.com/channels/799027393297514537/1239813439136993280 +status: Draft +type: +created: 2025-02-24 +--- + + + +## Simple Summary + +This SWIP describes a more efficient way to synchronise content between peers in the same neighbourhood. + +### Glossary +_from the SWIP perspective_ + +- **Pullsync**: A protocol that is responsible for syncing all the chunks that our node needs to store. +- **Proximity Order (PO)**: How many starting bits are common between two addresses. +- **Bin X**: Bin X of a node M contains all the chunks in the network that has X as PO with M. +- **Storage Radius**: Smallest integer `D` such that all chunks in the network whose proximity order with the pivot node address is at least `D` fit into the storage space that node dedicates to its reserve. +- **Neighbourhood** A set of peers in the network which proximity order with the pivot node address is at least `D`. + +## Abstract + +If a node is connected to swarm as a full node, it fires up the pullsync protocol, which is responsible for syncing all the chunks that our node needs to store.Currently the algorithm we use makes sure that on each peer connection, both parties try synchronising their entire reserve. More precisely, each peer start streaming the chunk hashes in batches for each proximity order that is greater or equal to the pull-sync depth (usually the neighbourhood depth). In this proposal, we offer a much more efficient algorithm, still capable of replicating the reserve. + +## Motivation + +Imagine, that a naive peer joins a neighbourhood, then they will 'subscribe to' each +depth of their peers within the neighbourhood. As they are receiving new chunks of course these are offering it too back to the peer they got it from. Plus they try to synchronise from each peer the entire reserve, not just part, which means a naive node's synchronisation involves exchange of `N*S` chunk hashrd where N is the neighbourhood size and S is the size of the reserve. This is hugely inefficient. + +## Specification + +Each peer `P` takes all their peers they are allowed to synchronise with: `p_0, p_1, ..., p_n`. +All chunks need to be syncronized only once. +How about we syncronize each chunks from its closest peer among the neighborhood peers. + +If all the peers we synced from are finished, the respective nodes reserve for any depth equal or higher to storage radius will be the same. + +Unlike the earlier algorithm, this one is extremely sensitive to the changing peerset, so every single time there is a change in the neighbours, pullsync stretegy needs to be reevaluated. + +## Rationale + + +One can see that each chunk is taken from its most immediate neighbourhood only. So depending on to what extent the peer addresses are balanced we save a lot on not taking anything more than once. Imagine a peer with neighbourhood depth `d`, and in the hood 2 neighbours each having a common 2 bit prefix. Their levels in the tree is `d+3` for each peer, and we synchronise chunks closest to them on their `Bin d+3`, `Bin d+4`, `Bin d+5`, etc. The peers share the same parent tree node on level `d+2` therefore their `Bin d+2` is not needed to be synchronised. `Bin d` and `Bin d+1` should contain the same chunks for both peers so each bin can be synchronised with one peer only. +This means the synchronisation is halved for the first 2 levels and one bin is not synchronised at all for the peers that we need to synchronise with the current process in this setting. + +One potential caveat is that if a peer quits or is no longer contactable before the pivot finished syncing with them, then another peer needs to start the process. + +## Backwards Compatibility + +Although it is a major strategic change, the subscription request wire protocol does not change and therefore, the SWIP is backward compatible. + +## Test Cases + +Thorough testing is neeeded, cos this can produce inconsistencies in the localstore and has major impact for retrievebility. + +## Implementation + +In order to find out what nodes share common chunk sets and what are unique ones, a leaf compacted binary tree of addresses from neighborhood peers can be made. The depth of any path extends only as far as is necessary to separate one group of addresses from another. +In this structure, every tree node represents a prefix and each step in the binary tree reflects a further position within the binary representation of the addresses and increments the `level` by 1. +Since the bins must be synchronised only above or equal to storage radius, the root node should represent the common prefix of the neighborhood and initialize the `level` with storage radius. + +Each leaf holds a particular peer $p$ and its `level` is $p$'s uniqueness depth. Conseqently, each chunk sharing the prefix represented by the leaf is closest to $p$. +Each compactible node (i.e. that has one child) is the indication that all the chunks on the missing branch has no single closest peer and are equidistant from two or more peers on the existing branch. + +Ideally To sync all the chunks we need to cover all the branches of the trie: +- all chunks of leaf nodes must be syncronized from its stored peer. +- all chunks on the missing branch of a compactible node must be synced from a peer on the existing branch. + +This is achieved if we traverse the trie in a depth-first manner and for each leaf node we subscribe to all bins greater or equal to its `level`. After then we accumulate peers at the intermediate nodes. While doing this, compatible nodes of level `X` we sync `bin X` from a peer from the accumulated set. + +Note that those tree nodes that have two children of the trie represent prefixes that is fully covered by one of the peers below. + +The assumption behind the loose specification is that we do not need to support for any kind of pull-sync change and existing data flow will be sufficient. In particular, the following assumptions are made: +- pullsync primary indexes the chunks by PO (relative to the node address) +- as secondary ordering within a bin is based on first time of storage. +- the chronology makes it possible to have live (during session) and historical syncing. + +## Copyright/ +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).