diff --git a/docs/Events.md b/docs/Events.md new file mode 100644 index 00000000..e369171b --- /dev/null +++ b/docs/Events.md @@ -0,0 +1,12 @@ +--- +title: Events +--- +**Conferences 2025** + +[CRYPTO](https://crypto.iacr.org/2025/) (August 17-21) + +Participants: Chen-Da Liu-Zhang, Elizabeth Crites, and Alistair Stewart + +[ACNS](https://acns2025.fordaysec.de/) (June 23-26) + +Participants: Jeff Burdges, Elizabeth Crites, Alistair Stewart, and Sergey Vasilyev \ No newline at end of file diff --git a/docs/Polkadot/Polkadot-Crypto.png b/docs/Polkadot/Polkadot-Crypto.png new file mode 100644 index 00000000..3cce4204 Binary files /dev/null and b/docs/Polkadot/Polkadot-Crypto.png differ diff --git a/docs/Polkadot/economics/1-validator-selection.md b/docs/Polkadot/economics/1-validator-selection.md index c63a442a..6ca4ff76 100644 --- a/docs/Polkadot/economics/1-validator-selection.md +++ b/docs/Polkadot/economics/1-validator-selection.md @@ -2,100 +2,94 @@ title: Validator selection --- -**Authors**: [Jonas Gehrlein](/team_members/Jonas.md) + -**Last updated**: 07.12.2020 +Validator elections play a critical role in securing the network, placing nominators in charge of selecting the most trustworthy and competent validators. This responsibility is both complex and demanding. The vast amount of validator data, constantly growing, requires significant technical expertise and sustained engagement. As a result, the process can become overly cumbersome, leading many nominators to either avoid staking altogether or refrain from investing the time needed to evaluate the data thoroughly. In this context, effective tools are essential, not only to support nominators in making informed selections, but also to help ensure the network's long-term health and resilience. -## Introduction +This entry outlines several potential steps to support nominators while preserving their freedom of choice. As a starting point, it is important to highlight why recommendations should consider individual user preferences rather than attempting to make them universal. -The validator elections are essential for the security of the network, where nominators have the important task to evaluate and select the most trustworthy and competent validators. However, in reality this task is quite challenging and comes with significant effort. The vast amount of data on validators (which is constantly increasing) requires a substantial technical expertise and engagement. Currently, the process is too cumbersome and many nominators are either not staking or avoiding spending too much time to go through the large amount of data. Therefore, we need to provide tools, which both aid nominators in the selection process, while still ensuring that the outcome is beneficial for the network. +**Problem.** Providing an exogenous recommendation for a set of validators is not advisable, as user preferences, particularly risk preferences, vary significantly. Comparing metrics accross different scales, such as self-stake in DOT versus performance in percentage, is not feasible in an exogenous framework. Moreover, even when considering a single dimension, the shape of marginal utility functions remains unclear and is inherently tied to individual preferences. Determining the trade-offs involved in the selection process on behalf of nominators lies beyond the scope of this note. Yet, to illustrate this issue, consider the following simple example: -The following write-up provides an overview of several potential steps, which benefit the nominators while maintaining their freedom of choice. As a first step, it is helpful to illustrate why recommendations should be based on user's preferences and cannot be universal for all individuals. - -### Problem -It is not desirable to provide an exogenous recommendation of a set of validators, because user's preferences (especially risk-preferences) are quite different. Therefore, a comparison between metrics on different scales (e.g., self-stake in DOTs vs. performance in %) is not exogenously not possible. In addition, the shape of the marginal utility functions even within one dimension is unclear and based on individual's preferences. It is outside our competence to decide on various trade-offs of the selection process on behalf of nominators. To illustrate this issue, consider the following simple example: | | Commission | Self-Stake | Identity | Era-Points | | -------- | -------- | -------- | -------- | -------- | -| Validator 1 | 4% | 26 DOTs | Yes | Average | -| Validator 2 | 7% | 280 DOTs | No | Average - 1%| +| Validator 1 | 4% | 26 DOT | Yes | Average | +| Validator 2 | 7% | 280 DOT | No | Average - 1%| | Validator 3 | 1% | 1 DOT | No | Average + 5% | -All validators in the table have different profiles, where none is dominated. Validator 3 potentially yield high profits but does not have much self-stake (skin-in-the-game) and is without registered identity. Validator 1 charges a higher fee for their service but might leverage a reputable identity. Validator 2 requires substantial fees but has the most self-stake. One could easily think of different preferences of users, who would prefer any one of those validators. While probably every user could make a choice from that selection, the problem gets increasingly difficult for a set of 200-1000 validators. + +The table above presents validators with diverse profiles, none of which clearly dominate. Validator 3 may offer high potential profits but lacks significant self-stake (skin in the game) and does not have a registered identity. Validator 1 charges a higher service fee, yet may benefit from a reputable identity. Validator 2 has the highest self-stake, but also demands substantial fees. Clearly, user preferences can vary, some may favor one validator over another depending on their priorities. While most users could reasonably make a choice from this small set, the complexity increases when faced with a selection of 200 to 1,000 validators. -### Code of conduct for recommendations -As mentioned before, we cannot and do not want to give an exogenous recommendation to the users. We prefer methods, which values this insight and generates a recommendation based on their stated preferences. While valuing the preferences of the users, we still can *nudge* their decisions in a direction beneficial for the network (e.g., to promote decentralization). Nevertheless, the recommendation should be as objective as possible and should not discriminate against any specific validator. +**Code of conduct for recommendations.** As previously mentioned, the goal is not to provide exogenous recommendations to users, but rather to offer strategies that respect their insight and generate suggestions aligned with their stated preferences. While valuing individual preferences, recommendations may nudge decisions toward outcomes beneficial for the network, such as promoting decentralization. These recommendations should remain as objective as possible and must not discriminate against any specific validator. -### Organization -Validator selection is divided into several chapters. In the sections "Underlying dataset" (Link), we illustrate which data might be useful and how additional metrics can be generated. Afterwards, we can apply a simple concept from economics to significantly reduce the size of potentially intresting validators. Afterwards, This is the first step to give users a way to choose at hand. Then, we discuss some ideas to further curate the set of validators to promote goals of the network. As a last section the UTAStar method illustrates a sophisticated approach to estimate the individual marginal preference functions of the user and make a more precise recommendation. +**Organization.** This entry is divided into several sections. "Underlying data" presents potentially useful data and explains how to derive additional metrics. "Filtering phase" demonstrates how a simple concept from economics can significantly reduce the number of potentially interesting validators, providing users with a more manageable set of choices. The third section explores ideas to further curate the validator set in support of the network's goals. Lastly, the "UTAStar" section outlines a sophisticated approach for estimating each user's individual marginal preference functions, enabling more precise recommendations. -# Underlying Data -This section explains which data can be gathered about validators in Polkadot and Kusama and are relevant for a selection process. Those metrics indicated with a * are used in the final data-set, the other variables are used to generate additional metrics. Currently, we focus on quantitative on-chain data as those are verifiable and easy to process. This purely quantitative approach should be regarded as complementary to a selection process based on qualitative data, where nominators are e.g., voting for validators based on their identity or influence / engagement in the community. +# 1. Underlying data +Collectible data from Polkadot and Kusama validators is extremely relevant to the selection process. Metrics marked with an asterisk (*) are included in the final data set, while other variables are used to derive additional metrics. The primary focus is on quantitative on-chain data, as it is verifiable and straightforward to process. This purely quantitative approach intends to complement a selection process that incorporates qualitative factors, such as a validator’s identity, reputation, or community engagement, which often influence how nominators cast their votes. ## Retrievable data | Name | Historical | On-Chain | Description | |- |- |- |- | -| Public Address* | No | Yes | The public identifier of the validator. | +| Public address* | No | Yes | The public identifier of the validator. | | Identity* | No | Yes | Is there a verified on-chain identity? | -| Self-stake* | No | Yes | The amount of tokens used to self-elect. Can be seen as skin-in-the-game. | -| Other-Stake | No | Yes | The amount of allocated stake (potentially) by other nominators. | -| Total-Stake | No | Yes | The sum of self-stake and other-stake. | -| Commission | Maybe | Yes | The amount of commission in % which is taken by the validator for their service. | -| Era-Points | Yes | Yes | The amount of points gathered per era. | -| Number of Nominators* | No | Yes | The amount of nominators allocated to a validator. | - -**Era-Points**: The era-points are awarded to a validator for performing beneficial action for the network. Currently this is mainly driven by block production. In general, the distribution of era-points should be uniformly distributed in the long run. However, this can vary if validators operates on a superior setup (stronger machine, more robust internet connection). In addition, there is significant statistical noise from randomness in the short-term, which can create deviations from the uniform distribution. +| Self stake* | No | Yes | Tokens used for self-election represent a form of "skin in the game". | +| Other stake | No | Yes | The amount of stake (potentially) allocated by other nominators. | +| Total stake | No | Yes | The combined total of self stake and other stake. | +| Commission | Maybe | Yes | Percentage of commission charged by the validator for providing their service. | +| Era points | Yes | Yes | The number of points accumulated per era. | +| Number of nominators* | No | Yes | The number of nominators assigned to a validator. | +**Era Points** are awarded to validators for performing beneficial actions that support the network, primarily driven by block production. Over time, these points should be uniformly distributed, although distribution may vary if validators operate on superior setups, like more powerful hardware or more reliable internet connections. In addition, randomness may introduce significant statistical noise in the short term, leading to deviations from a uniform distribution. ## Generated metrics -Some of the retrieved on-chain data might be not very useful for nominators or can serve some additional metrics, which help in the selection process. +Some of the retrieved on-chain data might not be particularly useful for nominators, but it can still provide additional metrics that help in the selection process. | Name | Historical | On-Chain | Description | |- |- |- |- | -| Average Adjusted Era-Points | Yes | Yes | The average adjusted era-points from previous eras. | -| Performance | Yes | Yes | The performance of a validator determined by era-points and commission. | -| Relative Performance* | Yes | Yes | The performance normalized to the set of validators. | -| Outperforming MLE | Yes | Yes | An indicator how often a validator has outperformed the average era-points. Should be 0.5 for an average validator. | -| Average Performer* | - | Yes | A statistical test of the outperforming MLE against the uniform distribution. Indicates if a validator statistically over- or underperforms. | -| Active Eras* | Yes | Yes | The number of active eras. | -| Relative total stake* | No | Yes | The total stake normalized to the set of validators. | -| Operator Size* | No | Yes | The number of validators which share a similar on-chain identity. | +| Average adjusted era-points | Yes | Yes | The average adjusted era points from previous eras. | +| Performance | Yes | Yes | Validator performance is determined by era points and commission. | +| Relative performance* | Yes | Yes | This represents performance normalized across the set of validators. | +| Outperforming MLE | Yes | Yes | An indicator of how frequently a validator has outperformed the average era points. A typical validator should score around 0.5. | +| Average performer* | - | Yes | A statistical test of the MLE for outperformance against a uniform distribution. It indicates whether a validator statistically overperforms or underperforms. | +| Active eras* | Yes | Yes | The number of active eras. | +| Relative total stake* | No | Yes | Total stake normalized across the validator set. | +| Operator size* | No | Yes | The number of validators that share a similar on-chain identity. | -**Average Adjusted Era-Points** -To get a more robust estimate of the era-points, additional data from previous eras should be gathered. Since the total era-points are distributed among all active validators, and the set of active validators might change, it could bias the results. To counter that, we can adjust the era-points of each era by the active set size of that era. As this is the only biasing factor on the theoretical per-capita era-points, we can thereby make the historic data comparable. +**Average adjusted era points.** +To obtain a more robust estimate of the era points, additional data from previous eras should be collected. Since the total era points are distributed among all active validators, and the validator set may vary over time, this could introduce bias into the results. To correct for this, era points from each era can be adjusted based on the active set size during that period. As this is the sole factor influencing theoretical per-capita era points, such normalization enables meaningful comparison across historical data. -It is unclear how many previous eras should be used as having a too long history might bias the results towards the average while too short of a history diminishes the robustness of the metric. One idea could be to use the average of $active-eras$. +The optimal number of previous eras to include remains uncertain. Using too long a history may bias results toward the average, while too short a history can weaken the metric’s robustness. One possible approach is to use the average number of $active-eras$. -**Performance**: The performance of a validator from the point of view of a nominator is determined by the amount of era-points gathered by that validator, the nominator's share of the total stake and the commission a validator is charging. In addition, the performance level is linear in the bond of the nominator and is thereby independent from that. We can combine those metrics into one: +**Performance.** From a nominator's perspective, validator performance is determined by three main factors: the number of era points earned, the nominator's share of the total stake, and the commission charged by the validator. Since performance scales linearly with the nominator's bond, it can be considered independent of the bond amount. These metrics can be combined into a single performance indicator: $$ performance = \frac{averageEraPoints \times (1 - commission)}{totalStake} $$ -The **relative performance** is then simply defined by: +The **relative performance** is then defined as follows: $$ \frac{performance - min(performance)}{max(performance) - min(performance)} $$ -This gives a more understandable measure as the performance is normalized between 0 and 1. Additionally, it is robust to potential changes within the network (e.g. with a larger number of validators the era-points are reduced per era) and prevents false anchoring effects. +These calculations offer a more intuitive measure, as the performance is normalized between 0 and 1. The measure remains robust against potential changes within the network. For instance, when the number of validators increases, the era points per validator tend to decrease. The metric also avoids false anchoring effects. -**Outperforming MLE**: By gathering the historic era-points per validator during past eras, we can calculate how often a validator outperformed the average. As era-points should be distributed uniformly, a validator should outperform the average 50% of times. However, as mentioned before, in reality additional factors as hardware-setup and internet connection can influence this. This helps nominators to select the best performing validators while creating incentives for validators to optimize their setup. +**Outperforming MLE.** By collecting historical era points per validator accross previous eras, one can determine how frequently a validator outperforms the average. Assuming a uniform distribution of era points, a validator is expected to outperform the average approximately 50% of the time. In practice, other factors like hardware-setup and internet connectivity, can influence this performance metric. These insights not only help nominators identify top-performing validators but also encourage validators to optimize their setup. -**Significance MLE**: As the expected value of the outperforming MLE is 0.5 and the distribution should be uniformly, we can calculate whether a validator significantly over- or underperforms by: +**Significance MLE.** Given that the expected value of the outperforming MLE is 0.5 under a presumably uniform distribution, a statistical test may help assess whether a validator significantly overperforms or underperforms relative to this benchmark: $$ z = \frac{outperformingMLE - 0.5}{\sqrt{\frac{0.5 \times (1-0.5)}{numberActive}}} $$ -If $z > 1.645$ we can say that the respective validator outperforms significantly (10% significance level), while $z < -1.645$ indicates significant underperformance. +If $z > 1.645$, the corresponding validator significantly outperforms at the 10% significance level, while $z < -1.645$ indicates significant underperformance. -**Operator Size**: Based on the identity of a validator, we can estimate how many validators are run by the same entity. It is both in the interest of users and the network that there are not too many operators and that those operators are not too large. Selecting validators of larger operators might increase the risk of superlinear slashing, because it is reasonable to assume that those operators follow similar security practices. A failure of one validator might mean a failure of several of those validators which increases the punishment superlinearly. A counter-argument to this might be that larger operators are much more sophisticated with their setup and processes. Therefore, this objective measure should be left to the user to judge. +**Operator size.** Based on the identity of a validator, it is possible to estimate how many validators are operated by the same entity. For both users and the network, a reduced number of moderately sized operators is often more convenient. Selecting validators from larger operators may increase the risk of superlinear slashing, as these entities likely follow similar security practices. The failure of one validator could therefore imply the failure of several others, increasing superlinearly the likelihood of punishment. On the other hand, larger operators may have more sophisticated setups and processes, which could mitigate such risks. This metric should ultimately be considered an objective measure, leaving the final judgment to the user. -# Filtering Phase +# 2. Filtering phase -## Dominance-Filtering -After constructing the dataset as elaborated in the section "underlying data", we can start reducing the set of validators to reduce the amount of information a nominator has to process. One concept is to remove dominated validators. As we do not make qualitative judgements e.g., which "identity" is better or worse than another, we can remove validators who are inferior to another, since there is no rational reason to nominate them. A validator is dominated by another validator if all her properties are equal and at least one property is worse. Consider the following example: +## Dominance-filtering +After shaping the dataset elaborated in the section "Underlying data," it is time to begin reducing the set of validators to ease the information load for nominators. One approach is to eliminate dominated validators. Since qualitative judgements remain out of the picture, such as determining whether one "identity" is better or worse than another, it is reasonable to remove validators that are objectively inferior, as there is no rational basis for nominating them. A validator is said to dominate another when all properties are equal and at least one is strictly better. Consider the following example: ## Example: | Number | Public Address | Identity | Self-stake | Nominators | Relative Performance | Outperformer | Active Eras | Operator Size | @@ -105,72 +99,71 @@ After constructing the dataset as elaborated in the section "underlying data", w | 3 | 1xgFnMhdOui | 1 | 100 | 89 | 0.3 | 0 | 16 | 3 | | 4 | 1vO7JLtSm4F | 1 | 5000 | 89 | 1 | 1 | 29 | 3 | -Validator 1 is dominated by Validator 2, which means that it is worse in every dimension (note, as mentioned above a user might prefer larger operators in which case this would not be true). Validator 3 is dominated by Validator 3 and therefore can be removed from the set. By this process the set can be reduced to two validators. In practice, this shows to be quite powerful to vastly reduce the set size. +Validator 2 dominates Validator 1, meaning the latter is strictly worse in every dimension[^1]. Validator 3 also dominates Validator 1, so it can be removed from the set. Through this process, the validator set can be reduced to two. In practice, this method proves to be a powerful tool for significantly shrinking the set size. ## Further curation -Here we have the opportunity to do additional cleanup to the remaining set. As mentioned in the code of conduct, those should be optional but we can suggest default values for users. -* Include at least 1 inactive validator. (we might suggest some inactive nodes based on some other processes.) -* Reduce risk of super-linear slashing (i.e., remove validators from operators). -* Remove validators who run on the same machine (some analysis of IP addresses possible?). +Additional cleanup can still be performed on the remaining set. As stated in the code of conduct, this step is optional, yet here are some suggested default actions for users: +* Include at least one inactive validator. A suggestion would be inactive nodes based on separate processes. +* Reduce the risk of super-linear slashing, for instance by removing multiple validators run by the same operator. +* Remove validators running on the same machine (perhaps some analysis of IP addresses). -# Manual selection -After the set has been reduced by removing dominated validators and giving some filter option the user can easily select preferred validators manually. In this step, the selection is purely based on personal preferences and for example a nominator might order the validators by their relative performance and select those who also satisfy some requirements on a minimum self-stake. +# 3. Manual selection +After reducing the set by removing dominated validators and applying some filtering options, the user can easily select preferred validators manually. In this step, the selection is purely based on personal preferences. For example, a nominator might order the validators by their relative performance, and select those who also meet certain minimum self-stake requirements. +# 4. UTAStar +As input, this method uses the filtered table from Section LINK and can be considered a natural extension of the previous method. +### Overview + UTilité Additive (UTA) is a preference disaggregation method introduced by [Jacquet-Lagrèze & Siskos (1982)](https://www.sciencedirect.com/science/article/abs/pii/0377221782901552). UTAStar is an enhanced version of the original algorithm. The core idea is that the marginal utility functions of a decision maker (DM), defined over each dimension of a given criterion, can be inferred from a previous ranked list of alternatives. The method employs linear programming to indentify utility functions that respect the DM's initial ranking while incorporating additional properties, such as normalizing the maximum utility to 1. -# UTAStar -This method takes the filtered table from section LINK as input and therefore can be seen as a natural extension to the method before. -## Overview -UTA (UTilité Additive) belongs to the methods of preference disaggregation ([Jacquet-Lagrèze & Siskos, 1982](https://www.sciencedirect.com/science/article/abs/pii/0377221782901552)). UTAStar is an improvement on the original algorithm. The general idea is that the marginal utility functions of a decision makers (DM) on each dimension of an alternative (i.e. criterion) can be deduced from a-priori ranked lists of alternatives. It uses linear programming to search for utility functions which satisfy the initial ranking of the DM while giving other properties (such as the maximum utility is normalized to 1). +### Some notation [^2] -### Some notation: -**This writeup relies strongly on [Siskos et al., 2005](https://www.researchgate.net/publication/226057347_UTA_methods)** -* $u_i$: marginal utility function of criteria i. +* $u_i$: Marginal utility function of criterion i. * $g_1,g_2,...g_n$: Criteria. -* $g_i(x)$: Evaluation of alternative x on the $i^{th}$ crterion. -* $\textbf{g}(x)$: Vector of performances of alternative $x$ on $n$ criteria. -* $x_1, x_2, ..., x_m \in X_L:$ Learning set which contain alternatives presented to the DM to give a ranking on. Note, that the index on the alternative is dropped. +* $g_i(x)$: Evaluation of alternative x on the $i^{th}$ criterion. +* $\textbf{g}(x)$: Performance vector of alternative $x$ across $n$ criteria. +* $x_1, x_2, ..., x_m \in X_L:$ Learning set containing alternatives presented to the decision maker (DM) for ranking. Note that the index on the alternative is dropped. ### Model -The UTAStar method infers an unweighted additive utility function: +The UTAStar method infers an additive utility function with equal weighting across criteria: $$ u(\textbf{g}) = \sum_{i=1}^{n} u_i(g_i) $$ -where $\textbf{g}$ is a vector of performances. with the following constraints: +where $\textbf{g}$ is a performance vector, subject to the following constraints: $$ \sum_{i=1}^{n} u_i(g^\star) = 1 \; \text{and} \; u_i(g_{i\star}) = 0 \; \forall i = 1,2,...,n $$ -where $u_i, i=1,2...,n$ are non decreasing valued functions which are normalized between 0 and 1 (also called utility functions). +Each $u_i, i=1,2...,n$ is a non-decreasing function normalized between 0 and 1, also referred to as a utility function. -Thereby the value of each alternative $x \in X_L$: +The estimated utility of each alternative $x \in X_L$ is given by: $$ u'[\textbf{g}(x)]=\sum_{i=1}^{n}u_i[g_i(x)])+ \sigma^{+}(x) + \sigma^{-}(x) \forall x \in X_L $$ -where $\sigma^{+}(x)$ and $\sigma^{-}(x)$ are the under- and overestimation error. is a potential error relative to $u'[\textbf{g}(x)]$ +where $\sigma^{+}(x)$ and $\sigma^{-}(x)$ represent the underestimation and overestimation errors, each reflecting potential deviation in the estimation of $u'[\textbf{g}(x)]$ -The corresponding utility functions are defined in a piecewise linear form to be estimated by linear interpolation. For each criterion, the interval $[g_{i\star}, g_i^\star]$ is cut into $(\alpha_i - 1)$ intervals and the endpoints $g_i^j$ are given by: +The utility functions are approximated in piecewise linear form using linear interpolation. For each criterion, the interval $[g_{i\star}, g_i^\star]$ is divided into $(\alpha_i - 1)$ subintervals, and the endpoints $g_i^j$ are defined as: $$ g_i^j = g_{i\star} + \frac{j - 1}{\alpha_i - 1} (g_i^\star - g_{i\star}) \forall j = 1,2,...\alpha_i $$ -The marginal utility function of x is approximated by linear interpolation and thus for $g_i(x) \in [g_i^j - g_i^{j+1}]$ +The marginal utility function of x is approximated by linear interpolation. Thus, for $g_i(x) \in [g_i^j - g_i^{j+1}]$, the result is: $$ u_i[g_i(x)]= u_i(g_i^j) + \frac{g_i(x)-g_i^j}{g_i^{j+1}-g_i^j}[u_i(g_i^{j+1}) - u_i(g_i^j)] $$ -The learning set $X_L$ is rearranged such that $x_1$ (best) is the head and $x_m$ is the tail (worst). This ranking is given by the user. +The learning set $X_L$ is rearranged such that $x_1$ (the best alternative) is placed at the head and $x_m$ is the tail. The user provides this ranking. The utility difference between two consecutive alternatives is defined as: $$ \Delta(x_k, x_{k+1}) = u'[\textbf{g}(x_k)] - u'(\textbf{g}(x_{k+1})) $$ -then we can be sure that the following holds: +then the following holds: $$ \Delta(x_k, a_{k+1}) \geq \delta \; \textrm{iff} \; x_k > x_{k+1} @@ -182,15 +175,15 @@ $$ \Delta(x_k, x_{k+1}) = \delta \; \textrm{iff} \; x_k \backsim x_{k+1} $$ -where $\delta$ is a small and positive number which is an exogenous parameter set as the minimum discrepancy between the utilities of two consecutive options. -In order to ensure monotonicity we further transform the utility differences between two consecutive interval endpoints: +Here, $\delta$ is a small, positive, exogenous parameter representing the minimum acceptable discrepancy between the utilities of two consecutive options. +Reinforcing monotonicity involves further transforming the utility differences between two consecutive interval endpoints: $$ w_{ij} = u_i(g_i^{j+1}) - u_i(g_i^j) \geq 0 \forall i=1,...n \; and \; j = 1,... \alpha_i -1 $$ ### Algorithm -**Step 1**: Express the global value of the alternatives in the learning set $u[g(x_k)], k=1,2,...m$ in terms of marginal values $u_i(g_i)$ and then transform to $w_{ij}$ according to the above mentioned formula and by means of +**Step 1.** Express the global utility of the alternatives in the learning set $u[g(x_k)], k=1,2,...m$, in terms of marginal utility functions $u_i(g_i)$. Transform these into coefficients $w_{ij}$ according to the formula provided, using the following constraints: $$ u_i(g_i^1) = 0 \; \forall i = 1,2...n @@ -202,13 +195,13 @@ $$ u_i(g_i^j) = \sum^{j-1}_{i=1}w_{ij} \; \forall i = 1,2..N \; and \; j=2,3,...\alpha_i - 1 $$ -**Step 2**: Introduce two error functions $\sigma^{+}$ and $\sigma^{-}$ on $X_L$ by writing each pair of consecutive alternatives as: +**Step 2.** Introduce two error functions, $\sigma^{+}$ and $\sigma^{-}$, on the learning set $X_L$. Represent each pair of consecutive alternatives as: $$ \Delta(x_k,x_k+1) = u[\textbf{g}(x_k)] - \sigma^{+}(x_k) + \sigma^{-}(x_k) - u[\textbf{g}(x_{k+1})] + \sigma^{+}(x_{k+1}) - \sigma^{-}(x_{k+1}) $$ -**Step 3**: Solve the linear problem: +**Step 3.** Solve the following linear optimization problem: $$ [min] z = \sum_{k=1}^{m}[\sigma^{+}(x_k) + \sigma^{-}(x_k)] \\ @@ -219,25 +212,30 @@ $$ w_{ij} \geq 0, \sigma^{+}(x_k)\geq 0, \sigma^{-}(x_k)\geq 0 \forall i,j,k $$ -**Step 4**: Robustness analysis to find find suitable solutions for the above LP. +**Step 4.** Perform a robustness analysis to identify suitable solutions for the linear program (LP) described above. -**Step 5**: Apply utility functions to the full set of validators and return the 16 best scoring ones. +**Step 5.** Apply the derived utility functions to the full set of validators and select the 16 highest-scoring ones. -**Step 6**: Make some ad hoc adjustments to the final set (based on input of the user). For example: -* include favorites -* at most one validator per operator -* at least X inactive validators -* etc. +**Step 6.** Introduce ad hoc adjustments to the final set based on user-defined preferences. For example: +* Include user-designated favorites +* Ensure no more than one validator per operator +* Require at least X inactive validators +* Additional custom constraints as needed ### Remaining Challenges -There remain a few challenges when we want to apply the theory to our validator selection problem. +Several challenges remain in applying the theoretical framework to the validator selection problem: + +1. **Constructing the learning set.** The algorithm requires sufficient information to generate the marginal utility functions. Key subchallenges include: + - Developing methods that ensure performance dispersion across criteria. + - Applying machine learning techniques to iteratively construct smaller learning sets to gradually improve the collected information. + - Using simulations to generate a wide number of learning sets and corresponding rankings, enabling evaluation of which configurations most effectively improve utility estimation. +2. **Limitations of UTAStar.** UTAStar assumes piecewise linear and monotonic marginal utility functions. While alternative methods offer improvements in this regard, they may introduce additional implementation complexity. + -1. One challenge is how to construct the learning set. The algorithm needs sufficient information to generate the marginal utility functions. - - Find methods to guarantee performance dispersion of the different criteria. - - Use machine learning approaches to iteratively provide smaller learning sets which gradually improve the information gathered. - - Potentially use simulations to simulate a wide number of learning sets and all potential rankings on them to measure which learning set improves the information the most. -2. UTAStar assumes piece-wise linear monotone marginal utility functions. Other, methods improve on that but might be more difficult to implement. +[^1] As mentioned above, a user might prefer larger operators in which case the statement would not be true. +[^2] This write-up relies heavily on [Siskos et al., 2005](https://www.researchgate.net/publication/226057347_UTA_methods). +**For inquieries or questions, please contact** [Jonas Gehrlein](/team_members/Jonas.md) diff --git a/docs/Polkadot/economics/2-parachain-theory.md b/docs/Polkadot/economics/2-parachain-theory.md index f9fa8c23..649dc9d5 100644 --- a/docs/Polkadot/economics/2-parachain-theory.md +++ b/docs/Polkadot/economics/2-parachain-theory.md @@ -2,15 +2,17 @@ title: Theoretical Analysis of Parachain Auctions --- -**Authors**: Samuel Häfner and [Alistair Stewart](team_members/alistair.md) + -**Last updated**: April 17, 2021 +Polkadot uses a [candle auction format](https://wiki.polkadot.network/docs/en/learn-auction) to allocate parachain slots. A candle auction is a dynamic auction mechanism characterized by a randomly ending time. Such a random-closing rule affects equilibrium behavior, particularly in scenarios where bidders have front-running opportunities. -As explained [here](/Polkadot/overview/3-parachain-allocation.md) and [here](https://wiki.polkadot.network/docs/en/learn-auction) Polkadot uses a candle auction format to allocate parachain slots. A candle auction is a dynamic auction with the distinguishing feature that the ending time is random. In this project, we analyze the effects of such a random-closing rule on equilibrium play when some bidders have front-running opportunities. +Front-running opportunities can arise on blockchains when upcoming transactions become visible to network participants before they are included in new blocks. In the context of blockchain auction implementations, this allows certain bidders to observe and potentially respond to others' bids before those bids take effect, such as when they are recorded on-chain and incorporated into the auction mechanism. In first-price auctions, this enables tech-savvy bidders to outbid competitors at will. In second-price auctions, an auctioneer could increase the payment of the winning bidder at no cost by registering their own (pseudonymous) bidder. -Front-running opportunities emerge on blockchains because upcoming transaction become known among the network participants before they are included in new blocks. For blockchain implementations of auctions, this means that some bidders can see and potentially react to other bidders' bids before they come into effect; i.e., are recorded on the chain and are thus taken into account by the auction mechanism. In first-price auctions, this gives tech-savvy bidders the possibility to outbid other bidders as they please. In second-price auctions, the auctioneer could raise the payment of the winning bidder at no cost by registering his own (pseudonymous) bidder. +While cryptographic solutions to these problems exist, they are either computationally intensive or require multiple actions by the bidders. In the presence of smart contracts, such approaches fail altogether, as smart contract actions are fully predictable. -Whereas cryptographic solutions to these problems exist, they are either very computing intensive or require multiple actions by the bidders. In the presence of smart contracts, they do not work at all, because the actions of smart contracts are perfectly anticipatable. As an alternative that works without encrypting bids, this project analyzes a dynamic single-unit first-price auction with a random ending time. Time is discrete and in every round two bidders move sequentially in a fixed order. We show that a random-closing rule both revenue-dominates a hard-closing rule and makes participation for the bidder being front-run more attractive. In particular, under a uniform ending time distribution both the utility of the disadvantaged bidder and total revenues approach that of a second-price auction as the number of rounds grows large. Furthermore, the good is allocated efficiently. +An alternative to encrypted bidding is the use of a dynamic, single-unit first-price auction with a random ending time. Time is modeled discretely, and in each round, two bidders move sequentially in a fixed order. A random-closing rule not only revenue-dominates a hard-closing rule but also makes participation more attractive for bidders subject to front-running. In particular, under a uniform ending time distribution, both the utility of the disadvantaged bidder and the total revenue converge toward those of a second-price auction as the number of rounds increases. Furthermore, the good is allocated efficiently. Reference: Samuel Häfner and Alistair Stewart (2021): Blockchains, Front-Running, and Candle Auctions. Working Paper. [SSRN](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3846363) + +**For inquiries or questions please contact:** [Alistair Stewart](team_members/alistair.md) diff --git a/docs/Polkadot/economics/3-parachain-experiment.md b/docs/Polkadot/economics/3-parachain-experiment.md index 022f11ca..e70b6483 100644 --- a/docs/Polkadot/economics/3-parachain-experiment.md +++ b/docs/Polkadot/economics/3-parachain-experiment.md @@ -2,136 +2,138 @@ title: Experimental Investigation of Parachain Auctions --- -**Authors**: [Jonas Gehrlein](/team_members/Jonas.md), Samuel Häfner + -**Last updated**: 16.08.2021 +This entry focuses on experimentally examining the combinatorial candle auction as implemented in the Polkadot and Kusama protocol. Specifically, it compares its outcome with those of more traditional dynamic combinatorial auction formats currently in use. -## Overview -The goal of this project is to experimentally test the combinatorial candle auction as it is used in the Polkadot and Kusama protocol. In particular, we want to compare its outcome with those of more traditional, dynamic combinatorial auction formats employed today. +What distinguishes the candle auction apart from other dynamic auctions formats is its randomly determined ending time. This closing rule enables auctions to operate on blockchains by mitigating several practical issues that more common formats often encounter (cf. Häfner & Stewart, 2021, for an analysis of the sinlge-unit case). -What sets the candle auction apart from other dynamic auctions is that it has a random ending time. Such a closing-rule is important for auctions run on blockchains, because it mitigates several practical problems that other, more common auction formats suffer from (cf. Häfner & Stewart, 2021, for an analysis of the sinlge-unit case). +Since the combinatorial candle auction has not yet been studied, either theoretically or empirically, this analysis aims to fill that gap in the literature. The central hypothesis suggests that the candle format performs on par with, or even surpasses, the performance of dynamic combinatorial auctions that rely on specfic activity rules. These rules restrict feasible bids and close the auction when no further bids are submitted, thereby exerting early and serious pressure on bidders. The random ending time is expected to create a similar effect. In particular, this pressure to act may lead to more efficient outcomes, comparable to those generated by activity rules, when contrasted with auctions that use a simple hard-closing rule, such as fixed ending time. -The combinatorial candle auction has never been studied theoretically and empirically. Therefore, this project fills a gap in the literature. We hypothesize that the candle format is at par with, or even outperforms, dynamic combinatorial auctions that use specfic activity rules. Activity rules specify the feasible bids and close the auction when no more bids are entered in the system. Thus, they put pressure on the bidders to bid seriously early on. We expect a similar effect from the random ending time. In particular, we expect that the pressure to act induces - akin to activity rules - more efficient outcomes than in an auction with a simple hard-closing rule (i.e., a fixed ending time). +The design of the experimental investigation mirrors the core mechanism of the Polkadot parachain auction. The experiment takes place in a setting where bidders can freely communicate and share non-binding strategies before the auction. It is conducted off-chain and follows standard experimental economics procedures. Insights from the experiment can enhance understanding of bidding behavior and enable comparisons of efficiency across auction formats. -We will conduct an experimental investigation with a design that mimics the basic mechanism of the Polkadot parachain auction. In particular, we conduct the experiment in a context where bidders can freely communicate and share their non-binding strategies before the auction. The implementation is off-chain and follows standard experimental economics procedures. Insights from the experiment can be used to gain an understanding of the bidding behavior and to compare efficiency across formats. +## Dynamic combinatorial auctions -## Dynamic Combinatorial Auctions +The first point of discussion is the current use of combinatorial auctions. Second, it is important to present the combinatorial candle auction as implemented on Polkadot, highlighting the appeal of this format, and discussing its expected performance relative to more conventional combinatorial auction models. -In this section, we first discuss how combinatorial auctions are currently used. Second, we describe the combinatorial candle auction as it used on Polkadot, explain why we use this format, and what we expect about its performance vis-a-vis more standard combinatorial auction formats. +### Currently used combinatorial auctions +Combinatorial auctions have emerged as successors to multi-unit auctions, primarily due to their ability to solve the so-called exposure problem that arises in the presence of complementarities (Porter and Smith, 2006; Cramton, 2013). In multi-unit auctions, bidders compete for each unit individually. As a consequence, bidders seeking specific combinations of items may end up acquiring only a subset, which may hold significanlty less value than the complete bundle. Combinatorial auctions resolve this issue by allowing bids on item packages directly. In other words, a bidder either wins the entire package or nothing at all. -### Currently used Combinatorial Auctions -Historically, combinatorial auctions have emerged as successors of multi-unit auctions. Combinatorial auctions solve the so-called exposure problem from which multi-unit auctions suffer in the presence of complementarities (Porter and Smith, 2006; Cramton, 2013). In a multi-unit auction, the bidders compete for every unit separately. As a consequence, bidders aiming (and bidding) for a certain combination of items might suddenly find themselves in a situation of obtaining only a subset thereof, which has substantially lower value to them than the whole package. Combinatorial auctions allow bidders to place bids on packages directly and thus avoid this problem. That is, if you bid on a package then you either get the whole package, or nothing. +Today, combinatorial auctions are applied in a wide range of contexts. Among the most well-known examples are radio spectrum auctions (Porter and Smith, 2006; Cramton, 2013). Additional applications include electricity markets (Meeus et al., 2009), bus routes allocations, and industrial procurement (cf. Cramton et al., 2006, for an overview). -Today, combinatorial auctions are employed in many contexts. The most well known applications of combinatorial auctions are radio spectrum auctions (Porter and Smith, 2006; Cramton, 2013). Other applications include electricity (Meeus et al., 2009), bus routes, and industrial procurement (cf. Cramton et al., 2006, for an overview). +Many combinatorial auctions are dynamic, typically employing one of two distinct formats: -Many combinatorial auctions are dynamic. There are two distinct formats: +1. *Ascending format*: While the auction remains open, bidders can submit progressively higher bids for the different packages (see Bichler et al., 2017 for an experimental study). +2. *Clock format*: The auctioneer incrementally raises the prices of individual items or packages, and in each round, bidders must submit their demand for the different packages. In some versions, bidders are allowed to place final bids in an additional round of simultaneous bidding after the clock phase concludes (originally proposed by Ausubel et al., 2006; further discussed in Cramton, 2013). -1. *Ascending format*: As long as the auction is open, bidders can submit increasing bids for the different packages (experimentally studied Bichler et al. 2017). -2. *Clock format*: The auctioneer raises the prices on the individual items or packages and in every round, the bidders have to submit their demand for the different packages. In some formats, bidders can submit last bids in one additional round of simultaneous bidding once the clock phase is over (initially suggested by Ausubel et al. 2006; further discussed in Cramton, 2013). +U.S. radio spectrum auctions employed simple ascending multi-unit auctions for the first time. In 2006 and 2008, among other instances, the FCC introduced package bidding on predefined licenses using an ascending format (Porter and Smith, 2006; Cramton, 2013). The transition to clock auctions occurred later (Levin and Skrzypacz, 2016). -For example, in the case of US radio spectrum auctions, simple, ascending multi-unit auctions were used first. Then, in 2006 and 2008 among others, the FCC allowed bidding on pre-defined packages of licences in an ascending format (Porter and Smith, 2006; Cramton, 2013). The switch to clock auctions occurred later on (Levin and Skrzypacz, 2016). +An important design feature of any dynamic combinatorial auction is the activity rule. Its primary role is to encourage serious bidding from the outset and to prevent sniping or jump bidding. -An important design feature of any dynamic combinatorial auction is its so-called activity rule. The primary role of the activity rules is to encourage serious bidding from the start and to prevent sniping or jump-bidding. +During the auction phase, the activity rule determines which bids are feasible for each bidder. In the ascending bid format, the rule tipically specifies a minimum and a maximum increment that a new bid must have on a given item or package relative to a previous bid (Scheffel et al., 2012). In clock auctions, the activity rule may prevent bidders from switching to packages they ignored in earlier rounds; that is, bidders may reduce demand but not increase it (Levin and Skrzypacz, 2016). In both ascending and clock auctions, the rule may also restrict bidders to packages that are weakly smaller than those previously bid on (Cramton, 2013). -During the auction phase, the activity rule determines what kind of bids are feasible for any given bidder. In the case of the ascending bid format, the rule usually defines a minimum and a maximum increment that a new bid on a given item or package can have over an older bid (Scheffel et al., 2012). In the clock auction, the activity rule may prevent bidders from jumping onto a given package that they have ignored in earlier rounds; i.e., bidders may reduce demand but not increase it (Levin and Skrzypacz, 2016). In both the ascending auction and the clock auction, the rule sometimes also restricts bidders to bid on packages that are weakly smaller than the ones previously bid on (Cramton, 2013). +The activity rule also determines when the auction ends based on all previously submitted bids. In an ascending auction, the auction closes once no new bids are placed in a given round (Scheffel et al., 2012). In an clock auction, prices for individual packages are raised simultaneously until there is no excess demand for any package, which is when the auction concludes (Bichler et al., 2013). -Second, the activity rule determines when the auctions end based on all previously entered bids. In the ascending auction, the activity rule closes the auction when no new bids are entered in a round (Scheffel et al., 2012). In the clock auction, the prices on the individual packages are (simultaneously) raised until there is no excess demand for a package and the auction concludes when there is no excess demand for any of the packages (Bichler et al., 2013). +### The Combinatorial candle auction +In the combinatorial candle auction employed in Polkadot, bidders can submit bids within a predefined time window. Bids must increase, without further restrictions imposed by an activitiy rule. After the window closes, the actual ending time is retroactively determined at random. -### The Combinatorial Candle Auction -In the combinatorial candle auction employed in Polkadot, bidders can submit bids in a pre-defined time window. Bids have to be increasinge but they are otherwise not restricted by an activitiy rule. After the window closes, the ending time is retroactively determined in a random fashion. +Originated in medieval Europe, the name "candle auctions" derive from the way they were conducted. The auctioneer would light a candle in view of all the bidders and accept bids until the flame extinguished. The highest bidder at the moment the candle went out was declared the winner (cf., e.g., Hobson, 1971). The earliest accounts of this kind of auction date back to 14th-century France, where they were used to sell chattels and leases. In England, furs were sold via candle auctions up to the 18th century (cf. Füllbrunn and Sadrieh, 2012, for more details and references). -Candle auctions are believed to have originated in medieval Europe and they derive their name from the particular way they were conducted. The auctioneer lights a candle in sight of all the bidders and accepts bids until the candle goes out. The highest bidder at the time the candle goes out is declared the winner (cf., e.g., Hobson, 1971). Earliest accounts of this kind of auction date back to 14th century France where they were used to sell chattels and leases. In England, furs were sold in candle auction up to the 18th century (cf. Füllbrunn and Sadrieh, 2012, for more details and references). +Candle auctions have become rare. Possible reasons include the technical difficulty of generating generic randomness and the challenge of verifying commitment to a random device. Recent advances in cryptography help circumvent these issues and have brought candle auctions back into consideration. For example, Google held a patent for a dynamic auction with a random ending time, which expired in 2020 (Patent No. US6665649B1). -Candle auctions have become rare today. A possible reason is that generic randomness is technically hard to achieve and that the commitment to a random device hard to verify. Recent cryptographic advances allow to circumvent these problems and put the candle auction back on the scene. For example, For example, Google held a patent on a dynamic auction with a random ending time that expired in 2020 (Patent No. US6665649B1). +Front-running is a significant challenge in blockchain-based auction implementations. To mitigate this issue, the Polkadot protocol employs a candle auction mechanism. Since block production occurs at discrete intervals and all pending transactions are stored in the chain's mempool, tech-savvy bidders can, in principle, inspect and react to upcoming bids. This raises concerns that such behavior may reduce overall incentives to bid, thereby lowering both revenue and potential efficiency. As Häfner & Stewart (2021) argue, while cryptographic solutions to front-running do exist, they are not feasible within Polkadot's automated setting, primarily because smart contracts among bidders are expected. -The main reason why the Polkadot protocol employs a candle mechnism is that it mitigates some of the problems associated with front-running in auctions. Front-running is a major problem of blockchain implementations of auctions. Because block production only happens at discrete intervals but all upcoming transactions are stored in the chain's mempool, tech-savvy bidders can in principle inspect and react to upcoming bids. The general worry is that this reduces the overall incentives to bid, thus reducing revenue and possibly efficiency. As argued in Häfner & Stewart (2021) cryptographic solutions to the problem -- though they exist -- are not feasible for the automated setting of Polkadot, primarily because we expect smart contracts among bidders. +As far as existing literature indicates, Füllbrunn and Sadrieh (2012) is the only experimental study to examine a candle auction format. Unlike the planned experiment, their study focuses on a single-unit auction with a second-price payment rule. In a second-price auction, bidding one's true value is a weakly dominant strategy whenever there is a positive probability that the current round will be the terminal round. The experimental evidence largely supports this prediction. In contrast to first-price auctions, where equilibrium bidding depends on termination probabilities, expected revenue in second-price auctions is independent of those probabilities. -To the best of our knowledge, Füllbrunn and Sadrieh (2012) is the only experimental paper to also study a candle format. Other than in our planned experiment, they consider a single-unit auction with a second-price payment rule. In the second price auction, it is a weakly dominant strategy to bid the true value whenever there is a positive probability that the current round will be the terminal round. The experimental evidence largely confirms such a prediction. Other than in the first-price auction, where equilibrium bidding depends on the termination probabilities, expected revenue is independent of the termination probabilities. +## Experimental design -## Experimental Design +The aim is to examine an ascending combinatorial auction with discrete rounds $t$, during which bids can be placed. Each auction involves three bidders. After every round, which lasts $6$ seconds, all newly submitted bids are revealed. -We want to look at an ascending combinatorial auction with discrete rounds $t$ in which bids can be placed. There will be three bidders in every auction. After every round, all new bids are revealed. A round lasts for $6$ seconds. +The set of items is $X = \{1,2\}$, resulting in three possible packages $\{\{1\},\{2\},\{1,2\}\}$. Bidders may submit bids, where a bid $b=(p,x)$ consists of a price $p$ and a package $x \subseteq X$. Prices must increase and lie on a finite (fine) grid. Winning bids are selected to maximize total payment. The payment rule is pay-as-bid; that is, winning bidders must pay the amount they bid. -The set of items is $X = \{1,2\}$ giving us three packages $\{\{1\},\{2\},\{1,2\}\}$ on which bidders can submit bids. A bid $b=(p,x)$ consists of a price $p$ and any package $x \subseteq X$. Prices have to be increasing and must lie in a finite (fine) grid. The winning bids are selected to maximize total payment. The payment rule is pay-as-bid; i.e., winning bids have to be paid. +### The three ending formats -### The Three Ending Formats -We want to compare three ending formats: a candle format, a hard-closing rule, and an activity rule. +As mentioned in the introduction, one main objective is to compare three auction-ending formats: the candle format, the hard-closing rule, and the activity rule. -| | Communication | -|----------------|------------------| -| Candle Auction | CA | -| Hard-Close | HC | -| Activity Rule | AR | +| Auction Format | Abbreviation | +|------------------|--------------| +| Candle Auction | CA | +| Hard-Close | HC | +| Activity Rule | AR | -**Candle Format** In the candle auction, bidders can freely submit increasing bids during the auction phase, and the auction is terminated at random. In the specification that we consider, the ending time is determined retroactively; i.e, bids on packages are accepted in a predefined number of rounds, $\bar T$, after which the auctioneer announces the ending time $T \in \{1,...,\bar T\}$. The ending time $T$ is random, the probability that the auction ends in round $t$ is publicly known and given by $q_t \in (0,1)$, where $\sum_{t=1}^{\bar T}q_t=1$. +**Candle Format.** In a candle auction, bidders can freely submit increasing bids during the auction phase, and the auction terminates at a random time. In this specification, the ending time is determined retroactively: bids on packages are accepted for a predefined number of rounds, denoted by $\bar T$, after which the auctioneer announces the actual ending time $T \in \{1,...,\bar T\}$. The ending time $T$ is random, and the probability that the auction ends in round $t$ is publicly known and given by $q_t \in (0,1)$, where $\sum_{t=1}^{\bar T}q_t=1$. -**Hard-Close Rule** In the hard-close auction, bidders can also freely submit increasing bids yet the auction ends at a fixed end time, $\bar T$. +**Hard-Close Rule.** In a hard-close auction, bidders can freely submit increasing bids, yet the auction ends at a fixed time, denoted by $\bar T$. -**Activity Rule** In the activity rule format, the ending time is determined by the activity rule. Specifically, bids have to be increasing and if no new bid is entered for $\tau$ rounds, then the auction concludes. For the experiment, we propose $\tau=5$ (corresponding to $30$ seconds). +**Activity Rule.** In an activity rule format, bidder activity determines the ending time. Specifically, bids must increase, and if no new bid is submitted for $\tau$ consecutive rounds, the auction concludes. In this experiment, $\tau$ is set to 5, corresponding to $30$ seconds. ### Communication -Communication is ubiquitous in the blockchain setting. The different bidders are teams that work on similar technical problems, share communication channels, post on social media, etc. +Communication is ubiquitous in blockchain environments. Different bidders often operate as teams working on similar technical problems, sharing communication channels, posting on social media, and more. -Consequently, we will allow our experimental subjects to communicate in a chat before each auction and discuss non-binding strategies. Specifically, the bidders will have both an open chat as well as closed bilateral chat channels available. The chats will be open prior to the auction start and close thereafter. +Consequently, experimental subjects are allowed to communicate via chat before each auction to discuss non-binding strategies. Specifically, both an open group chat and closed bilateral chats are available. These channels are accessible prior to the auction and close once it begins. ### Valuations -In every auction, three bidders will be active. Bidders can have one of two roles that are commonly known when entering the auction: (1) global bidder, (2) local bidder. There will be one global bidder and two local bidders in every auction. +In each auction, three bidders participate. Bidders are assigned one of two roles, global bidder or local bidder, which are known prior to the start of the auction. Each auction features one global bidder and two local bidders. -The global bidder has a positive valuation only for the grand package, $\{1,2\}$. The local bidders hold valuations for the individual packages that add up in case they win the grand package. Specifically, we will assume +The global bidder has a positive valuation only for the grand package, denoted as $\{1,2\}$. Local bidders, on the other hand, hold valuations for individual packages, which are added up if they win the grand package. Specifically, it is assumed that + +
![](https://i.imgur.com/feIk9Hu.png) -In words, the global bidder draws a valuation $v$ for the package $\{1,2\}$ and always holds a valuation of zero for the packages $\{1\}$ and $\{2\}$. On the other hand, local bidder $i = 1,2$ draws a valuation $v_i$ for $\{1\}$, implying that she values item $\{2\}$ at $80-v_i$ and package $\{1,2\}$ at $80$. +
+ +This means that the global bidder draws a valuation $v$ for the package $\{1,2\}$, while always holding a zero valuation for the individual packages $\{1\}$ and $\{2\}$. In contrast, each local bidder $i = 1,2$ draws a valuation $v_i$ for item $\{1\}$, which implies a valuation of $80-v_i$ for item $\{2\}$, and a total valuation of $80$ for the combined package $\{1,2\}$. -Under this value model it is efficient for the global bidder to obtain the grand package whenever $v \geq \max \{80-v_1+v_2,80-v_2+v_1\}$ and for the two local bidders to each receive one of the items otherwise. In order to win against the global bidder, though, the local bidders must coordinate their bids accordingly. +Under this valuation model, the global bidder can efficiently obtain the grand package whenever $v \geq \max \{80-v_1+v_2,80-v_2+v_1\}$, while the two local bidders receive one item each otherwise. To successfully outbid the global bidder, the local bidders must coordinate their bids strategically. ### Hypotheses -We will be interested in the following outcome variables: +The following outcome variables are of main interest: -* Efficiency: In what fraction of the auctions does the resulting allocation correspond to the first-best allocation? -* Revenue: Equals to the total bids paid. This also allows us to compute average shading ratios. -* Bidding dynamic: How fast do bids increase? Do we see sniping? +* Efficiency: What fraction of the auctions result in an allocation that corresponds to the first-best outcome? +* Revenue: Total amount of bids paid. This also allows for the computation of average shading ratios. +* Bidding dynamic: How quickly do bids increase? Does sniping occur? -In general, the random ending time puts pressure to submit serious bids early on in the auction. We expect this to have two effects vis-a-vis a hard-closing rule (under which the auction ends at a fixed end date) that are similar to what activity and feedback rules should achieve. That is, we conjecture that a candle format can replace these rules to some extent. +The random ending time generally encourages participants to submit serious bids earlier in the auction. This mechanism is expected to produce two beneficial effects, compared to a hard-closing rule where the auction ends at a fixed time, that resemble the intended impact of activity and feedback rules. The underlying conjecture is that the candle auction format may replace these rules to some extent. -* Hypothesis I: Early bids in the candle auction are higher than under the activity rule; and they are higher under the activity rule than they are under the hard-close rule. -* Hypothesis II: The candle format and the hard-close rule fare better than the hard-close rule in terms of revenue and efficiency. -* Hyptothesis III: The candle format and the hard-close rule fare similarly in terms of revenue and efficiency. Perhaps: Efficiency is slightly worse in the candle auction while revenue is slightly better. +* Hypothesis I: Early bids in the candle auction are higher than under the activity rule, and bids under the activity rule are already higher than under the hard-close rule. +* Hypothesis II: The candle format and the activity rule outperform the hard-close rule in terms of revenue and efficiency. +* Hyptothesis III: The candle format and the hard-close rule perform similarly in terms of revenue and efficiency. Efficiency may be slightly lower in the candle auction, while revenue may be slightly higher. ### Procedure #### Stage 1: Instructions -At the beginning of the experiment, participants are randomly allocated to one of the three different auction formats and receive information about specific rules of the game. To ensure that subjects understand the game, we will also ask a few comprehension question. +As the experiment begins, participants are randomly assigned to one of the three auction formats and receive information about the specific rules of the game. They must then answer a few comprehension questions to confirm their understanding of how the game unfolds. #### Stage 2: The Auctions -Before each auction, all bidders learn their type and their private valuations for the individual packages. Each market consists of one global and two local bidders. Their roles remain fixed throughout the experiment but new values are drawn each new auction. To better compare the results across the treatments, we can fix the random draws (i.e., the seed) for each auction across treatments. Every subject participates at n=X auctions while we make sure that we re-shuffle subjects into markets to match a (near-) perfect stranger design. Then, the communication phase starts where all participants of an auction can discuss openly in a chat-format for 45 seconds. After this, the auction starts and subjects are free to submit bids. +Before each auction, all bidders learn their type and their private valuations for individual packages. Each market consists of one global bidder and two local bidders, with roles remaining fixed throughout the experiment. New valuations are drawn at the start of each auction. These random draws are held constant across treatments (i.e., by fixing the seed) to better compare the results. Every participant takes part in n=X auctions. Subjects are re-shuffled into new market groups between auctions to approximate a (near-) perfect stranger design. Following this, a communication phase begins during which all participants in a given auction openly discuss strategies via chat for 45 seconds. The auction then starts, and participants are free to submit bids. The trading page features two tables: -1. (Table 1) Current winning bids: This shows all current bids per package. +1. Table 1 displays the current winning bids, listing all active bids for each package. -2. (Table 2) Winning Allocation: This shows how the packages are currently allocated to bidders based on the current winning bids. +2. Table 2 shows the winning allocation, indicating how packages are currently assigned to bidders based on the prevailing winning bids. -Especially table 2 is considered to significant help with this complex auction design. +Table 2 is particularly useful for assessing this complex auction design. #### Stage 3: Feedback and Payoff -After the end of the auction (depending on the treatment), participants receive feedback about the final winning bids and allocation of packages. In addition, subjects in the candle auction format are informed about the realization of $T$ and the respective snapshot of winning bids to that time. Profits are calculated and shown to the subjects. Afterwards, the next auction (if there are any) is started and new valuations are drawn for each subject. +Once the auction concludes, timing depending on the treatment, participants receive feedback on the final winning bids and allocation of packages. In addition, participants in the candle auction format are informed of the realization of $T$ and the corresponding snapshot of winning bids at that moment. Profits are then calculated and displayed to participants. If another auction follows, new valuations are drawn for each subject before it begins. ### Outcome variables -* Success of coordination (given the realized values, were the local bidders able to form a coalition?) -* Efficiency (Did the packages go to those with the highest valuation? Did they coordinate on the right allocation) -* Bidding dynamic (how quickly converges the auction) -* Revenue +* Success of coordination: Given the realized values, were the local bidders able to form a coalition? +* Efficiency: Did the packages go to those with the highest valuations? Did bidders coordinate on the optimal allocation? +* Bidding dynamic: How quickly did the auction converge? +* Revenue: What was the total reveneu generated? ### Implementation -The experiment will be implemented with [oTree](https://www.sciencedirect.com/science/article/pii/S2214635016000101), which is a software to conduct online experiments and provide the necessary infrastructure to create sessions, distribute links to users and maintain a database of behavioral data. It combines python in the back-end with a flexible front-end implementation of HTML/CSS and Django. +The experiment will be implemented using [oTree](https://www.sciencedirect.com/science/article/pii/S2214635016000101), a platform for conducting online experiments. oTree provides the necessary infrastructure to create sessions, distribute links to participants, and maintain a database of behavioral data. It combines Python on the backend with a flexible frontend built using HTML/CSS and Django. ## Literature Ausubel, L. M., Cramton, P., & Milgrom, P. (2006). The clock-proxy auction: A practical combinatorial auction design. Combinatorial Auctions, 120-140. @@ -155,3 +157,5 @@ Meeus, Leonardo, Karolien Verhaegen, and Ronnie Belmans. “Block order restrict Porter, David, and Vernon Smith. “FCC license auction design: A 12-year experiment.” Journal of Law, Economics & Policy 3 (2006): 63. Scheffel, T., Ziegler, G., & Bichler, M. (2012). On the impact of package selection in combinatorial auctions: an experimental study in the context of spectrum auction design. Experimental Economics, 15(4), 667-692. + +**For inquieries or questions please contact** [Jonas Gehrlein](/team_members/Jonas.md) diff --git a/docs/Polkadot/economics/4-gamification.md b/docs/Polkadot/economics/4-gamification.md index c06bb857..1b1a260f 100644 --- a/docs/Polkadot/economics/4-gamification.md +++ b/docs/Polkadot/economics/4-gamification.md @@ -2,106 +2,113 @@ title: Non-monetary incentives for collective members --- -**Authors**: [Jonas Gehrlein](/team_members/Jonas.md) + -**Last updated**: 13.04.2023 +Behavioral economics has demonstrated that non-monetary incentives can be powerful motivators, offering a viable alternative to financial rewards (see, e.g., [Frey & Gallus, 2015](https://www.bsfrey.ch/articles/C_600_2016.pdf)). This is especially true in environments where intrinsic motivation drives behavior. In such contexts, monetary incentives may even crowd out intrinsic motivation, ultimately reducing engagement ([Gneezy & Rustichini, 2000](https://academic.oup.com/qje/article-abstract/115/3/791/1828156)). -## Overview - -Behavioral economics has proven that non-monetary incentives are viable motivators and resemble an alternative to incentives created by money (see, e.g., [Frey & Gallus, 2015](https://www.bsfrey.ch/articles/C_600_2016.pdf)). This is especially true for environments where behavior is mainly driven by intrinsic motivation. In those situations, monetary incentives can even crowd-out the intrinsic behavior leading to less motivation ([Gneezy & Rustichini, 2000](https://academic.oup.com/qje/article-abstract/115/3/791/1828156)). The current advances in technologies surrounding Non-fungible Tokens (NFTs) can be utilized as an additional incentive layer for governance participation and the engagement of collective members. NFTs as tool can be perfectly combined with insights from the academic literature about the concept of "gamification" to foster engagement and reward good behavior. - -This can help improve on a few issues that are inherent to governance, especially low participation. +Recent advances in technologies surrounding Non-fungible Tokens (NFTs) present a promising new layer of incentives for governance participation and collective engagement. NFTs, as a tool, can be effectively combined with insights from academic literature on gamification to encourage participation and reward good behavior. This can help address several inherent challenges in governance, particularly low participation. ### Problem statement -Governance is one of the most important aspects of the future of decentralized systems such as DAOs and other collectives. They rely on active participation of the token holders to achieve an efficient decision making. However, turnout-rates of tend to be quite low, which opens up the danger of exploits by a very motivated minority. There are many points to prevent this from happening, for example usability and user experience improvements to the governance process. +Governance is one of the most critical aspects for the future of decentralized systems, such as DAOs and other collectives. These systems rely on active paritcipation from token holders to enable efficient decision-making. However, turnout rates tend to be quite low, which creates the risk of governance exploits by a highly motivated minority. Several factors can help mitigate this risk, for example enhancing usability and user experience within the governance process. -This write-up focuses on providing non-monetary incentives as motivator to engage more actively in a collective. It can be applied to layer0 governance or smaller collectives (DAOs). +This entry explores non-monetary incentives as a means to encourage more active engagement within a collective. The approach can be applied to layer0 governance as well as smaller collectives, such as DAOs. ### Goals -The goals is to design a mechanism which automatically applies certain tools from gamification (e.g., badges, achievements, levels) to collective members to... +The goal is to design a mechanism that automatically applies selected gamification tools such as badges, achievements, and levels to collective members in order to: -* ... promote the engagement and liveness of members. -* ... use established techniques from the literature to improve on the whole governance process. -* ... make it easier for users to evaluate and compare members. +* Promote the engagement and liveness of members. +* Leverage established techniques from the literature to improve the overall governance process. +* Enable users to evaluate and compare members more efficiently. -Improving on all those domains would further strengthen the position of the network in the blockchain ecosystem. +Advancing in these areas would further strengthen the network's position within the blockchain ecosystem. ## Literature -Gamification received increasing attention in the recent years and was even called the "most notable technological developments for human -engagement" ([Majuri et al., 2018](https://trepo.tuni.fi/bitstream/handle/10024/104598/gamification_of_education_2018.pdf)). It is used to enhance learning outcomes (e.g., [Denny, 2013](https://dl.acm.org/doi/abs/10.1145/2470654.2470763?casa_token=XsWtSZeFt-QAAAAA:MPWbtFfjzQZgWzyTI9hWROarJb1gJDWqDHNG4Fyozzvz3QIK-kMuMxfSwE26y9lKYUuZnV7aDZI)), model online communities (e.g., [Bista, 2012a](https://ieeexplore.ieee.org/abstract/document/6450959)) and improve sustainable behavior (e.g., [Berengueres et al., 2013](https://ieeexplore.ieee.org/abstract/document/6483512?casa_token=tmdUK7mtSSEAAAAA:ZxJnvYNAcuRaMHbwNqTJnahpbxal9xc9kHd6mY4lIahFhWn2Gmy32VDowMLVREQjwVIMhd9wcvY)). Gamification can be used as "means of supporting user engagement and enhancing positive patterns in service use, such as increasing user activity, social interaction, or quality and productivity of actions" ([Hamari, Koivisto & Sarsa, 2014](https://ieeexplore.ieee.org/abstract/document/6758978?casa_token=F2o_LQE-CNgAAAAA:vA_xBEe0ltKmMPRmTfkyW78LThHP9hLKK06oj1gKpOeDfoCTG7l_p-KSVlcdhNpaErLjzrm8p90)). While there is no agreed-upon definition, it can be best described as "a process of enhancing a service with affordances for gameful experiences in order to support user's [sic] overall value creation” ([Huotari & Hamari, 2012, p. 19](https://dl.acm.org/doi/abs/10.1145/2393132.2393137?casa_token=MU2yq2P4TOoAAAAA:Xuy9ZEzo2O7H-WCbqMheezkrodpab2DlFWkLjVt3jYExuP--vsjEROt4BKt5ZEbVou9rVnQSQBs)). That means, applying this concept does change the underlying service into a game but rather enriches it with motivational affordances popular in gaming (points, levels, badges, leaderboards) ([Deterding, 2012](https://dl.acm.org/doi/fullHtml/10.1145/2212877.2212883?casa_token=B9RD9ZPneIMAAAAA:34lrdGKwOUZyZu8fLobERuPLIBzNQxxwlgWLJnonn5Ws8Ya65aO_pdifhlHiSBwjDb0mWyFD0aM), [Hamari, Koivisto & Sarsa, 2014](https://ieeexplore.ieee.org/abstract/document/6758978?casa_token=F2o_LQE-CNgAAAAA:vA_xBEe0ltKmMPRmTfkyW78LThHP9hLKK06oj1gKpOeDfoCTG7l_p-KSVlcdhNpaErLjzrm8p90)). -Zichermann & Linder (2010) argue that that intrinsic motivation is unreliable and variable. Thereby, gamification can craft extrinsic motivators to internalize the intrinsically motivated behavior. It is crucial that this is done with non-economic incentives, because monetary incentives could lead to the crowding-out of intrinsic motivation ([Gneezy & Rustichini, 2000](https://academic.oup.com/qje/article-abstract/115/3/791/1828156)). A field where gamification has not yet been (explicitly) applied systematically is voting behavior (i.e., governance participation). One notable exception is a large-scale experiment with 61 million users of facebook, where researchers found that an *I voted* indication on their status page, could have been responsible for about 340'000 additional voters in the 2010 election ([Bond et al., 2012](https://www.nature.com/articles/nature11421) and [this article](https://www.nature.com/news/facebook-experiment-boosts-us-voter-turnout-1.11401)). The main driver here is considered to be peer-pressure elicited on facebook friends. While the researchers did not explicitly link this intervention with gamification, it could be perceived as such and might also work to incentivize participation of a small group. A similar application is the famous *I voted* badge in US elections, which has proven to be successful ([see](https://www.usvotefoundation.org/voter-reward-badge)). Voters like to show off badge and motivate others to go as well (some shops even offer perks for customers having that badge). +In recent years, gamification has received growing attention, so much that it has been called the "most notable technological developments for human +engagement" ([Majuri et al., 2018](https://trepo.tuni.fi/bitstream/handle/10024/104598/gamification_of_education_2018.pdf)). Gammification is used to enhance learning outcomes (e.g., [Denny, 2013](https://dl.acm.org/doi/abs/10.1145/2470654.2470763?casa_token=XsWtSZeFt-QAAAAA:MPWbtFfjzQZgWzyTI9hWROarJb1gJDWqDHNG4Fyozzvz3QIK-kMuMxfSwE26y9lKYUuZnV7aDZI)), model online communities (e.g., [Bista, 2012a](https://ieeexplore.ieee.org/abstract/document/6450959)), and promote sustainable behavior (e.g., [Berengueres et al., 2013](https://ieeexplore.ieee.org/abstract/document/6483512?casa_token=tmdUK7mtSSEAAAAA:ZxJnvYNAcuRaMHbwNqTJnahpbxal9xc9kHd6mY4lIahFhWn2Gmy32VDowMLVREQjwVIMhd9wcvY)). Gamification can serve as a "means of supporting user engagement and enhancing positive patterns in service use, such as increasing user activity, social interaction, or quality and productivity of actions" ([Hamari, Koivisto & Sarsa, 2014](https://ieeexplore.ieee.org/abstract/document/6758978?casa_token=F2o_LQE-CNgAAAAA:vA_xBEe0ltKmMPRmTfkyW78LThHP9hLKK06oj1gKpOeDfoCTG7l_p-KSVlcdhNpaErLjzrm8p90)). + +While a universally accepted definition is still lacking, gammification is best described as "a process of enhancing a service with affordances for gameful experiences in order to support user's [sic] overall value creation” ([Huotari & Hamari, 2012, p. 19](https://dl.acm.org/doi/abs/10.1145/2393132.2393137?casa_token=MU2yq2P4TOoAAAAA:Xuy9ZEzo2O7H-WCbqMheezkrodpab2DlFWkLjVt3jYExuP--vsjEROt4BKt5ZEbVou9rVnQSQBs)). In other words, applying gammification does not turn a service into a game, it rather enriches it with motivational elements popular in gaming, such as points, levels, badges, and leaderboards ([Deterding, 2012](https://dl.acm.org/doi/fullHtml/10.1145/2212877.2212883?casa_token=B9RD9ZPneIMAAAAA:34lrdGKwOUZyZu8fLobERuPLIBzNQxxwlgWLJnonn5Ws8Ya65aO_pdifhlHiSBwjDb0mWyFD0aM), [Hamari, Koivisto & Sarsa, 2014](https://ieeexplore.ieee.org/abstract/document/6758978?casa_token=F2o_LQE-CNgAAAAA:vA_xBEe0ltKmMPRmTfkyW78LThHP9hLKK06oj1gKpOeDfoCTG7l_p-KSVlcdhNpaErLjzrm8p90)). +Zichermann & Linder (2010) argue that intrinsic motivation is unreliable and variable. Therefore, gamification can be used to design extrinsic motivators that support the internalization of intrinsically motivated behavior. It is crucial to rely on non-economic incentives, as monetary rewards may crowd out intrinsic motivation ([Gneezy & Rustichini, 2000](https://academic.oup.com/qje/article-abstract/115/3/791/1828156)). + +One field where gamification has not yet been systematically applied is voting behavior, particularly governance participation. A notable exception is a large-scale experiment involving 61 million Facebook users, where researchers found that an *I voted* indication on users' status pages may have led to approximately 340,000 additional voters in the 2010 U.S. election ([Bond et al., 2012](https://www.nature.com/articles/nature11421) and [this article](https://www.nature.com/news/facebook-experiment-boosts-us-voter-turnout-1.11401)). The main driver was considered to be peer pressure among Facebook friends. While researchers did not explicitly link this intervention to gamification, it can be interpreted as such, and may have incentivize participation among a small group. + +A similar example is the well-known *I voted* badge used in U.S. elections, which has proven quite [successful](https://www.usvotefoundation.org/voter-reward-badge). Voters enjoy displaying the badge and often motivate others to vote as well. Some businesses even offer perks to customers who show the badge. + + A review of 91 scientific studies on gamificaton in education revealed that 71% reported mainly positive effects, such as increased engagement and enjoyment, while 25% showed no significant effect, and only 3% reported negative outcomes ([Majuri et al., 2018](https://trepo.tuni.fi/bitstream/handle/10024/104598/gamification_of_education_2018.pdf)). While acknowledging the effectiveness is context-dependent, and despite the overwhelming majority of positive results, some studies do not report negative effects, highlighting important caveats. One source of negative impact is the heightened perception of competition among peers, which can demotivate certain users ([Hakulinen et al., 2013](https://ieeexplore.ieee.org/abstract/document/6542238)). Another common critique is the lack of a clear theoretical foundation, leading to diverse and inconsistent approaches across studies. + +The design process behind gamification elements can be further informed by research from related social science, particularly in countering psychological biases that affect decision-making in small committees, and in leveraging additionally motivational factors such as *loss-aversion* and the resulting *endowment effect*. - A review on 91 scientific studies reveals that gamification provides overall positive effects in 71% of cases, 25% of cases no effect and only in 3% of studies were negative results reported ([Majuri et al., 2018](https://trepo.tuni.fi/bitstream/handle/10024/104598/gamification_of_education_2018.pdf)). such as increased engagement and enjoyment, while awcknowledging that the effectiveness is context-dependent. Despite the overwhelming majority of positive results, some studies indicate negative effects of gamification and suggest that there are some caveats. One source of negative effects are higher perceived competition of the interaction with peers, which could demotivate some users ([Hakulinen et al., 2013](https://ieeexplore.ieee.org/abstract/document/6542238)). Another reason for critique is the lack of clear theoretical foundation and the resulting diverse approach to the questions. +Literature has shown that small decision-making groups often suffer from *groupthink*, a bias in which the outcome of the decision process is far from optimal, as individuals do not freely express their opinions ([Janis, 1971](http://agcommtheory.pbworks.com/f/GroupThink.pdf)) or are influenced to act against their better judgement, whether consciously or unconsciously. This issue is particularly pronounced in groups comprised of members with differing levels of power and status. Major disasters have been attributed to *groupthink*, including the *Bay of Pigs Invasion* and the *Space Shuttle Challenger disaster* ([Janis, 1991](https://williamwolff.org/wp-content/uploads/2016/01/griffin-groupthink-challenger.pdf)). -The design process of the gamification elements can be further influenced by insights from related social science research. Namely how to counter some psychological biases affecting decision making in small committees as well as leveraging additionally motivational factors generated by *loss-aversion* and the resulting *endowment effect*. +Subsequent analyses revealed that committe members often willfully ignored substantial evidence. This problem is closely linked to the pressure to conform to authority figures, as demonstrated by well-known psychological experiments such as those conducted by [Milgram, 1963](https://www.demenzemedicinagenerale.net/pdf/MilgramOriginalWork.pdf) and [Asch, 1961](https://psycnet.apa.org/record/1952-00803-001). -Literature has shown, that small decision making groups tend to suffer from *group think*. This bias describes the situation, where the outcome from the decision process is far from optimal, because the individuals of the group do not speak their opinions freely ([Janis, 1971](http://agcommtheory.pbworks.com/f/GroupThink.pdf)) or are influenced in a way that they act against their best knowledge (consciously or unconsciously). This issue arises especially in groups comprised of members with different power and status. Major disasters have been accounted to *group think*, such as the *Bay of Pigs Invasion* and the *Space Shuttle Challenger disaster* ([Janis, 1991](https://williamwolff.org/wp-content/uploads/2016/01/griffin-groupthink-challenger.pdf)). In later analyses it was found that there were plenty of evidence available, which had been willingful neglected by committee members. This problem is also related to the pressure to behave conform with authority figures, as illustrated by famous psychological experiments (e.g., [Milgram, 1963](https://www.demenzemedicinagenerale.net/pdf/MilgramOriginalWork.pdf), [Asch, 1961](https://psycnet.apa.org/record/1952-00803-001)). It is crucial to keep that in mind, to mitigate that problem by dividing the final decision between important stake-holders. However, knowing about this issue, we can implement mechanisms to further improve the outcome of the decision making. A study by [MacDougall & Baum (1997)](https://journals.sagepub.com/doi/abs/10.1177/104973239700700407) has shown that explicitly announcing a "devil's advocate" can improve the outcome by challenging the consensus frequently. +It is crucial to remain aware of these dynamics and mitigate them by distributing final decision-making power among key stake-holders. With this awareness, mechanisms can be implemented to further improve decision outcomes. For example, a study by [MacDougall & Baum (1997)](https://journals.sagepub.com/doi/abs/10.1177/104973239700700407) demonstrated that explicitly appointing a "devil's advocate" can enhance results by regularly challenging group consensus. -Studies in behavioral economics further show that individual decision making is influenced by *loss-aversion*. This results from a non-linear utility function with different shapes in the gain and loss domain of a subjective evaluation of an outcome relative to some reference point. Specifically, the absolute dis-utility of a loss is higher than the gain in utility of a corresponding gain ([Kahneman & Tversky, 1992](https://link.springer.com/article/10.1007/BF00122574)). A resulting effect of that is the *endowment effect* ([Kahneman, Knetsch & Thaler, 1990](https://www.journals.uchicago.edu/doi/abs/10.1086/261737)), which describes the situation where a good is valued much more only because of the fact of possessing it. A practical implication for design of incentive systems is that users are exerting higher effort to keep something once there is the option to lose it again. +Studies in behavioral economics show that individual decision-making is influenced by *loss-aversion*. This phenomenon arises from a non-linear utility function, where the subjective evaluation of outcomes differs between gains and lossess relative to a reference point. Specifically, the disutility of a loss is greater than the utility gained from an equivalent gain ([Kahneman & Tversky, 1992](https://link.springer.com/article/10.1007/BF00122574)). One consequence of this is the *endowment effect* ([Kahneman, Knetsch & Thaler, 1990](https://www.journals.uchicago.edu/doi/abs/10.1086/261737)), which describes the tendency to value an item simply because one possesses it. A practical implication for incentive system design is that users tend to exert greater effort to retain something when there is a possibility of losing it. -In conclusion, a carefully designing gamified experience can improve the overall governance process and result in more active discussions, and hopefully better decisions. +In conclusion, carefully designing a gamified experience can enhance the overall governance process, leading to more active discussions, and, ideally, better decisions. ## Awarding mechanism (WIP) -In general, the most commonly used gamification elements are ([Hamari, Koivisto & Sarsa, 2014](https://ieeexplore.ieee.org/abstract/document/6758978?casa_token=F2o_LQE-CNgAAAAA:vA_xBEe0ltKmMPRmTfkyW78LThHP9hLKK06oj1gKpOeDfoCTG7l_p-KSVlcdhNpaErLjzrm8p90)): +As [Hamari, Koivisto & Sarsa, 2014](https://ieeexplore.ieee.org/abstract/document/6758978?casa_token=F2o_LQE-CNgAAAAA:vA_xBEe0ltKmMPRmTfkyW78LThHP9hLKK06oj1gKpOeDfoCTG7l_p-KSVlcdhNpaErLjzrm8p90) have pointed out, the most commonly used gamification elements generally include: * Points -* Badges (Trophies) +* Badges (or trophies) * Achievements * Levels -A very complex task is to design an automatic mechanism to award members NFTs based on their on-chain (and potentially off-chain) behavior. On the one hand, focusing only on easily measurable outcome levels of participation (e.g., speed of voting, pure quantity of propositions) can easily backfire and are prone to be abused. In addition, it is hard to deduce the quality of a vote by those quantitative measurements. To mitigate this, it is important to observe the whole process and the later outcome of the election. +Designing an automatic mechanism to award members NFTs based on their on-chain (and potentially off-chain) behavior is highly complex. On one hand, focusing solely on easily measurable outcomes, such as voting speed or the sheer number of propositions, can backfire and is prone to abuse. Moreover, assessing the quality of a vote through quantitative metrics alone is challenging. To address this, it is essential to observe the entire process and the eventual outcome of the election. -On the other hand, only incentivizing positive election outcomes could make members too conservative, only proposing winners, neglecting provocative but potentially beneficial proposals. The best strategy is to come up with a mix of different NFTs where the positive weights of the individual NFTs are less severe and therefore leave enough space for all behavior. +On the other hand, incentivizing only positive election outcomes could lead members to become overly conservative, proposing only safe, likely-to-win ideas while neglecting provocative but potentially beneficial proposals. The best strategy, therefore, is to design a mix of different NFTs, where the positive weighting of each individual NFTs is less pronounced, allowing room for a broader range of behaviors. -In addition, the proposed NFTs should also incorporate important insights from social science research (as mentioned above e.g., to incorporate preventive measures against *Groupthink* or design some NFTs to leverage *Loss-Aversion*). +In addition, the proposed NFTs should incorporate important key insights from social science research, as mentioned above. For example, some NFTs could include preventive measures against *groupthink*, while others could be designed to leverage *Loss-Aversion*. ### Achievements (static) -Achievements are absolute steps to be reached and cannot be lost, once obtained. Potential triggers could be: +Achievements are absolute milestones that, once reached, cannot be lost. Potential triggers include: -* Become a collective member of a certain age +* Becoming a collective member of a certain age ### Badges (perishable) -Generally, Badges are perishable and resemble an achievement relative to something. This means, once the relative status is lost, so is the badge. This is a very interesting concept as it incorporates the motivating factor of the *endowment-effect* (see literature section), where individuals exert higher motivation to hold on to the badge. +Badges are generally perishable and resemble an achievement relative to a specific status or condition. In other words, once the relative status is lost, the badge is forfeited. This dynamic introduces an intriguing motivational factor known as the *endowment-effect*. -Those are good to include states of the situation such as: +Badges are well-suited to reflect situational states such as: -* Be the most backed member (if there is some hierarchy in the system) -* Be the oldest member -* The devil's advocate (frequently vote against the majority of other members) +* Being the most backed member (if a hierarchy exists within the system) +* Being the oldest member +* Acting as the devil's advocate (frequently voting against the majority) ### Levels (ranks) -Gaining certain badges could also mean we can implement some level system which could essentially sum up all the badges and achievements into one quantifiable metric. +Earning certain badges opens the possibility of implementing a level system that could essentially sum up all badges and achievements into one quantifiable metric. ### Actions -The following list, composed by Raul Romanutti, illustrates several frequent actions members can perform and build a good basis of outcome variables to be entwined in an awarding mechanism. This is highly context-specific but might give some examples and are suited to treasury spendings and other proposals. +The following list, compiled by Raul Romanutti, highlights several common actions that members can perform, offering a solid basis of outcome variables to be integrated into an awarding mechanism. While highly context-specific, the list may serve as a useful reference for treasury expenditures and other proposals. * Vote on a treasury proposal motion * Vote on a runtime upgrade motion -* Vote on referendum +* Vote on a referendum * Submit an external motion proposal * Submit a preimage for a proposal * Close a motion after majority is reached -* Vote on a treasury proposal motion (proposed by community members) -* Endorse a tip proposal (proposed by community members) -* Open a tip to a community member +* Vote on a treasury proposal motion (submitted by community members) +* Endorse a tip proposal (submitted by community members) +* Open a tip for a community member * Open a bounty proposal * Vote on a bounty proposal -* Vote on a Bounty curator nomination -* Open a motion to unassign a bounty curator -* Become the curator of an active bounty +* Vote on a bounty curator nomination +* Propose a motion to unassign a bounty curator +* Serve as the curator of an active bounty * Propose an external motion for a specific chain to use a common-good chain slot * Vote on an external motion for a specific chain to use a common-good chain slot ## NFT Gallery -A prerequisite for NFTs to develop their motivating effect, it is necessary to visually display them and make them viewable in NFT galleries. This requires the support of wallets and explorers. Due to the popularity of NFTs, many of projects are currently working on those solutions and it is expected that solutions will further improve. +For NFTs to develop a motivating effect, they must be visually displayed and accessible through NFT galleries. Support from wallets and blockchain explorers is essential to achieve this. Given the popularity of NFTs, many projects are actively developing such solutions, from which further improvements are expected. -As an additional benefit, governance focused applications could orderly display the members, their achievements / badges and levels, which can make it also much more easy and enjoyable for outsiders of the decision-making process to compare and engage with the collective members. This could substantially improve the engagement of members, and results are more precise in representing the opinion of all stakeholders. This, in turn, would further increase the incentives exerted by the NFTs on the members. +As an additional benefit, governance-focused applications could present members, their achievements, badges, and levels in an organized an appealing way. This would make it easier and more enjoyable for outsiders, those not direcly involved in the decision-making process, to compare and engage with collective members. A possible outcome would be a substantial improvement in member engagement, leading to a more accurate representation of all stakeholders' opinions. In turn, this could further enhance the incentives that NFTs offer to members. +**For inquieries or questions please contact**: [Jonas Gehrlein](/team_members/Jonas.md) diff --git a/docs/Polkadot/economics/5-utilitytokendesign.md b/docs/Polkadot/economics/5-utilitytokendesign.md index c8ac55ab..ef74e37b 100644 --- a/docs/Polkadot/economics/5-utilitytokendesign.md +++ b/docs/Polkadot/economics/5-utilitytokendesign.md @@ -2,18 +2,18 @@ title: Utility Token Design --- -**Authors**: Samuel Häfner + -**Last updated**: October 13, 2021 -**Paper Link:** [[SSRN]](http://ssrn.com/abstract=3954773) +Utility tokens are cryptographic tokens primarily used to access and consume services offered by the token issuer. One of the objectives of this [project](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3954773) was to analyze general design principles of utility tokens native to proof-of-stake blockchains. -In this project, I analyze some general design principles of utility tokens that are native to a proof-of-stake blockchain. Utility tokens are cryptographic tokens whose main economic use is to access and consume the respective token issuer’s services. +The Polkadot network provides services through parachain slots, which offer shared security and interoperability with other parachains. To secure a slot, users (namely the teams building on Polkadot) must stake DOT tokens in recurring slot auctions. -The services offered by the Polkadot network consist of parachain slots, which come with shared security and means to communicate with other parachains. To obtain one of the slots, the users --- i.e., the teams building on Polkadot --- need to put forth DOTs in recurrent slot auctions. +The analysis required a dynamic general equilibrium model of utility tokens, which served as a medium for consuming services on a two-sided market platform. -For the analysis, I set up a dynamic general equilibrium model of utility tokens that serve as a means to consume services on a two-sided market platform. +On the one side of the platform, users derive utility from consuming the services it provides. On the other side, validators supply the necessary security and are compensated with tokens. Validators must regularly sell a portion of their tokens to cover operational costs, while users must continually purchase tokens to access services. A token market facilitates the balance between token supply and demand. -On the one side of the platform, there are users that derive utility from consuming the services provided by the platform. On the other side, there are validators that provide the required security and receive tokens in return. Validators need to repeatedly sell some of their tokens to cover their costs; users need to repeatedly buy tokens to consume the services. A token market balances token supply and token demand. +The main results of the analysis are as follows: First, utility token markets are generally efficient, as they lead to the socially optimal provision of services. Second, a key tension has been identified between the dynamics of utility token value, specifically between the evolution of the services provided and the payment mechanisms on the users’ side. + +**For inquieries or questions please contact**: -The main results of the analysis are the following: First, I find that utility token markets are generally efficient because they result in the socially optimal provision of services. Second, I uncover a tension between the dynamics of utility tokens' value, the evolution of the provided services, and the payment details on the users’ side. diff --git a/docs/Polkadot/economics/Combinatorial-candle-auction.png b/docs/Polkadot/economics/Combinatorial-candle-auction.png new file mode 100644 index 00000000..729b2af5 Binary files /dev/null and b/docs/Polkadot/economics/Combinatorial-candle-auction.png differ diff --git a/docs/Polkadot/economics/Economics.png b/docs/Polkadot/economics/Economics.png new file mode 100644 index 00000000..b338e819 Binary files /dev/null and b/docs/Polkadot/economics/Economics.png differ diff --git a/docs/Polkadot/economics/Experimental-investigations.png b/docs/Polkadot/economics/Experimental-investigations.png new file mode 100644 index 00000000..e96236c2 Binary files /dev/null and b/docs/Polkadot/economics/Experimental-investigations.png differ diff --git a/docs/Polkadot/economics/collective-members.png b/docs/Polkadot/economics/collective-members.png new file mode 100644 index 00000000..4383e64f Binary files /dev/null and b/docs/Polkadot/economics/collective-members.png differ diff --git a/docs/Polkadot/economics/index.md b/docs/Polkadot/economics/index.md index 07abdc8e..e65a9c32 100644 --- a/docs/Polkadot/economics/index.md +++ b/docs/Polkadot/economics/index.md @@ -4,8 +4,8 @@ title: Economics import DocCardList from '@theme/DocCardList'; -This chapter covers the economic research done at the Web3 Foundation. +The first section, Token Economics, introduces Polkadot's high-level design and architecture, including parachain functionality and interoperability with external blockchains. The following sections showcase the economic research conducted by the Web3 Foundation. Applying tools from microeconomics, behavioral economics, and game theory, the research team analyzes different aspects of the core Polkadot protocol. Other sections included here pertain to token economics and parachain allocation. -We use tools from microeconomics, behavioral economics, and game theory to analyze different aspects of the protocol. +![](Economics.png) diff --git a/docs/Polkadot/economics/parachain-auctions.png b/docs/Polkadot/economics/parachain-auctions.png new file mode 100644 index 00000000..7b93a071 Binary files /dev/null and b/docs/Polkadot/economics/parachain-auctions.png differ diff --git a/docs/Polkadot/economics/utility-token.png b/docs/Polkadot/economics/utility-token.png new file mode 100644 index 00000000..912e1125 Binary files /dev/null and b/docs/Polkadot/economics/utility-token.png differ diff --git a/docs/Polkadot/economics/validator-selection.jpeg b/docs/Polkadot/economics/validator-selection.jpeg new file mode 100644 index 00000000..d673dd0d Binary files /dev/null and b/docs/Polkadot/economics/validator-selection.jpeg differ diff --git a/docs/Polkadot/index.md b/docs/Polkadot/index.md new file mode 100644 index 00000000..02007dae --- /dev/null +++ b/docs/Polkadot/index.md @@ -0,0 +1,11 @@ +--- +title: Polkadot is in the house +--- + +import DocCardList from '@theme/DocCardList'; + +The research conducted at Web3 Foundation primarily focuses on three main areas: the economics, protocols, and security of the Polkadot platform. + +![](Polkadot-Crypto.png) + + \ No newline at end of file diff --git a/docs/Polkadot/overview/2-token-economics.md b/docs/Polkadot/overview/2-token-economics.md deleted file mode 100644 index ee31fd6f..00000000 --- a/docs/Polkadot/overview/2-token-economics.md +++ /dev/null @@ -1,319 +0,0 @@ ---- -title: Token Economics ---- - -**Authors**: [Alfonso Cevallos](/team_members/alfonso.md), [Jonas Gehrlein](/team_members/Jonas.md) - -**Last Updated**: October 17, 2023 - -Polkadot will have a native token called DOT. Its main functions are as follows: - -1. Economics: Polkadot will mint or burn DOTs in order to reward the nodes that run the consensus protocol, to fund the treasury, to control the inflation rate, etc. - -2. Slashing: DOTs also play a role in the slashing protocols designed to desincentivize attacks or adversarial behaviors. - -3. Governance: DOTs are also used as voting power, to let DOT holders express their opinion in governance decisions via referenda. - -3. Parachain allocation: Finally, DOTs are used to decide which projects are allocated a parachain slot, via auctions and deposits. - - -In this section we focus on the first use above, while each of the other three uses is analyzed in a separate section. - -## Introduction - -Polkadot is a proof-of-stake based platform where a set of validators, who have staked DOTs, produce blocks and reach consensus. If a validator steers away from the protocol, some of his DOTs are slashed, but otherwise he gets paid for their contribution (roughly) proportional to his staked DOTs. The set of nodes elected as validators changes constantly (in each era, i.e. around once a day), but the number remains limited. However, any number of DOT holders can also participate indirectly in the decision-making processes as *nominators*, in what we call *nominated proof-of-stake*. A nominator indicates which validator candidates she trusts, and puts some DOTs at stake to support her nomination. If one or more of her nominated candidates are elected as validators in an era, she shares with them any economical rewards or punishments, proportional to her stake. Being a nominator is a way of investing one's DOTs, and of helping in the security of the system. Indeed, the larger the total amount of DOTs staked by nominators and validators, the higher the system security, because an adversary needs that much more stake - or nominators' trust - before it gets any nodes elected as validators. - -We therefore aim at having a considerable percentage of the total DOT supply be staked by validators and nominators. Another large percentage of the DOT supply will be frozen as deposits by the commercial blockchains who get a parachain slot. - -## Organization - -This note contains the following subsections. - -* **NPoS payment and inflation:** We describe how we reward well-behaving validators and nominators in our nominated proof-of-stake. Since the DOT minting for this end is the main cause of inflation in the system, we also describe our inflation model here. **Note, that the currently implemented inflation model has different parameters.** -* **Transaction fees:** We analyse the optimal transaction fees on the relay chain to cover for costs, discourage harmful behaviors, and handle eventual peaks of activity and long inclusion times. -* **Treasury:** We discuss how and when to raise DOTs to pay for the continued maintenance of the network. - -Finally, in the last paragraph of the note we provide links to additional relevant references about the Polkadot protocol. - -## NPoS payments and inflation - -We consider here payments to validators and nominators for their participation in the protocols of block production (BABE) and finality (GRANDPA). We consider only the payments coming from minting new tokens, in normal circumstances. In particular we do not consider slashings, rewards to misconduct reporters and fishermen, or rewards from transaction fees. These will be considered in other sections. - -As these payments are the main driver of inflation in the system, we first study our inflation model. Note that we suggest two sets of adjustable parameters: One for the eventual situation of launched parachains and one for the meantime, where liquidity is not constrained by parachain bonds. - -### Inflation model - -Let $x$ be the *staking rate* in NPoS at a particular point in time, i.e. the total amount of tokens staked by nominators and validators, divided by the total token supply. $x$ is always a value between 0 and 1. - -__Adjustable parameter:__ Let $\chi_{ideal}$ be the staking rate we would like to attain ideally in the long run. If it falls, the security is compromised, so we should give strong incentives to DOT holders to stake more. If it rises, we lose liquidity, which is also undesirable, so we should decrease the incentives sharply. - -Let $i=i(x)$ be the yearly *interest rate* in NPoS; i.e., the total yearly amount of tokens minted to pay all validators and nominators for block production and Grandpa, divided by the total amount of tokens staked by them. We consider it as a function of $x$. Intuitively, $i(x)$ corresponds to the incentive we give people to stake. Hence, $i(x)$ should be a monotone decreasing function of $x$, as less incentive is needed when $x$ increases. - -* We study the yearly interest rate (instead of the interest rate per block or per epoch) for ease of comprehension. This means that $i(x)$ is the total payout perceived by somebody that continuously stakes one DOT during a year. The interest rate per block can be easily computed from it. -* Not every staked party will be paid proportional to their stake. For instance, a validator will be paid more than a nominator with equal stake, and a validator producing a block will be temporarily paid more than a validator not producing a block. So, $i(x)$ only works as a guide of the average interest rate. - -__Adjustable parameter:__ Let $i_{ideal}:=i(\chi_{ideal})$ be the interest rate we pay in the ideal scenario where $x=\chi_{ideal}$, which is indirectly determined by the overall inflation of the system. - -Let $I$ be the yearly *inflation rate*; i.e. - -$$ -I=\frac{\text{token supply at end of year} - \text{token supply at beginning of year}}{\text{token supply at beginning of year}} -$$ - -The inflation rate is given by - -$$ -I=I_{NPoS}+I_{treasury}-I_{slashing} - I_{tx-fees} -$$ - -where $I_{NPoS}$ is the inflation caused by token minting to pay nominators and validators, $I_{treasury}$ is the inflation caused by minting for treasury, $I_{slashing}$ is the deflation caused by burning following a misconduct, and $I_{tx-fees}$ is the deflation caused by burning transaction fees. - -* The rewards perceived by block producers from transaction fees (and tips) do not come from minting, but from tx senders. Similarly, the rewards perceived by reporters and fishermen for detecting a misconduct do not come from minting but from the slashed party. This is why these terms do not appear in the formula above. - -$I_{NPoS}$ should be by far the largest of these amounts, and thus the main driver of overall inflation. Notice that by channelling all of the tokens destined to burning -due to both slashing and transaction fees- into treasury, we decrease the other terms in the formula (see the section on treasury). If we consider $I_{NPoS}$ as a function of the staking rate $x$, then clearly the relation between $I_{NPoS}(x)$ and $i(x)$ is given by - -$$ -I_{NPoS}(x)=x\cdot i(x) -$$ - -From our previous analysis, we can see that $I_{NPoS}(\chi_{ideal})=\chi_{ideal}\cdot i_{ideal}$. Since we want to steer the market toward a staking rate of $x=\chi_{ideal}$, it makes sense that the inflation rate **$I_{NPoS}(x)$ should be maximal at this value**. - -__Adjustable parameter:__ Let $I_0$ be the limit of $I_{NPoS}(x)$ as $x$ goes to zero (i.e. when neither validators nor nominators are staking any DOTs). The value of $I_0$ shoud be close to zero but not zero, because we need to make sure to always cover at least the operational costs of the validators, even if nominators get paid nothing. Hence, $I_0$ represents an estimate of the operational costs of all validators, expressed as a fraction of the total token supply. We will make sure that $I_{NPoS}(x)$ is always above $I_0$ for all values of $x$, in particular also in the limit when $x$ goes to one. - -For simplicity, we propose that the inflation function grow linearly between $x=0$ and $x=\chi_{ideal}$. On the other hand, we propose that it decay exponentially between $x=\chi_{ideal}$ and $x=1$. We choose an exponential decrease for $I_{NPoS}(x)$ because this implies an exponential decrease for $i(x)$ as well, and we want the interest rate to fall sharply beyond $\chi_{ideal}$ to avoid illiquidity, while still being able to control its rate of change, $i(x+\varepsilon)/i(x)$, when $x$ increases by a small amount $\varepsilon$. Bounding how fast the interest rate changes is important for the nominators and validators. - -__Adjustable parameter:__ Define the *decay rate* $d$ so that the inflation rate decreases by at most 50% when $x$ shifts $d$ units to the right of $\chi_{ideal}$, i.e. $I_{NPoS}(\chi_{ideal} + d) \geq I_{NPoS}/2$. We suggest $d=0.05$. - - From the previous discussion, we propose the following interest rate and inflation rate functions, which depend on the parameters $\chi_{ideal}$, $i_{ideal}$, $I_0$ and $d$. Let - -$$ -\begin{align} -I_{NPoS}(x) &= \begin{cases} -I_0 + \Big(I_{NPoS}(\chi_{ideal}) - I_0\Big)\frac{x}{\chi_{ideal}} -&\text{for } 0 - -Note, that the curves shift based for other values of $\chi_{ideal}$. Different parameters can be applied following this link: https://www.desmos.com/calculator/2om7wkewhr - - -### Payment details - -There are several protocols that honest validators are involved in, and we reward their successful participation or slash their lack thereof (whichever is easier to detect). From this point of view, we decide to reward validators (and their nominators) only for *validity checking* and for *block production*, because they are easy to detect. - -In the branch of validity checking, we reward: - -* a parachain validator for each validity statement of the parachain block that it issues. - -In the branch of block production, we reward: - -* the block producer for producing a (non-uncle) block in the relay chain, -* the block producer for each reference to a previously unreferenced uncle, and -* the producer of each referenced uncle block. - -These are thus considered "payable actions". We define a point system where a validator earns a certain amount of points for each payable action executed, and at the end of each era they are paid proportional to their earned points. (The exact DOT value of each point is not known in advance because it depends on the total number of points earned by all validators in a given era. This is because we want the total payout per era to depend on the inflation model established above, and not on the number of payable actions executed). - -__Adjustable parameters:__ We propose the following point system: - -* 20 points for each validity statement, -* 20 points for each (non-uncle) block produced, -* 2 points (to the block producer) for each reference to a previously unreferenced uncle, and -* 1 point to the producer of each referenced uncle. - -Notice that what is important here is not the absolute points but the point ratios, which establish the reward ratios of the payable actions. These points are parameters to be adjusted by governance. - -In each era $e$, and for each validator $v$, we keep a counter $c_v^e$ on the number of points earned by $v$. Let $c^e = -\sum_{\text{validators } v} c_v^e$ be the total number of points earned by all validators in era $e$, and let $P^e_{NPoS}$ be our target total payout to all validators -- and their nominators -- in that era (see previous section on inflation model to see how to establish $P^e_{NPoS}$). Then, at the end of era $e$, the payout corresponding to validator $v$ and his nominators is given by - -$$ -\frac{c_v^e}{c^e} \cdot P^e_{NPoS} -$$ - -We remark that we can also use the counters to combat unresponsiveness: if a validator has earned close to zero points in payable actions during an era (or any other period of time being measured), we kick them out. See the note on Slashings for more details. - -### Distribution of payment within a validator slot - -In any given era, the stake of a nominator $n$ is typically distributed among several validators, e.g. 70% of $n$'s stake is assigned to validator 1, 20% to validator 2, 10% to validator 3, etc. This distribution is decided automatically by the NPoS validator election mechanism that runs at the beginning of each era (see notes on NPoS for details). - -If there are $m$ validators, then this stake distribution partitions the global stake pool into $m$ slots: one per validator. The stake in each validator slot is comprised of 100% of that validator's stake, and some fraction (possibly zero) of the stake of each nominator that approved of the validator. We sometimes refer to a validator's stake as "self-stake", to distinguish it from the *validator slot's stake*, which is typically much larger. In the previous subsection we explained how the payouts are assigned to each validator slot in a given era. In this subsection, we explain how this payout is distributed within a slot, i.e. among the validator and the nominators in it. Ultimately, a nominator's payout in a given era corresponds to the sum of her payouts with respect to each slot that contains some of her stake. - -We remark that, since none of the nominators or validators can individually control the above-mentioned stake partition into validator slots (which is decided automatically by the validator election mechanism) or the exact payouts (which depend on global parameters such as the staking rate), none of the participants knows in advance exactly how much reward they will get during an era. In the future, we might allow nominators to specify their desired interest rates. We block this feature for the time being to simplify the corresponding optimization problem that the validator election mechanism solves. - -We also remark that our mechanism takes as much of the nominators' available stake as possible; i.e. if a nominator has at least one of her approved validators elected, all of her available stake will be used. The idea is that the more stake, the more security we have. In contrast, we follow the policy that validator slots are paid equally for equal work, and they are NOT paid proportional to their stakes. So if a validator slot A has less stake than another validator slot B, then the parties in A are paid more per staked DOT. This should motivate nominators to rapidly adjust their preferences in future eras, to favor less popular validators, so that we can achieve a more balanced distribution of stake across validator slots (which is one of the main objectives of the validator election mechanism; see notes on NPoS for more details). This should also help new validator candidates have a better chance to get elected, which is important to ensure decentralization. - -Within a validator slot, the payment is as follows: First, validator $v$ is paid his "commission fee", which is an amount entirely up to $v$ to decide, and which is publicly announced in advance by him, before nominators reveal their votes for the era. This fee is intended to cover $v$'s operational costs. Then, the remainder is shared among all parties (i.e. $v$ and the nominators) proportional to their stake within the validator slot. In other words, when it comes to payment, validator $v$ is considered as two entities: a non-staked validator that is rewarded a fixed commission fee, and a staked nominator that is treated like any other nominator and rewarded pro rata. Notice that a higher commission fee set by the validator means a higher total payout for him and a lower payout to his nominators, but since this fee is publicly known in advance, nominators will prefer to back validators with low fees (all else being equal). We thus let the market regulate itself. On one hand, a validator candidate with a high commission fee risks not getting enough votes to be elected as validator. On the other hand, validators who have built a strong reputation of being reliable and high performing will likely get away with charging a higher fee (which is fair), as they will still be preferred over other validators. And for a nominator, supporting riskier validators will be correlated with more rewards (which makes sense). - - -## Relay-chain transaction fees and per-block transaction limits - -Some of the properties we want to achieve relative to relay-chain transactions are as follows: - -1. Each relay-chain block should be processed efficiently, even on less powerful nodes, to avoid delays in block production. -2. The growth rate of the relay chain state is bounded. 2'. Better yet if the absolute size of the relay chain state is bounded. -3. Each block has *guaranteed availability* for a certain amount of operational, high-priority txs such as misconduct reports. -4. Blocks are typically far from full, so that peaks of activity can be dealt with effectively and long inclusion times are rare. -5. Fees evolve slowly enough, so that the fee of a particular tx can be predicted accurately within a frame of a few minutes. -6. For any tx, its fee level is strictly larger than the reward perceived by the block producer for processing it. Otherwise, the block producer is incentivized to stuff blocks with fake txs. -7. For any tx, the processing reward perceived by the block producer is high enough to incentivize tx inclusion, yet low enough not to incentivize a block producer to create a fork and steal the transactions of the previous block. Effectively, this means that the marginal reward perceived for including an additional tx is higher than the corresponding marginal cost of processing it, yet the total reward for producing a full block is not much larger than the reward for producing an empty block (even when tips are factored in). - -For the time being, we focus on satisfying properties 1 through 6 (without 2'), and we leave properties 2' and 7 for a further update. We also need more analysis on property 2. - -The amount of transactions that are processed in a relay-chain block can be regulated in two ways: by imposing limits, and by adjusting the level of tx fees. We ensure properties 1 through 3 above by imposing hard limits on resource usage, while properties 4 through 6 are achieved via fee adjustments. These two techniques are presented in the following two subsections respectively. - - -### Limits on resource usage - -We identify four resources which can be consumed when processing a tx: - -* Length: data size of the tx in bytes within the relay-chain block, -* Time: time it takes to import it (i/o and cpu), -* Memory: amount of memory it takes when processing, -* State: amount of state storage increase it induces. - -Notice that unlike the other three resources which are consumed only once, state storage has a permanent cost over the network. Hence for state storage we could have rent or other Runtime mechanisms, to better match fees with the true cost of a tx, and ensure the state size remains bounded. This needs further consideration. We could also consider a mechanism that doesn't impose a hard limit on state increase but rather controls it via fees; however we prefer to add a limit for soundness, in order to avoid edge cases where the state grows out of control. - -**Adjustable parameters.** For the time being, we suggest the following limits on resource usage when processing a block. These parameters are to be further adjusted via governance based on real-life data or more sophisticated mechanisms. - -* Length: 5MB -* Time: 2 seconds -* Memory: 10 GB -* State: 1 MB increase - -In principle, a tx consumes some amount of the last three resources depending on its length, type, input arguments, and current state. However, for simplicity we decided to consider, for each transaction type, only the worst-case state, and only the byte length of its input arguments. Consequently, we classify transactions based on length, type and argument length, and run tests (based on worst-case state) to examine their typical resource usage. - -For the time being, we are considering a model where every transaction within a block is processed in sequence. So, in order to ensure the block memory bound above, it is sufficient to ensure that each tx observes the memory bound. We make sure this is the case. However, in the future we may consider parallelism. - -To simplify our model further, we define a tx *weight* as a parameter that captures the time usage and state increase of a tx. Specifically, we define a tx weight as the *max* of its typical time and state usage, each measured as a fraction of the corresponding block limit. Then, given a collection of txs, we will sum up their lengths on one hand, and their weights on the other hand, and we will allow them within the same block only if both limits are respected. This is a hard constraint on resource usage which must be respected in each block. - -We add a further constraint on resource usage. We distinguish between "normal" txs and "operational" txs, where the latter type corresponds to high-priority txs such a fisherman reports. A collection of normal txs is allowed within the same block only if both their sum of lengths and their sum of weights are below 75% of the respective limits. This is to ensure that each block has a guaranteed space for operational txs (at least 25% of resources). - -**Details about establishing typical resource usage for txs.** Length is easy to determine by inspection. For time and memory usage, we prepare the chain with the worst-case state (the state for which the time and memory requirements to import this tx type should be the largest). We generate 10k transactions for a given transaction type with input which should take the longest to import for that state, and we measure the mean and standard deviation for the resource usage with the Wasm environment. If the standard deviation is greater than 10% of the mean, we increase the sample space above 10k. Finally, state increase is by inspection, based on worst cases for a large sample of txs. - - -### Setting transaction fees - -We use the model described above to set the fee level of a tx based on three parameters: the tx type, its length, and its weight (parameters defined in the previous subsection). This fee differentiation is used to reflect the different costs in resources incurred per transaction, and to encourage/discourage certain tx market behaviors. - -As mentioned earlier, part of the tx fee needs to go to the block producer, to incentivize inclusion, but not all of it, so the block producer is discouraged from stuffing blocks with bogus txs. For simplicity, we originally suggest that 20% of each tx fee goes to the block producer, with the remaining 80% going to treasury. We remark that a fraction could also be set for burning, but we choose not to do so to keep better control of the inflation rate. In the future this percentage may be adjusted, and could be made dependent on the tx type, to encourage the block producer to include certain tx types without necessarily adjusting the fee. - -A transaction fee tx is computed as follows: - -$$ -fee(tx) = base\_fee + type(tx) \cdot length(tx) + c_{traffic} \cdot weight(tx) -$$ - -where $c_{traffic}$ is a parameter independent from the transaction, that evolves over time depending on the network traffic; we explain this parameter in the next subsection. Parameter $type(tx)$ depends on the transaction type only; in particular for operational transactions, we currently set $type(tx)$ to zero. - -Intuitively, the term $weight(tx)$ covers the processing cost of the block producer, while the term $type(tx) \cdot length(tx)$ covers the opportunity cost of processing one transaction instead of another one in a block. - -### Adjustment of fees over time - -The demand for transactions is typically quite irregular on blockchains. On one hand, there are peaks of activity at the scale of hours within a day or days within a month. On the other hand, there are long term tendencies. We need a mechanism that automatically updates the transaction fees over time taking these factors into consideration. By the law of supply and demand, raising the fee should decrease the demand, and vice-versa. - -To deal with peaks of activity, we face a trade-off between hiking up transaction fees rapidly or potentially having long transaction inclusion times - both undesirable effects. We propose two mechanisms. The first one adjusts the price very quickly, at the same pace as the peaks and valleys of activity. The second one adjusts slowly, at the pace of long term tendencies, and uses tipping to give users the possibility of controlling waiting times at peak hours. We propose to use the slow adjusting mechanism with tips, but provide details of both mechanisms for completeness. - -#### 1. Fast adjusting mechanism - -In this mechanism the transaction fees vary greatly through time, but are fixed for all users at each block (no tipping). - -Recall that we set a hard limit on the sum of lengths and weights of all transactions allowed on a block. We also set a second hard limit, this time on the sum of lengths and weights of "normal" txs (non-operational txs), which is equal to 75% of the first limit. - -**Definition.** We define a block's saturation level (relative to normal txs) as a fraction $s$ between 0 and 1 which describes how close the limit on normal txs is from being full. Explicitly, the saturation level of a block $B$ is - -$$ -s(B):=\max\{\frac{\sum_{\text{normal } tx \in B} length(tx)}{\text{normal length limit}}, \frac{\sum_{\text{normal } tx \in B} weight(tx)}{\text{normal weight limit}}\} -$$ - -where the normal length limit (the block length limit on normal transactions) is 75% of the overall length limit, and the normal weight limit is 75% of the overall weight limit. - -**Adjustable parameter** Let $s^*$ be our target block saturation level. This is our desired long-term average of the block saturation level (relative to normal txs). We originally suggest $s^*=0.25$, so that blocks are 25% full on average and the system can handle sudden spikes of up to 4x the average volume of normal transactions. This parameter can be adjusted depending on the observed volumes during spikes compared to average volumes, and in general it provides a trade-off between higher average fees and longer transaction inclusion times during spikes. - -Recall that a transaction fee is computed as $fee(tx) = base\_fee + type(tx) \cdot length(tx) + c_{traffic} \cdot weight(tx)$, for a parameter $c_{traffic}$ that is independent of the transaction. Let $s$ be the saturation level of the current block. If $s>s^*$ we slightly increase $c_{traffic}$, and if $s diff --git a/docs/Polkadot/protocols/Grandpa.png b/docs/Polkadot/protocols/Grandpa.png new file mode 100644 index 00000000..59f4a67c Binary files /dev/null and b/docs/Polkadot/protocols/Grandpa.png differ diff --git a/docs/Polkadot/protocols/LightClientsBridges.md b/docs/Polkadot/protocols/LightClientsBridges.md index b24d2db9..1a5cb93f 100644 --- a/docs/Polkadot/protocols/LightClientsBridges.md +++ b/docs/Polkadot/protocols/LightClientsBridges.md @@ -1,9 +1,22 @@ # Accountable Light Client Systems for Secure and Efficient Bridges -**Authors**: Oana Ciobotaru +![](secure-and-efficient-bridges.png) -A major challenge for blockchain interoperability is having an on-chain light client protocol that is both efficient and secure. We present [a protocol that provides short proofs about the state of a decentralised consensus](https://eprint.iacr.org/2022/1205) while being able to detect misbehaving parties. To do this naively, a verifier would need to maintain an updated list of all participants' public keys which makes the corresponding proofs long. In general, existing solutions either lack accountability or are not efficient. We define and design a committee key scheme with short proofs that do not include any of the individual participants' public keys in plain. Our committee key scheme, in turn, uses a custom designed SNARK which has a fast prover time. Moreover, using our committee key scheme, we define and design an accountable light client system as the main cryptographic core for building bridges between proof of stake blockchains. Finally, [we implement a prototype of our custom SNARK](https://github.com/w3f/apk-proofs) for which we provide benchmarks. +One of the key challenges in blockchain interoperability is establishing an efficient and secure on-chain light client protocol. To address this, researchers at the Web3 Foundation have developed a [protocol](https://eprint.iacr.org/2022/1205) that generates short proofs of the state of a decentralized consensus, while also enabling detection of misbehaving parties. -More concretely, we aim to use the solution described above for building a BLS-based bridge between Kusama and Polkadot. The light client verifier of any such bridge would be [GRANDPA-based](https://github.com/paritytech/grandpa-bridge-gadget/blob/master/docs/beefy.md) and, if designed naively, would require verifying hundreds of signatures for every justification. Using aggregation of BLS signatures, we can reduce this to verifying one signature against hundreds of public keys. In our solution linked above, we do not need to communicate either hundreds of public keys or hundreds of signatures. +A straightforward albeit naive approach would require the verifier to maintain an up-to-date list of all participants' public keys, resulting in lengthly proofs. While existing solutions adopt this method, they often suffer from inefficiencies or a lack of accountability. -Classical BLS signatures (as described for example in [Chapter 15.5, construction 15.5.3.2.](http://toc.cryptobook.us/book.pdf)) have fast aggregated signature verification but slow individual signature verification. Since our accountable light client system linked above and, implicitly our bridge design can benefit from BLS signatures with more efficient verification in the individual and aggregated case, [we propose a three part optimisation that dramatically reduces CPU time in large distributed systems using BLS signatures](https://eprint.iacr.org/2022/1611): First, public keys should be given on both source groups, with a proof-of-possession check for correctness. Second, aggregated BLS signatures should carry their particular aggregate public key in the second source group, so that verifiers can do both hash-to-curve and aggregate public key checks in the first source group. Third, individual non-aggregated BLS signatures should carry short [Chaum-Pedersen DLEQ proofs of correctness](https://link.springer.com/content/pdf/10.1007/3-540-48071-4_7.pdf), so that verifying individual signatures no longer requires pairings, which makes their verification much faster. We prove security for these optimisations. The proposed scheme is implemented and benchmarked to compare with classical BLS scheme. +To mitigate the challenge at hand, a committee key scheme is designed to produce short proofs that omit plain public keys of individual participants. This scheme incorporates a custom-designed SNARK that enables fast proving times. Leveraging this cryptographic construction, it is possible to build an accountable light client system that can serve as the core of cross-chain bridges between proof-of-stake blockchains. A prototype of the custom SNARK, along with corresponding benchmarks, is [available](https://github.com/w3f/apk-proofs) for exploration. + +More concretely, the solution can be used to build a BLS-based bridge between Kusama and Polkadot. The light client verifier of any such bridge would be [GRANDPA-based](https://github.com/paritytech/grandpa-bridge-gadget/blob/master/docs/beefy.md). If designed naively, the bridge would require verifying hundreds of signatures for every justification. By aggregating BLS signatures, the verification process is reduced to a single operation over hundreds of public keys. The solution linked above eliminates the need to transmit either hundreds of public keys or individual signatures. + +Classical BLS signatures, as described for example in [Chapter 15.5 (construction 15.5.3.2.)](http://toc.cryptobook.us/book.pdf), offer fast verification for aggregated signatures, yet relatively slow verification for individual ones. + +Since the accountable light client system (linked above), and by extension the bridge design, benefits from BLS signatures offering efficient verification for both individual and aggregated cases, the [proposal](https://eprint.iacr.org/2022/1611) introduces a three-part optimization that dramatically reduces CPU time in large distributed systems using BLS signatures. +1. **Dual-group public keys.** Public keys should be provided on both source groups, accompanied by proof-of-possession checks to ensure correctness. +2. **Aggregate pulic keys in the second group.** Aggregated BLS signatures should include their specific aggregate public key in the second source group. This enables verifiers to perform both hash-to-curve operations and aggregate public key checks in the first source group. +3. **Efficient individual verificaton.** Individual (non-aggregated) BLS signatures should include short [Chaum-Pedersen DLEQ proofs of correctness](https://link.springer.com/content/pdf/10.1007/3-540-48071-4_7.pdf). This eliminates the need for pairings when verifying individual signatures, significantly speeding up their validation. + +The security of these optimizations has been formally proven. Moreover, the proposed scheme has been fully implemented and benchmarked, demonstrating clear improvements over the classical BLS scheme. + +**For inquieries or questions, please contact** [Bhargav Nagaraja Bhatt](/team_members/bhargav.md) diff --git a/docs/Polkadot/protocols/NPoS/1. Overview.md b/docs/Polkadot/protocols/NPoS/1. Overview.md index 85e23ff2..e25b2127 100644 --- a/docs/Polkadot/protocols/NPoS/1. Overview.md +++ b/docs/Polkadot/protocols/NPoS/1. Overview.md @@ -1,52 +1,61 @@ --- -title: Overview of NPoS +title: Nominated Proof-of-Stake (NPoS) Overview --- -**Authors**: [Alfonso Cevallos](/team_members/alfonso.md) + -![](NPoS_Cover.png) - -The Polkadot blockchain will implement nominated proof-of-stake (NPoS), a relatively new type of scheme used to select the validators who are allowed to participate in the consensus protocol. In this note we give an intro to NPoS, and a peek inside the research carried out at the Web3 Foundation. We also explain the peculiar way in which validators get elected. So how does NPoS work in Polkadot? +Nominated Proof-of-Stake (NPoS) is a relatively new scheme that the Polkadot blockchain implements to select validators for participation in its consensus protocol. Apart from explaining how this distinctive mechanism works, this entry also offers a glimpse into the Web3 Foundation's research. ## Validators and nominators -About once per day, the system elects a group of entities called **validators**, who in the next few hours will play a key role in highly sensitive protocols such as [block production](Polkadot/protocols/block-production/index.md) and the [GRANDPA finality gadget](Polkadot/protocols/finality.md). Their job is demanding as they need to run costly operations, ensure high communication responsiveness, and build a long-term reputation of reliability. They also must stake their DOTs, Polkadot’s native token, as a guarantee of good behavior, and this stake gets slashed whenever they deviate from their protocol. In contrast, they get paid well when they play by the rules. Any node that is up to the task can publicly offer itself as a validator candidate. However, for operational reasons only a limited number of validators can be elected, expected to be hundreds or thousands. +**Validators** are entities that play a vital role in blockchain operation and development. Approximately once per day, the system elects a group of validators who, over the next several hours, participate in highly sensitive protocols such as [block production](Polkadot/protocols/block-production/index.md) and the [GRANDPA finality gadget](Polkadot/protocols/finality.md). -The system also encourages any DOT holder to participate as a **nominator**. A nominator publishes a list of validator candidates that she trusts, and puts down an amount of DOTs at stake to support them with. If some of these candidates are elected as validators, she shares with them the payments, or the sanctions, on a per-staked-DOT basis. Unlike validators, an unlimited number of parties can participate as nominators. As long as a nominator is diligent in her choice and only supports validator candidates with good security practices, her role carries low risk and provides a continuous source of revenue. There are other special roles in the Polkadot network, but we focus only on the relation between these two roles. +In this demanding role, validators run resource-intesive operations, maintain high communication responsiveness, and build a long-term reputation for reliability. They must also stake DOT, Polkadot’s native token, as collateral to guarantee good behavior. Validators are well compensated for protocol adherence, but they may be slashed and lose their stake if they act negligently or maliciously. Any node capable of meeting the required standards can publicly offer itself as a validator candidate. For operational efficiency, however, only a limited number of validators can be elected. The current number of validators is in the hundreds, with the potential to scale into the thousands as the number of parachains continues to grow. -![](NPoS_1.png) +Any DOT holder in the system can participate as a **nominator**. Nominators publish a list of validator candidates they trust and stake an amount of DOT to support them. If any of these candidates are elected as validators, both validators and nominators share rewards or penalties on a per-staked-DOT basis. Unlike validators, an unlimited number of participants can act as nominators. As long as they make informed choices, supporting only candidates with strong security practices, their role carries relatively low risk while offering a steady stream of revenue. The Polkadot network includes other specialized roles, but this section focuses solely on the relationship between validators and nominators. +![](NPoS_1.png) ## The NPoS scheme -This nominator-validator arrangement gives strong security guarantees. It allows for the system to select validators with massive amounts of aggregate stake — much higher than any single party’s DOT holdings — and eliminate candidates with low stake. In fact, at any given moment we expect there to be a considerable fraction of all the DOTs supply be staked in NPoS. This makes it very difficult for an adversarial entity to get validators elected (as they need to build a fair amount of reputation to get the required backing) and very costly to attack the system (because any attack will result in large amounts of DOTs being slashed). +The nominator–validator duality offers considerable security guarantees. It enables the selection of validators backed by large aggregate stakes, significantly higher than any single party's DOT holdings, while filtering out low-stake candidates. At any given time, a substantial portion of the total DOT supply is expected to be staked through NPoS. This design makes it extremely difficult for adversarial entities to have validators elected, as building the necessary reputation to gain support is a high barrier. Moreover, any malicious activity would result in large amounts of DOT being slashed, making attacks extremely costly. -Our NPoS scheme is much more efficient than proof-of-work (PoW) and faster than standard proof-of-stake (PoS). Networks with deterministic finality must have a limited validator set (the size can be changed with governance). NPoS allows for virtually all DOT holders to continuously participate, thus maintaining high levels of security by putting more value at stake and allowing more people to earn a yield based on their holdings. +The NPoS scheme is significantly more efficient than Proof-of-Work (PoW) and faster than traditional Proof-of-Stake (PoS) mechanisms. Networks that rely on deterministic finality require a limited validator set, though the size of this set can be adjusted through governance. NPoS enables virtually all DOT holders to participate continuously, thereby enhancing security by increasing the total value at stake and expanding opportunities for individuals to earn a yield based on their holdings. ## The election process ![](NPoS_2.png) -How to elect the validators, given the nominators’ votes? Unlike other PoS-based projects where validators are weighted by stake, Polkadot gives elected validators equal voting power in the consensus protocol. To reflect this fact, the nominators’ stake should be distributed among the elected validators as evenly as possible, while still respecting the nominators’ preferences. At the Web3 Foundation research team, we use tools ranging from election theory to game theory to discrete optimization, to develop an efficient election process that offers fair representation and security, and can be applied in the future to any blockchain using NPoS. We explore these objectives below, together with some examples. +To develop an efficient election process, the Web3 Foundation research team employs a range of tools, from election theory and game theory to discrete optimization. This process aims to deliver fair representation and robust security, and it can be applied to any blockchain utilizing NPoS. + +Unlike other PoS-based projects where validators' voting power is proportional to their stake, Polkadot assigns equal voting power to elected validators in the consensus protocol. To support this design, nominators’ stakes should be distributed as evenly as possible among the elected validators, while still respecting nominators’ preferences. -**Fair representation.** In the late 19th century, Swedish mathematician Lars Edvard Phragmén proposed a method for electing members to his country’s parliament. He noticed that the election methods at the time tended to give all the seats to the most popular political party; in contrast, his new method ensured that the number of seats assigned to each party were proportional to the votes given to them, so it gave more representation to minorities. The property achieved by his method is formally known as proportional justified representation, and is very fitting for the NPoS election because it ensures that any pool of nodes is neither over-represented nor under-represented by the elected validators, proportional to their stake. Our heuristics build on top of Phragmén’s suggested method and ensure this property in every election. +Let’s explore how the election process unfolds through a few illustrative examples. + +**Fair representation.** In the late 19th century, Swedish mathematician Lars Edvard Phragmén proposed a method for electing members of parliament. He observed that prevailing election systems tended to allocate all seats to the most popular political party. His approach aimed to ensure that seats were distributed proportionally to the votes each party received, thereby granting greater representation to minorities. + +The principle underlying his method, known as proportional justified representation, is especially well-suited to NPoS elections. It ensures that no pool of nodes is over- or under-represented among the elected validators relative to its stake. The heuristics, built upon Phragmén’s original concept, ensure that this property holds in every election. ![](NPoS_3.png) -The illustration represents a typical input to the election process, with nominators on the left having different amounts of stake, and connected by lines to those validator candidates on the right that they trust (for simplicity, validators have no stake of their own in this example, though they will in a real scenario). Suppose we need to elect k = 4 validators. The fair representation property roughly translates to the rule that any nominator holding at least one k-th of the total stake is guaranteed to have at least one of their trusted validators elected. As the total stake is 40 DOTS and a fourth of it is 10 DOTS, the first two nominators are guaranteed to be represented by a validator. In the image below we see three possible election results: one that violates the fair representation property and two that achieve it. +The illustration above represents a typical input to the election process. Nominators on the left hold varying amounts of stake and are connected by lines to trusted validator candidates on the right. In this simplified example, validators do not hold any stake of their own, although they typically do in real-world scenarios. +Now, suppose that 𝑘 = 4 validators are elected. According to the fair representation property, any nominator holding at least one-fourth of the total stake is guaranteed to have at least one of their trusted validators elected. In this example, the total stake is 40 DOT, so any nominator with at least 10 DOT is guaranteed representation. Therefore, the first two nominators are assured to have one of their chosen validators elected. The illustration below displays three possible election outcomes: one that violates the fair representation property, and two that satisfy it. ![](NPoS_4.png) -**Security.** If a nominator gets two or more of its trusted validators elected, we need to distribute her stake among them, in such a way that the validators’ backings are as balanced as possible. Recall that we want to make it as difficult as possible for an adversarial pool to get a validator elected, and they can achieve this only if they get a high enough backing. Therefore, we equate the level of security of an election result to *the minimum amount of backing of any elected validator*. For the last two election results with fair representation, we provide stake distributions which show that they achieve security levels of 6 and 9 respectively. +**Security.** If a nominator has two or more of their trusted validators elected, their stake must be distributed among those validators in a way that balances the validators’ backing as evenly as possible. This maximizes resistance to adversarial manipulation, as an adversarial pool can elect a validator only by securing sufficiently high backing. + +For this reason, the security level of an election result is the minimum backing received by any elected validator. For the last two election outcomes with fair representation, the provided stake distributions demonstrate security levels of 6 and 9, respectively. ![](NPoS_5.png) +The election result on the right achieves a higher security level by distributing the nominators’ stake more evenly across validators, resulting in backings of roughly equal size. The objective of the NPoS election process is to produce an outcome that ensures fair representation and maximizes security, an optimization challenge that is [NP-complete](https://www.britannica.com/science/NP-complete-problem). To address this complexity, the Web3 Foundation research team has developed fast, approximate heuristics with strong guarantees on both security and scalability. -The election result on the right achieves a higher security level, and clearly does a better job at splitting the nominators’ stake into validators’ backings of roughly equal size. The goal of the NPoS election process is thus to provide a result that achieves fair representation and a security level that is as high as possible. This gives rise to a rather challenging optimization problem (it is [NP-complete](https://www.britannica.com/science/NP-complete-problem)), for which we have developed fast approximate heuristics with strong guarantees on security and scalability. +The technical advancements introduced by Polkadot, and the opportunities unlocked by NPoS and other highly efficient blockchain mechanisms, are significant, paving the way for innovative new projects. To learn more about Nominated Proof-of-Stake, visit our [Wiki pages](https://wiki.polkadot.network/docs/en/learn-staking), and explore our [research paper](2.%20Paper.md). -We are excited about the technical developments brought forward by Polkadot, and the possibilities enabled by NPoS and other highly efficient schemes being developed in the blockchain space. To learn more about Nominated Proof-of-Stake, visit our [Wiki pages](https://wiki.polkadot.network/docs/en/learn-staking), and read our [research paper](2.%20Paper.md). +**For any inquiries or questions, please contact** diff --git a/docs/Polkadot/protocols/NPoS/2. Paper.md b/docs/Polkadot/protocols/NPoS/2. Paper.md index 4a3108a2..d9000b2b 100644 --- a/docs/Polkadot/protocols/NPoS/2. Paper.md +++ b/docs/Polkadot/protocols/NPoS/2. Paper.md @@ -1,13 +1,18 @@ --- title: A verifiably secure and proportional committee election rule --- + -**Authors**: [Alfonso Cevallos](/team_members/alfonso.md) and [Alistair Stewart](/team_members/alistair.md) +Proportional representation in approval-based committee elections, an idea explored in social choice literature for over a century, is traditionally understood as a mechanism for avoiding the underrepresentation of minorities. In this [paper](https://arxiv.org/abs/2004.12990), the authors discuss how the security of certain distributed systems hinges on the opposite goal: preventing the overrepresentation of any minority. To date, this objective has not been formally defined. -**[arXiv link to reseach paper](https://arxiv.org/abs/2004.12990)** +Such a perspective leads to an optimization objective known as *maximin support*. Through a detailed analysis of its approximability, Web3 Foundation researchers developed a new and efficient election rule inspired by Phragmén's methods. This rule provides a) a constant-factor approximation guarantee for the objective, and b) satisfaction of *proportional justified representation* (PJR). + +A notable feature of the proposed rule is that its output can be *verified* in linear time to meet both the maximin support and PJR criteria, even if the algorithm is executed by an untrusted party who communicates only the output. Finally, the efficient post-computation step, when combined with any approximation algorithm for maximin support, yields a new solution that preserves the approximation guarantee and can be efficiently verified to satisfy PJR. + +This paper is motivated by a blockchain application that implements *Nominated Proof-of-Stake*, in which the community elects a committee of validators to participate in the consensus protocol. In this setting, preventing overrepresentation is critical to protecting the network from attacks by adversarial minorities. -**Abstract.** The property of proportional representation in approval-based committee elections, which has appeared in the social choice literature for over a century, is typically understood as avoiding the underrepresentation of minorities. However, we argue that the security of some distributed systems depends on the opposite goal of preventing the overrepresentation of any minority, a goal not previously formalized which leads us to an optimization objective known as *maximin support*. We provide a thorough analysis of its approximability, and propose a new efficient election rule inspired in Phragmén's methods -that achieves a) a constant-factor approximation guarantee for the objective, and b) the property of *proportional justified representation* (PJR). However, the most striking feature of the new rule is that one can *verify* in linear time that the winning committee satisfies the two aforementioned properties, even when the algorithm is executed by an untrusted party who only communicates the output. Finally, we present an efficient post-computation that, when paired with any approximation algorithm for maximin support, returns a new solution that a) preserves the approximation guarantee and b) can be efficiently verified to satisfy PJR. +The proposed election rule gives rise to a validator election protocol with formal guarantees of both security and proportionality. Crucially, the ability to verify these guarantees for the winning committee is key to adapting the protocol to the trustless and resource-limited environment of blockchains. Implementation details can be found in the Polkadot network, which was launched in 2020. + +**[arXiv link to reseach paper](https://arxiv.org/abs/2004.12990)** -Our work is motivated by an application on blockchains that implement *Nominated Proof-of-Stake*, where the community elects a committee of validators to participate in the consensus protocol, and where preventing overrepresentation protects the network against attacks by an adversarial minority. Our election rule gives rise to a validator election protocol with formal guarantees on security and proportionality, in which the ability to efficiently verify these guarantees on the winning committee proves to be key in adapting the protocol to the trustless and resource-limited nature of blockchains. -We provide details of such an implementation in the Polkadot network, launched in 2020. \ No newline at end of file +**For inquieries or questions, please contact** [Alistair Stewart](/team_members/alistair.md) \ No newline at end of file diff --git a/docs/Polkadot/protocols/NPoS/3. Balancing.md b/docs/Polkadot/protocols/NPoS/3. Balancing.md index 4d85f47f..014e1103 100644 --- a/docs/Polkadot/protocols/NPoS/3. Balancing.md +++ b/docs/Polkadot/protocols/NPoS/3. Balancing.md @@ -1,69 +1,78 @@ --- title: Computing a balanced solution --- + -This is a technical note with algorithmic considerations related to the validator election protocol under NPoS. We consider a scenario where a committee of validators has already been elected, and we explore the best way to assign the nominators' stake to them. The reader should already be familiar with our [research paper](2.%20Paper.md), and in particular the concept of balanced solutions defined in it. Although we prove in that paper that balanced solutions can be computed efficiently, not many details are given about it. Such details are presented in this note. +The so-called *balancing problem* arises within the validator election protocol under NPoS, specifically when a committee of validators has already been elected. To identify the most effective method for assigning nominators’ stake to these validators, two algorithmic approaches can be considered. The first examines parametric flow algorithms; the second explores a heuristic known as star balancing. By the end of this entry, a comparative evaluation may help determine which approach is more effective. -After establishing some notation, we introduce the *balancing problem* and explain why this is exactly the problem we need to solve. We then establish two algorithmic ways to solve the balancing problem, namely 1) using parametric flow algorithms, and 2) using a heuristic called star balancing, and we compare them. +To follow the rationale effectively, the reader must be familiar with our [research paper](2.%20Paper.md), particularly the concept of balanced solutions defined therein. Although that paper proves that balanced solutions can be computed efficiently, it provides limited details on the underlying procedures. These details are elaborated in this entry. + +Let us then begin by establishing some notation. ## 1. Notation -We consider an instance of NPoS consisting of a bipartite graph $(N\cup A, E)$, where $N$ is the set of nominators, $A$ is a committee of elected validators of size $k$, with $k:=|A|\ll |N|$, and there is an edge $nv\in E$ whenever nominator $n$ approves of validator $v\in A$. We are also given a vector $s\in\mathbb{R}^N_{\geq 0}$ of nominator stakes, where $s_n$ is the stake of nominator $n$. An edge weight vector $w\in \mathbb{R}^E_{\geq 0}$ is _feasible_ if it is component-wise non-negative and observes the constraints: $\sum_{v\in A: \ nv\in E} w_{nv} \leq s_n$ for each nominator $n\in N$. We say that $w$ is _tight_ if the previous inequality is tight for each nominator $n$ that has at least one neighbor in $A$. +An instance of NPoS is first modelled as a bipartite graph $(N\cup A, E)$, where $N$ is the set of nominators and $A$ is a committee of elected validators of size $k$, with $k:=|A|\ll |N|$. In this setup, an edge $nv\in E$ exists whenever nominator $n$ endorses validator $v\in A$. + +In the vector $s\in\mathbb{R}^N_{\geq 0}$, the component $s_n$ denotes the stake of nominator $n$. An edge weight vector $w\in \mathbb{R}^E_{\geq 0}$ is called _feasible_ if it is component-wise, non-negative, and satisfies the constraint $\sum_{v\in A: \ nv\in E} w_{nv} \leq s_n$ for each nominator $n\in N$. The vector $w$ is _tight_ if this inequality holds with equality for every nominator $n$ that has at least one neighbor in $A$. -Let $B\in \{0,1\}^{A\times E}$ be the node-edge incidence matrix for the validator set $A$. For any $w\in \mathbb{R}_{\geq 0}^E$, the total support that $w$ assigns to each validator in $A$ is given by the vector $supp_w :=Bw\in \mathbb{R}^A$, so that for any validator $v\in A$, its support -$$supp_w(v)=(Bw)_v = \sum_{n\in N: \ nv\in E} w_{nv}$$ -is the total amount of stake that $w$ assigns to $v$ from the nominators. +Let $B\in \{0,1\}^{A\times E}$ be the node-edge incidence matrix for the validator set $A$. For any weight vector $w\in \mathbb{R}_{\geq 0}^E$, the total support assigned by $w$ to each validator in $A$ is given by the vector $supp_w :=Bw\in \mathbb{R}^A$. For any validator $v\in A$, the support is given by +$$supp_w(v)=(Bw)_v = \sum_{n\in N: \ nv\in E} w_{nv}$$, which represents the total amount of stake that $w$ allocates to $v$ from the nominators. -Given an instance as above, the *balancing problem* consists of finding a tight vector $w$ that minimizes the squared $\ell_2$ norm of the support vector, i.e. minimize the value +Given an instance as described above, the *balancing problem* consists of finding a tight vector $w$ that minimizes the squared $\ell_2$ norm of its support vector, that is, minimizing the following expression: -$$val(w):= \|supp_w\|^2 = \|Bw\|^2.$$ +$$ +val(w):= \|supp_w\|^2 = \|Bw\|^2 +$$ -Clearly, an optimal solution to this problem corresponds precisely to a balanced solution, as defined in our paper. +An optimal solution to this problem corresponds exactly to a balanced solution, as defined in the mentioned paper. ## 2. Algorithms -There are three possible ways to solve the balancing problem: +To address the balancing problem, consider three different approaches: -1. Via convex programming: it can be solved with numerical methods for convex quadratic programs, but this is too computationally expensive to consider any further. -2. Via parametric flow algorithms: We show in the [research paper](2.%20Paper.md) that the balancing problem can potentially be solved in time $O(|E|k + k^3)$ using some advanced techniques for parametric flow problems. -3. Via a simple combinatorial heuristic: the *star balancing* heuristic starts with any tight vector $w$ and converges to an optimal vector $w^*$ by following a local weight-balancing rule. It executes in time $\tilde{O}(|E|k^2)$, ignoring logarithmic factors. +1. **Convex programming**: The problem can be solved using numerical methods for convex quadratic programs. This approach is too computationally expensive to pursue further. +2. **Parametric flow algorithms**: As shown in the [research paper](2.%20Paper.md), the balancing problem can potentially be solved in time $O(|E|k + k^3)$ using advanced techniques for parametric flow problems. +3. **Simple combinatorial heuristic**: The *star balancing* heuristic begins with any tight vector $w$ and converges to an optimal vector $w^*$ by applying a local weight-balancing rule. It runs in time $\tilde{O}(|E|k^2)$, ignoring logarithmic factors. -At first look, the worst-case complexity bound is much better for technique 2 than for technique 3. However, we point out that [Babenko et al. (2007)](https://pdfs.semanticscholar.org/6f44/e6d773cb5093e441f0370b1ec9dd047a5c24.pdf) studied a parametric max flow problem closely related to the balancing problem and performed experimental evaluations of both of these techniques, over real data for an application in revenue optimization as well as over synthetic data. They concluded that the performance of star balancing is actually comparable to that of parametric flow algorithms, except for instances with degenerate graph topologies. In fact, they conjecture that these two techniques have similar complexities whenever the underlying graph has moderately good expansion properties. +Although the worst-case complexity bound seems more favorable for technique 2 than for technique 3, this may not hold in practice. As discussed by [Babenko et al. (2007)](https://pdfs.semanticscholar.org/6f44/e6d773cb5093e441f0370b1ec9dd047a5c24.pdf), who studied a parametric max flow problem closely related to the balancing problem, both techniques were experimentally evaluated using real-world and synthetic data. Their application focused on revenue optimization, and their results showed that the performance of star balancing is actually comparable to that of parametric flow algorithms, except in cases involving degenerate graph topologies. In fact, they conjecture that the two techniques exhibit similar complexities when the underlying graph has reasonably good expansion properties. -In view of this and of the fact that star balancing is vastly easier to implement than the algorithm based in parameter flow, we suggest that star balancing be used for NPoS. +In light of this, and given that star balancing is significantly easier to implement than the parametric flow-based algorithm, star balancing is recommended for NPoS. ## 3. The star balancing heuristic -Star balancing is a combinatorial randomized algorithm that outputs a solution arbitrarily close to optimal with high probability (this is what is known as a polynomial-time randomized approximation scheme, or PRAS). We remark that a different analysis to this algorithm can be found in [Tarjan et al. (2006)](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.7945&rep=rep1&type=pdf). We show the following. +Star balancing is a combinatorial randomized algorithm that yields a solution arbitrarily close to optimal with high probability. This qualifies as a polynomial-time randomized approximation scheme (PRAS). An alternative analysis of this algorithm can be found in [Tarjan et al. (2006)](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.7945&rep=rep1&type=pdf). The following result then holds: -__Theorem:__ For any fixed parameters $\varepsilon, \delta>0$, the star balancing algorithm returns a tight weight vector $w$ whose value $val(w)$ has a probability at least $(1 - \delta)$ of being within a multiplicative factor at most $(1+\varepsilon)$ from minimal, and runs in time +__Theorem:__ For any fixed parameters $\varepsilon, \delta>0$, the star balancing algorithm outputs a tight weight vector $w$ such that, with probability at least $(1 - \delta)$, its value $val(w)$ lies within a multiplicative factor of at most $(1+\varepsilon)$ from the minimum. The algorithm runs in time $$O(|E|k^2 \log (k/\varepsilon \delta)).$$ __Algorithm: Star balancing.__ -Consider an instance $(N\cup A, E, s)$. For each nominator $n\in N$ let $A_n\subseteq A$ be its set of neighbors in $A$. +Consider the instance $(N\cup A, E, s)$, where each nominator $n\in N$ has a set of neighbors $A_n\subseteq A$ in the validator committee. -Fix constants $\varepsilon, \delta>0$. The algorithm starts with an arbitrary tight vector $w$, and improves it iteratively by performing $r$ rounds, where we will give a precise value for $r$ and prove that $r = O(|N|k^2\log(k/\varepsilon \delta))$. +Fix constants $\varepsilon, \delta>0$. The algorithm starts with an arbitrary tight vector $w$, and iteratively refines it over $r$ rounds. Later, the value of $r$ will be specified, and it will be shown that $r = O(|N|k^2\log(k/\varepsilon \delta))$. -1. Find any tight vector $w$. +**Algorithm Steps:** -2. Repeat $r$ times: - a. Select a nominator $n\in N$ uniformly at random. - b. Modify the weights of the edges incident to $n$, keeping $w$ tight and observing the non-negativity constraints, so that the supports of the neighboring validators are as close to each other as possible, i.e. so that +1. **Initialization**: Find any tight vector $w$. - $$\forall v,v'\in A_n, \ supp_w(v)>supp_w(v') \rightarrow w_{nv}=0.$$ +2. **Iterative refinement** (repeat $r$ times): First, select a nominator $n\in N$ uniformly at random. Then, update the weights of the edges incident to $n$, ensuring tightness and non-negativity to balance the support among its neighboring validators. Specifically, ensure that: -3. Return $w$. + $$ + \forall v,v'\in A_n, \ supp_w(v)>supp_w(v') \rightarrow w_{nv}=0 + $$ -__Running time:__ Consider a round of the algorithm. If nominator $n$ is selected, the running time of the round is $O(|A_n|)$, assuming that floating-point arithmetic operations take constant time. Hence, the average running time per round is proportional to $\frac{1}{|N|}\sum_{n\in N} |A_n|=\frac{|E|}{|N|}$. Together with the bound on $r$, we obtain a global running time of +3. **Output**: Return the final tight vector $w$. + +__Running time:__ After a single round of the algorithm, if nominator $n$ is selected, the running time of that round is $O(|A_n|)$, assuming that floating-point arithmetic operations take constant time. The expected running time per round is thus proportional to $\frac{1}{|N|}\sum_{n\in N} |A_n|=\frac{|E|}{|N|}$. Combining this with the bound on the number of rounds $r$, the total running time of the algorithm is $$O(r|E|/|N|) = O(|E|k^2\log(k/\varepsilon \delta)).$$ -__Analysis:__ For each $i\leq r$, let $w^i$ be the state of weight vector $w$ at the end of the $i$-th round, and let $w^0$ be the initial vector. Let $w^*$ be an optimal solution. Let's start with an easy observation. +__Analysis:__ Let $w^0$ be the initial weight vector and $w^*$ the optimal solution. For each round $i\leq r$, denote by $w^i$ the state of $w$ at the end of the $i$-th round. Begin with a single observation. + __Lemma 1:__ $val(w^0)\leq k\cdot val(w^*)$. -_Proof:_ Recall that the objective value to minimize is $val(w)=\|Bw\|^2_2=\|supp_w\|_2^2$. As both $w^0$ and $w^*$ are tight, the $\ell_1$ norm of their support vectors are equal. Hence +_Proof:_ The objective value to minimize is $val(w)=\|Bw\|^2_2=\|supp_w\|_2^2$. Since both $w^0$ and $w^*$ are tight, their support vectors have equal $\ell_1$ norms. Therefore, $$val(w^0)=\|Bw^0\|_2^2 \leq \|Bw^0\|_1^2 = \|Bw^*\|_1^2 \leq k\cdot \|Bw^*\|_2^2 = k\cdot val(w^*).$$ @@ -74,20 +83,20 @@ $$

-Next we show that, in expectation, the progress in objective value perceived in each round is proportional to the difference between the current and optimal values. +The next step is to show that, in expectation, the progress in objective value achieved during each round is proportional to the difference between the current value and the optimal one. -__Lemma 2:__ For each round $i\in\{1,\cdots,r\}$ that starts with vector $w^{i-1}$ and ends with vector $w^i$, the expected objective value of $w^i$ is such that +__Lemma 2:__ For each round $i\in\{1,\cdots,r\}$, with initial vector $w^{i-1}$ and final vector $w^i$, the expected value of the objective function satisfies $$val(w^{i-1}) - \mathbb{E}[val(w^{i})] \geq \frac{1}{k^2|N|} [val(w^{i-1}) - val(w^*)].$$ -_Proof:_ We fix a round $i$, and for notational convenience we drop the superscripts $i$ and $i-1$ within the scope of this proof. In particular, we let $w$ be the initial vector, and let $w'^n$ be the final vector in the case that nominator $n$ is picked in the round. Clearly, the expected progress in objective value equals the average progress $\frac{1}{|N|}\sum_{n\in N} [val(w) - val(w'^n)]$. To lower bound the latter, it is sufficient to exhibit a different family of weight vectors $\{w^n\}_{n\in N}$ such that $val(w'^n)\leq val(w^n)$ for each $n$, and then bound the average progress when moving from $w$ to a member of that family. +_Proof:_ Fix a round $i$, and for notational convenience, drop the superscripts $i$ and $i-1$ throughout this proof. Let $w$ denote the initial weight vector, and let $w'^n$ be the resulting vector when nominator $n$ is selected in that round. The expected improvement in the objective value is given by the average progress $\frac{1}{|N|}\sum_{n\in N} [val(w) - val(w'^n)]$. To establish a lower bound on this quantity, it suffices to define an alternative family of vectors $\{w^n\}_{n\in N}$ such that $val(w'^n)\leq val(w^n)$ for each $n$. It is then possible to analyze the average improvement obtained by transitioning from $w$ to a corresponding vector in this family. -Define the vector $f:=w-w^*\in\mathbb{R}^E$. The following is a necessary technical observation whose proof we delay temporarily. +To proceed, define the vector $f:=w-w^*\in\mathbb{R}^E$. The following technical observation is essential, though its proof will be deferred. __Lemma 3:__ $\|f\|^2 \leq k^2 \|Bf\|^2.$ -Consider the decomposition of vector $f$ as $f=\sum_{n\in N} f^n$, where $f^n$ is the restriction of $f$ over the edges incident to nominator $n$, and define the family of weight vectors $\{w^n:= w-\frac{1}{k^2} f^n\}_{n\in N}$. We have $val(w'^n) \leq val(w^n)$ for all $n\in N$ as desired, because by construction (step 2.b. of the algorithm), $w'^n$ is precisely the vector of minimum objective value among all maximally affordable vectors that differ from $w$ only at the edges incident to $n$. Hence, it only remains to bound the average progress in objective value with respect to the new family. +Consider the decomposition of vector $f$ as $f=\sum_{n\in N} f^n$, where each $f^n$ denotes the restriction of $f$ to the edges incident to nominator $n$. Define the family of weight vectors $\{w^n:= w-\frac{1}{k^2} f^n\}_{n\in N}$. Then $val(w'^n) \leq val(w^n)$ holds for all $n\in N$, as desired. This follows from the construction in step 2.b of the algorithm: $w'^n$ is the tight, maximally affordable vector minimizing the objective among all vectors differing from $w$ only on edges incident to $n$. All that remains is to bound the average progress in objective value with respect to the newly defined family. -For a fixed $n\in N$, we have +For a fixed $n\in N$, $$\begin{align} val(w) - val(w^n) &= \|Bw\|^2 - \|B(w-\frac{1}{k^2} f^n)\|^2 \\ @@ -114,14 +123,14 @@ $$

-_Proof of Lemma 3:_ We interpret $f$ as a flow over the network $(N\cup A, E)$. As both $w$ and $w^*$ are tight, there is flow preservation over all nominators. Let $A_s, A_t\subseteq A$ be respectively the sets of sources and sinks, i.e. the sets of validators with net excess and net demand. By the flow decomposition theorem, there exists a decomposition $f=\sum_{v\in A_s} f^v$ into single-source subflows, where $f^v$ has $v$ as its single source. We can assume that this decomposition generates no cycles by adjusting the choice of the optimal solution $w^*=w-f$. +_Proof of Lemma 3:_ The vector $f$ can be interpreted as a flow over the network $(N\cup A, E)$. Since both $w$ and $w^*$ are tight, flow preservation holds at all nominators. Let $A_s, A_t\subseteq A$ denote the sets of sources and sinks, respectively. That is, the subsets of validators with net excess and net demand. By the flow decomposition theorem, the flow *f* can be expressed as the sum of single-source subflows $f=\sum_{v\in A_s} f^v$, where each $f^v$ originates from a single source validator. This decomposition generates no cycles by suitably adjusting the choice of the optimal solution $w^*=w-f$. -Consider one of these subflows $f^v$. Its edge support looks like a directed acyclic graph (DAG) with single root $v$. We arrange the edges on this DAG by levels, where the level of an edge is the length of the longest path from $v$ containing this edge. These levels start at 1 for the edges incident to $v$, up to at most $2k$ because any simple path alternates between a nominator and a validator and there are only $k$ validators. We now split $f^v$ by levels, $f^v=\sum_{i\leq 2k} f^{v,i}$, where $f^{v,i}$ is the restriction of $f^v$ over the edges at level $i$. Since the excess in node $v$ is $supp_w(v)-supp_{w^*}(v)=(Bf)_v$ and no other node in the DAG has any excess, the sum of edge weights along each level $i$ is $\|f^{v,i}\|_1 \leq (Bf)_v$. Therefore, +The edge support of each subflow $f^v$ resembles a directed acyclic graph (DAG) rooted at the single source node $v$. Edges on this DAG are organized by levels, where the level of an edge is defined by the length of the longest path from $v$ that includes this edge. These levels begin at 1 for edges incident to $v$, and go up to at most $2k$, since any simple path alternates between nominators and validators, with only $k$ validators. The next step is to decompose $f^v$ by levels as $f^v=\sum_{i\leq 2k} f^{v,i}$, where each $f^{v,i}$ is the restriction of $f^v$ over edges at level $i$. Because node $v$ is the only source of excess, quantified by $supp_w(v)-supp_{w^*}(v)=(Bf)_v$, and all other nodes in the DAG preserve flow, the total weight along any level $i$ is $\|f^{v,i}\|_1 \leq (Bf)_v$. It follows that: $$\|f^v\|_2^2 = \sum_{i\leq 2k}\|f^{v,i}\|_2^2 \leq \sum_{i\leq 2k} \|f^{v,i}\|_1^2 \leq 2k\cdot (Bf)^2_v.$$ -Putting things together, we get +Putting things together, the result is: \begin{align} \|f\|^2_2 &= \|\sum_{v\in A_s} f^v\|_2^2 \\ @@ -132,42 +141,43 @@ Putting things together, we get where the first inequality is an application of a Cauchy-Schwarz inequality. -In a similar manner, working with sinks instead of sources, we can obtain the bound $\|f\|^2 \leq 2k|A_t| \cdot \|Bf\|^2$. Summing up these two bounds and dividing by two, we get +Similarly, by considering sinks instead of sources, the resulting bound is $\|f\|^2 \leq 2k|A_t| \cdot \|Bf\|^2$. Summing this with the previous bound and dividing by two, yields $$\|f\|^2 \leq k(|A_s|+|A_t|) \cdot \|Bf\|^2 \leq k^2 \|Bf\|^2,$$ -which proves the claim. +which establishes the claim. $$ \tag{$\blacksquare$} $$

-For each round $i\leq r$, consider the random variable $\Delta^i:= val(w^i) - val(w^*)$, which represents how far from optimal the current solution is in terms of objective value. We now use Lemma 2 to show that $\Delta^i$ decays exponentially fast in expectation. +For each round $i\leq r$, let the random variable $\Delta^i:= val(w^i) - val(w^*)$ denote the deviation of the current solution from optimality in terms of objective value. Referring back to Lemma 2, one can demonstrate that $\Delta^i$ decreases exponentially in expectation. -__Lemma 4:__ For any $0
-Recall now that we want the value of the output solution $val(w^r)$ to be within a factor of $(1+\varepsilon)$ from $val(w^*)$ with probability at least $(1-\delta)$. The next lemma completes the analysis of the algorithm and the proof of the main theorem. +Since the value of the final output $val(w^r)$ must lie within a factor of $(1+\varepsilon)$ of the optimal value $val(w^*)$ with probability at least $(1-\delta)$, the next lemma completes the algorithm's analysis and establishes the main theorem. __Lemma 5:__ If $r=\lceil |N|k^2\ln(k/\epsilon \delta) \rceil$, then $\mathbb{P}[val(w^r) > (1+\varepsilon)val(w^*)]\leq \delta$. -_Proof:_ By Lemma 4 and the choice of value $r$, it follows that +_Proof:_ By Lemma 4 and the choice of $r$, it follows that $$\mathbb{E}[\Delta^r]\leq \epsilon\cdot \delta\cdot val(w^*).$$ -As the variable $\Delta^r$ is non-negative, we can use Markov's inequality: +Since $\Delta^r$ is non-negative, we apply Markov's inequality: -$$\delta \geq \mathbb{P}[\Delta^r > \frac{\mathbb{E}[\Delta^r]}{\delta}] -\geq \mathbb{P}[\Delta^r > \epsilon\cdot val(w^*)] +$$\delta \geq \mathbb{P}[\Delta^r > \frac{\mathbb{E}[\Delta^r]}{\delta}]\geq \mathbb{P}[\Delta^r > \epsilon\cdot val(w^*)] = \mathbb{P}[val(w^r) > (1+\epsilon)\cdot val(w^*)],$$ -which is the claim. +and thus the claim is proved. $$ \tag{$\blacksquare$} $$

+ +**For inquieries or questions, please contact** diff --git a/docs/Polkadot/protocols/NPoS/Committee-election-rule.png b/docs/Polkadot/protocols/NPoS/Committee-election-rule.png new file mode 100644 index 00000000..1e5f5a1f Binary files /dev/null and b/docs/Polkadot/protocols/NPoS/Committee-election-rule.png differ diff --git a/docs/Polkadot/protocols/NPoS/Computed-balanced-solution.jpeg b/docs/Polkadot/protocols/NPoS/Computed-balanced-solution.jpeg new file mode 100644 index 00000000..d69cfd6e Binary files /dev/null and b/docs/Polkadot/protocols/NPoS/Computed-balanced-solution.jpeg differ diff --git a/docs/Polkadot/protocols/NPoS/NPoS_Cover1.png b/docs/Polkadot/protocols/NPoS/NPoS_Cover1.png new file mode 100644 index 00000000..2f9a56f5 Binary files /dev/null and b/docs/Polkadot/protocols/NPoS/NPoS_Cover1.png differ diff --git a/docs/Polkadot/protocols/NPoS/Nominated-proof-of-stake.png b/docs/Polkadot/protocols/NPoS/Nominated-proof-of-stake.png new file mode 100644 index 00000000..bab21ee5 Binary files /dev/null and b/docs/Polkadot/protocols/NPoS/Nominated-proof-of-stake.png differ diff --git a/docs/Polkadot/protocols/NPoS/index.md b/docs/Polkadot/protocols/NPoS/index.md index a4965262..ee7e6657 100644 --- a/docs/Polkadot/protocols/NPoS/index.md +++ b/docs/Polkadot/protocols/NPoS/index.md @@ -2,18 +2,20 @@ title: Nominated Proof-of-Stake --- -**Authors**: [Alfonso Cevallos](/team_members/alfonso.md) +![](Nominated-proof-of-stake.png) -Many blockchain projects launched in recent years substitute the highly inefficient Proof-of-Work (PoW) component of Nakamoto’s consensus protocol with Proof-of-Stake (PoS), in which validators participate in block production with a frequency proportional to their token holdings, as opposed to their computational power. While a pure PoS system allows any token holder to participate directly, most projects propose some level of centralized operation, whereby the number of validators with full participation rights is limited. Arguments for this limited validator set design choice are that: +In recent years, many blockchain projects have replaced the highly inefficient Proof-of-Work (PoW) component of Nakamoto’s consensus protocol with Proof-of-Stake (PoS). In PoS systems, validators participate in block production at a frequency proportional to their token holdings, rather than their computational power. Although a pure PoS model allows any token holder to participate directly, most projects adopt some degree of centralization by limiting the number of validators with full participation rights. The rationale for this limited validator set design is based on the following considerations: -- The increase in operational costs and communication complexity eventually outmatches the increase in decentralization benefits as the number of validators grows. -- While many token holders may want to contribute in maintaining the system, the number of candidates with the required knowledge and equipment to ensure a high quality of service is limited. -- It is typically observed in networks (both PoW- and PoS-based) with a large number of validators that the latter tend to form pools anyway, in order to decrease the variance of their revenue and profit from economies of scale. +- As the number of validators increases, operational costs and communication complexity eventually outweigh the benefits of decentralization. +- While many token holders may wish to contribute to system maintenance, the number of candidates with the necessary knowledge and equipment to ensure high-quality service remains limited. +- In both PoW- and PoS-based networks, when the number of validators becomes large, participants in the latter tend to form pools to reduce revenue variance and benefit from economies of scale. -Therefore, rather than let pools be formed off-chain, it is more convenient for the system to formalize and facilitate pool formation on-chain, and allow users to vote with their stake to elect validators that represent them and act on their behalf. Networks following this approach include Polkadot, Cardano, EOS, Tezos, and Cosmos, among many others. While similar in spirit, the approaches in these networks vary in terms of design choices such as the incentive structure, the number of validators elected, and the election rule used to select them. +Rather than allowing pools to form off-chain, it is more effective to formalize and facilitate their formation on-chain, enabling users to vote with their stake to elect validators who represent them and act on their behalf. Networks that follow this model include Polkadot, Cardano, EOS, Tezos, and Cosmos, among others. While united in principle, these networks differ in design choices such as incentive structures, validator set sizes, and election mechanisms. -Polkadot introduces a variant of PoS called Nominated Proof-of-Stake, with design choices based on first principles and having security, fair representation and satisfaction of users, and efficiency as driving goals. In NPoS, users are free to become validator candidates, or become nominators. Nominators approve of candidates that they trust and back them with their tokens, and once per era a committee of validators is elected according to the current nominators' preferences. In Polkadot, the number k of validators elected is in the order of hundreds, and may be thousands in the future as the number of parachains increases. +Polkadot introduces a Nominated Proof-of-Stake (NPoS) system. Its design choices are rooted in first principles, with security, fair representation, user satisfaction, and efficiency as guiding goals. In NPoS, users may either become validator candidates or act as nominators. Nominators select trusted candidates and support them by backing their stake. Once per era, a validator committee is elected based on the preferences of the current set of nominators. The number of elected validators k, is currently in the hundreds and may scale into the thousands as the number of parachains grows. -Both validators and nominators lock their tokens as collateral and receive staking rewards on a pro-rata basis, but may also be slashed and lose their collateral in case a backed validator shows negligent or adversarial behavior. Nominators thus participate indirectly in the consensus protocol with an economic incentive to pay close attention to the evolving set of candidates and make sure that only the most capable and trustworthy among them get elected. +Validators and nominators lock their tokens as collateral and receive staking rewards on a pro-rata basis. They may be slashed and lose their collateral if a supported validator engages in negligent or adversarial behavior. Nominators participate indirectly in the consensus protocol and have an economic incentive to closely monitor the evolving candidate set, helping ensure that only the most capable and trustworthy validators are elected. -Visit our [overview page](1.%20Overview.md) for a first introduction to NPoS, and our [research paper](2.%20Paper.md) for an in-depth analysis. We also encourage the reader to visit the [token economics research section](Polkadot/overview/2-token-economics.md) for further information about staking rewards, [the section on slashing](Polkadot/security/slashing/amounts.md), and our [Wiki pages](https://wiki.polkadot.network/docs/en/learn-staking) for more hands-on information about the staking process. We also remark that, unlike other projects, Polkadot keeps validator selection completely independent from [governance](https://wiki.polkadot.network/docs/en/learn-governance), and in particular the user's right to participate in governance is never delegated. \ No newline at end of file +In the [overview page](1.%20Overview.md) you can learn about NPoS, and for an in-depth analysis you can read the [research paper](2.%20Paper.md). For more details on staking rewards, check out the [token economics research section](Polkadot/overview/2-token-economics.md), and learn about slashing in [this section](Polkadot/security/slashing/amounts.md). For a broader understanding of the staking process, explore [Wiki pages](https://wiki.polkadot.network/docs/en/learn-staking). + +Unlike other projects, Polkadot maintains complete independence between validator selection and [governance](https://wiki.polkadot.network/docs/en/learn-governance). In particular, users' rights to participate in governance are never delegated. diff --git a/docs/Polkadot/protocols/Polkadot-protocols.png b/docs/Polkadot/protocols/Polkadot-protocols.png new file mode 100644 index 00000000..f8e44b49 Binary files /dev/null and b/docs/Polkadot/protocols/Polkadot-protocols.png differ diff --git a/docs/Polkadot/protocols/Sassafras/SLEP.jpeg b/docs/Polkadot/protocols/Sassafras/SLEP.jpeg new file mode 100644 index 00000000..58ef399b Binary files /dev/null and b/docs/Polkadot/protocols/Sassafras/SLEP.jpeg differ diff --git a/docs/Polkadot/protocols/Sassafras/SLEP.png b/docs/Polkadot/protocols/Sassafras/SLEP.png new file mode 100644 index 00000000..9dd23cb5 Binary files /dev/null and b/docs/Polkadot/protocols/Sassafras/SLEP.png differ diff --git a/docs/Polkadot/protocols/Sassafras/Sassafras-part-3.md b/docs/Polkadot/protocols/Sassafras/Sassafras-part-3.md index e6cbb696..36e53ab3 100644 --- a/docs/Polkadot/protocols/Sassafras/Sassafras-part-3.md +++ b/docs/Polkadot/protocols/Sassafras/Sassafras-part-3.md @@ -1,16 +1,16 @@ # Sassafras Part 3: Compare and Convince -Authors: Elizabeth Crites, Handan Kılınç Alper, Alistair Stewart, and Fatemeh Shirazi This is the third in a series of three blog posts that describe the new consensus protocol Sassafras, which is planned to be integrated into Polkadot, replacing the current [BABE](https://wiki.polkadot.network/docs/learn-consensus#block-production-babe)+[Aura](https://openethereum.github.io/Aura.html) consensus mechanism. -Here is an overview of the three blog posts: +:::note Overview of the three blog posts: **[Part 1 - A Novel Single Secret Leader Election Protocol](sassafras-part-1):** The aim of this blog post is to give an introduction that is understandable to any reader with a slight knowledge of blockchains. It explains why Sassafras is useful and gives a high-level overview of how it works. **[Part 2 - Deep Dive](sassafras-part-2):** The aim of this blog post is to dive into the details of the Sassafras protocol, focusing on technical aspects and security. **Part 3 - Compare and Convince:** The aim of this blog post is to offer a comparison to similar protocols and convince the reader of Sassafras's value. +::: ## If you have not read Part 1 and Part 2 Here is a a summary: @@ -93,7 +93,6 @@ In particular, we conducted a comprehensive comparison of various protocols base While the setup phase is expected to be rare in PoS blockchain protocols, shuffle-based solutions (with the exception of WHISK) impose impractical levels of message overhead. For election messages on the blockchain, Shuffle-2 and Shuffle-3 are highly inefficient. In stark contrast, Sassafras introduces a mere 7.64 MB overhead on the blockchain. - | Protocol || Setup | Election | | -------- |--------| -------- | -------- | |Shuffle-1|
Off-Chain
On-Chain
|
-
$8790.15$ MB
|
-
$123.7$ MB
| @@ -123,7 +122,6 @@ In terms of both communication and computational overhead, Sassafras outperforms Table 3: Computational overhead of SSLE protocols on a blockchain. $N$ is the total number of participants. - ## Key Takeaways This concludes the three-part blog post series on Sassafras. Here are some key takeaways: @@ -131,6 +129,10 @@ This concludes the three-part blog post series on Sassafras. Here are some key * **Single leader election:** Sassafras elects a single block producer for each slot, ensuring faster consensus compared to protocols that rely on probabilistic leader election, which may not guarantee a unique leader or a leader at all times. * **Maintaining the secrecy of a block producer:** Sassafras ensures the secrecy of block producers to mitigate against denial-of-service (DoS) attacks. * **Lightweight:** Sassafras features exceptionally low communication and computational complexity and scales better than existing solutions. + + +**For inquieries or questions, please contact:** [Elizabeth Crites](team_members/elizabeth.md) + diff --git a/docs/Polkadot/protocols/Sassafras/index.md b/docs/Polkadot/protocols/Sassafras/index.md index 514cbfe5..a269c1ed 100644 --- a/docs/Polkadot/protocols/Sassafras/index.md +++ b/docs/Polkadot/protocols/Sassafras/index.md @@ -4,6 +4,8 @@ title: Understanding Sassafras import DocCardList from '@theme/DocCardList'; -A Blog Series: +This blog series covers the most important aspects of the secret leader election protocol. + +![](SLEP.png) \ No newline at end of file diff --git a/docs/Polkadot/protocols/Sassafras/sassafras-part-1.md b/docs/Polkadot/protocols/Sassafras/sassafras-part-1.md index c05bfacf..437023bd 100644 --- a/docs/Polkadot/protocols/Sassafras/sassafras-part-1.md +++ b/docs/Polkadot/protocols/Sassafras/sassafras-part-1.md @@ -1,6 +1,5 @@ # Sassafras Part 1: A Novel Single Secret Leader Election Protocol -Authors: Armando Caracheo, Elizabeth Crites, and Fatemeh Shirazi Polkadot is set to replace the [BABE](https://wiki.polkadot.network/docs/learn-consensus#block-production-babe)+[Aura](https://openethereum.github.io/Aura.html) consensus protocol with a new one: *Sassafras*. While Sassafras will be used to generate blocks on Polkadot's relay chain, it is also suitable for use in other proof-of-stake (PoS) blockchains. So, what key advantages does this new protocol bring to the blockchain ecosystem? @@ -67,5 +66,6 @@ Our next blog post [Sassafras Part 2 - Deep Dive](sassafras-part-2), will explai So stay tuned, brave reader. There's much more to discover in our upcoming Sassafras series, which is packed with valuable insights! +**For inquieries or questions, please contact:** [Elizabeth Crites](/team_members/elizabeth.md) [def]: Sassafras-diagram.png diff --git a/docs/Polkadot/protocols/Sassafras/sassafras-part-2.md b/docs/Polkadot/protocols/Sassafras/sassafras-part-2.md index b6796e8d..90475c3a 100644 --- a/docs/Polkadot/protocols/Sassafras/sassafras-part-2.md +++ b/docs/Polkadot/protocols/Sassafras/sassafras-part-2.md @@ -1,19 +1,17 @@ # Sassafras Part 2: Deep Dive -Authors: Elizabeth Crites and Fatemeh Shirazi - This is the second in a series of three blog posts that describe the new consensus protocol Sassafras, which is planned to be integrated into Polkadot, replacing the current [BABE](https://wiki.polkadot.network/docs/learn-consensus#block-production-babe)+[Aura](https://openethereum.github.io/Aura.html) consensus mechanism. -Here is an overview of the three blog posts: + +:::note Overview of the three blog posts: **[Part 1 - A Novel Single Secret Leader Election Protocol](sassafras-part-1):** The aim of this blog post is to give an introduction that is understandable to any reader with a slight knowledge of blockchains. It explains why Sassafras is useful and gives a high-level overview of how it works. **Part 2 - Deep Dive:** The aim of this blog post is to dive into the details of the Sassafras protocol, focusing on technical aspects and security. **[Part 3 - Compare and Convince](Sassafras-part-3):** -The aim of this blog post is to offer a comparison to similar protocols and convince the reader of Sassafras's value. - -Let's now take a deep dive into the Sassafras protocol, starting with some background on leader election protocols. +::: +The aim of this blog post is to offer a comparison to similar protocols and convince the reader of Sassafras's value. Let's now take a deep dive into the Sassafras protocol, starting with some background on leader election protocols. ## Sassafras: Efficient Batch Single Leader Election @@ -115,6 +113,8 @@ which makes the tickets sent to different receivers indistinguishable, We now move to [Part 3](Sassafras-part-3), which gives a detailed efficiency analysis and comparison with other approaches to leader election. +**For inquieries and questions, please contact:** [Elizabeth Crites](/team_members/elizabeth.md) + [^1]: We show how to chose the parameter $n_t$ in [the paper](https://eprint.iacr.org/2023/002). For example, $n_t$ should be at least 6 for $2^{13}$ elections, under the assumption that the fraction $\alpha$ of corrupt parties is less than $\approx 0.3$ with $2^{14} = 16384$ total parties. (This is the number of validators running the leader election protocol proposed for Ethereum; see [Part 3](Sassafras-part-3).) [^2]: Formally, the communication between sender and receiver occurs via a secure diffusion functionality $\mathcal{F}_{\mathsf{com}}^s$, which hides the message and the receiver. Here, we describe Sassafras with a simple and efficient instantiation of $\mathcal{F}_{\mathsf{com}}^s$ using symmetric encryption. By "multicasting the ciphertext to all parties," we mean one-to-many communication via a standard diffusion functionality $\mathcal{F}_{\mathsf{com}}$. Details are given in [the paper](https://eprint.iacr.org/2023/002). diff --git a/docs/Polkadot/protocols/block-production/BABE.png b/docs/Polkadot/protocols/block-production/BABE.png new file mode 100644 index 00000000..77dc7189 Binary files /dev/null and b/docs/Polkadot/protocols/block-production/BABE.png differ diff --git a/docs/Polkadot/protocols/block-production/Babe.md b/docs/Polkadot/protocols/block-production/Babe.md index fef3b2b4..09da6cf0 100644 --- a/docs/Polkadot/protocols/block-production/Babe.md +++ b/docs/Polkadot/protocols/block-production/Babe.md @@ -2,152 +2,129 @@ title: BABE --- -**Authors**: [Handan Kilinc Alper](/team_members/handan.md) + -## 1. Overview +Polkadot produces relay chain blocks using the **B**lind **A**ssignment for **B**lockchain **E**xtension protocol (BABE), which assigns block production slots based on a randomness cycle similar to that used in Ouroboros Praos.[^2] The process unfolds as follows: All block producers possess a verifiable random function (VRF) key, which is registered alongside their locked stake. These VRFs generate secret randomness, determining when each producer is eligible to create a block. The process carries an inherent risk that producers may attempt to manipulate the outcome by grinding through multiple VRF keys. To mitigate this, the VRF inputs must incorporate public randomness created only after the VRF key is established. -In Polkadot, we produce relay chain blocks using our - **B**lind **A**ssignment for **B**lockchain **E**xtension protocol, - abbreviated BABE. -BABE assigns block production slots - using roughly the randomness cycle from Ouroboros Praos [2]. +As a result, the system operates in epochs, during which fresh public on-chain randomness is created by hashing together all the VRF outputs revealed through block production within that epoch This establishes a cycle that alternates between private, verifiable randomness and collaborative public randomness. -In brief, all block producers have a verifiable random function (VRF) -key, which they register with the locked stake. These VRFs produce secret -randomness, which determines when they produce blocks. A priori, there -is a risk that block producers could grind through VRF keys to bias -results, so VRF inputs must include public randomness created only -after the VRF key. We therefore have epochs in which we create fresh -public on-chain randomness by hashing together all the VRF outputs -revealed in block creation during the epoch. In this way, we cycle -between private but verifiable randomness and collaborative public -randomness. - - -The main differences of BABE from Ouroboros Praos [2] are the best chain selection mechanism and slot synchronization assumption i.e.: - -1. BABE's best chain selection is based on GRANDPA and longest chain. -2. Block producers in BABE do not have access to a central authority (e.g., Network Time Protocol (NTP)) to count slots instead, they construct their own clock to follow the slots. +BABE differs from Ouroboros Praos [^2] in two main aspects: 1) its best chain selection mechanism, which integrates GRANDPA with the longest-chain rule, and 2) its slot synchronization assumptions. In the latter case, BABE block producers do not depend on a central authority, such as Network Time Protocol (NTP), to count slots. Instead, they build and maintain local clocks to track slot progression. --- + +## 1. Epochs, slots, and keys -## 2. BABE - -In BABE, we have sequential non-overlapping epochs $(e_1, e_2,\ldots)$, each of which consists of a number of sequential block production slots ($e_i = \{sl^i_{1}, sl^i_{2},\ldots,sl^i_{t}\}$) up to some bound $t$. At the beginning of an epoch, we randomly assign each block production slot to a "slot leader", often one party or no party, but sometimes more than one party. These assignments are initially secrets known only to the assigned slot leader themselves, but eventually they publicly claim their slots when they produce a new block in one. +BABE consists of sequential, non-overlapping epochs $(e_1, e_2,\ldots)$, each with a set of consecutive block production slots ($e_i = \{sl^i_{1}, sl^i_{2},\ldots,sl^i_{t}\}$) up to a bound $t$. At the start of each epoch, block production slots are randomly assigned to "slot leaders," sometimes to one party, no party, or multiple parties. The assignments are initially private, known only to the designated slot leader. This changes once they publicly claim their slots by producing a new block. -Each party $P_j$ has as *session key* containing at least two types of secret/public key pair: +Each party $P_j$ possesses a *session key* that includes at least two types of secret/public key pairs: -* a verifiable random function (VRF) key $(\skvrf_{j}, \pkvrf_{j})$, and -* a signing key for blocks $(\sksgn_j,\pksgn_j)$. +* A verifiable random function (VRF) key pair $(\skvrf_{j}, \pkvrf_{j})$ +* A signing key pair for blocks $(\sksgn_j,\pksgn_j)$ -We favor VRF keys being relatively long-lived because new VRF keys cannot be used until well after creation and submission to the chain. Yet, parties should update their associated signing keys from time to time to provide forward security against attackers who might exploit from creating slashable equivocations. There are more details about the session key available [here](Polkadot/security/keys/3-session.md). +VRF keys are preferred because they are relatively long-lived; new VRF keys cannot be used until well after they've been created and submitted to the chain. Yet, parties should periodically update their associated signing keys to maintain forward security, protecting against attackers who might exploit outdated keys to create slashable equivocations. For more details on session keys see [here](Polkadot/security/keys/3-session.md). -Each party $P_j$ keeps a local set of blockchains $\mathbb{C}_j =\{C_1, C_2,..., C_l\}$. All these chains have some common blocks, at least the genesis block, up until some height. +Each party $P_j$ maintains a local set of blockchains $\mathbb{C}_j =\{C_1, C_2,..., C_l\}$. These chains share a common prefix of blocks, at minimum the genesis block, up to a certain height. -We assume that each party has a local buffer that contains a set of transactions to be added to blocks. All transactions in a block are validated with a transaction validation function before entering this buffer. +Each party also maintains a local buffer containing a set of transactions to be added to blocks. Before entering this buffer, all transactions are validated using a transaction validation function. - -In BABE, we would like to achieve that each validator has the same chance to be selected as a block producer on a slot. Therefore, we define the probability that a validator is selected on a slot as +The aim is to ensure that each validator has an equal opportunity to be selected as a block producer for any given slot. The probability of selection for each validator is: $$ p = \phi_c(\theta) = 1-(1-c)^{\frac{1}{n}} $$ -where $0 \leq c \leq 1$ is a constant parameter and $n$ is the number of validators. +where $0 \leq c \leq 1$ is a constant parameter and $n$ denotes the number of validators. -In order to achieve the equality of validators in BABE, we define a threshold parameter as in [2] for the slot assignment: +To ensure equitable slot assignment among validators in BABE, it is necessary to define a threshold parameter. To guide the slot selection process, the approach described in [^2] is useful. The result then is: $$ -\tau = 2^{\ell_{vrf}}\phi_c(\theta) +\tau = 2^{\ell_{vrf}}\phi_c(\theta), $$ where $\ell_{vrf}$ is the length of the VRF's first output (randomness value). -BABE consists of three phases: +## 2. Phases -#### 1st: Genesis Phase +BABE consists of three phases: -In this phase, we manually produce the unique genesis block. +#### 1st: Genesis phase -The genesis block contain a random number $r_1$ for use during the first two epochs for slot leader assignments. Session public keys of initial validators are ($\pkvrf_{1}, \pkvrf_{2},..., \pkvrf_{n}$), $(\pksgn_{1}, \pksgn_{2},..., \pksgn_{n}$). +The unique genesis block, manually produced in this phase, contains a random number $r_1$ used during the first two epochs for slot leader assignments. Session public keys of initial validators are ($\pkvrf_{1}, \pkvrf_{2},..., \pkvrf_{n}$), $(\pksgn_{1}, \pksgn_{2},..., \pksgn_{n}$). -#### 2nd: Normal Phase +#### 2nd: Normal phase -We assume that each validator divided their timeline in slots after receiving the genesis block. They determine the current slot number according to their timeline as explained in [Section 4](./Babe.md#-4.-clock-adjustment--relative-time-algorithm-). Similarly, when a new validator joins to BABE after the genesis block, this validator divides his timeline into slots. +By the time the second phase begins, each validator must have divided their timeline into slots after receiving the genesis block. Validators determine the current slot number according to their local timeline, as explained in [Section 4](./Babe.md#-4.-clock-adjustment--relative-time-algorithm-). If validators join BABE after the genesis block, they should also divide their timelines into slots. -In normal operation, each slot leader should produce and publish a block. All other nodes attempt to update their chain by extending with new valid blocks they observe. +During normal operation, the designated slot leader should produce and publish a block. All other nodes update their chains based on the new valid blocks they observe. -We suppose each validator $V_j$ has a set of chains $\mathbb{C}_j$ in the current slot $sl_k$ in the epoch $e_m$ and has a best chain $C$ selected in $sl_{k-1}$ by our selection scheme in Section 3, and the length of $C$ is $\ell\text{-}1$. +Each validator $V_j$ maintains a set of chains $\mathbb{C}_j$ for the current slot $sl_k$ in epoch $e_m$, and a best chain $C$ selected during slot $sl_{k-1}$ according to the selection scheme described in Section 3. The length of $C$ is $\ell\text{-}1$. -Each validator $V_j$ produces a block if he is the slot leader of $sl_k$. If the first output ($d$) of the following VRF computation is less than the threshold $\tau$ then he is the slot leader. +A validator $V_j$ may produce a block if selected as the slot leader for $sl_k$. If the first output $d$ of the following VRF computation is less than the threshold $\tau$, the validator becomes the slot leader. $$ \vrf_{\skvrf_{j}}(r_m||sl_{k}) \rightarrow (d, \pi) $$ -If $P_j$ is the slot leader, $P_j$ generates a block to be added on $C$ in slot $sl_k$. The block $B_\ell$ should at least contain the slot number $sl_{k}$, the hash of the previous block $H_{\ell\text{-}1}$, the VRF output $d, \pi$, transactions $tx$, and the signature $\sigma = \sgn_{\sksgn_j}(sl_{k}||H_{\ell\text{-}1}||d||\pi||tx))$. $P_i$ updates $C$ with the new block and sends $B_\ell$. - -In any case (being a slot leader or not being a slot leader), when $V_j$ receives a block $B = (sl, H, d', \pi', tx', \sigma')$ produced by a validator $V_t$, it validates the block with $\mathsf{Validate}(B)$. $\mathsf{Validate}(B)$ must at least check the followings in order to validate the block: +If $P_j$ is the slot leader, it generates a block to be added to chain $C$ during slot $sl_k$. The block $B_\ell$ must contain at minimum: the slot number $sl_{k}$, the hash of the previous block $H_{\ell\text{-}1}$, the VRF output $d, \pi$, the transactions $tx$, and the signature $\sigma = \sgn_{\sksgn_j}(sl_{k}||H_{\ell\text{-}1}||d||\pi||tx))$. Validator $P_i$ then updates $C$ with the new block and relays $B_\ell$. -* if $\mathsf{Verify}_{\pksgn_t}(\sigma')\rightarrow \mathsf{valid}$ (signature verification), +Regardless of whether $V_j$ is a slot leader, upon receiving a block $B = (sl, H, d', \pi', tx', \sigma')$ produced by validator $V_t$, it excecutes $\mathsf{Validate}(B)$. To validate the block, the function $\mathsf{Validate}(B)$ must, at minimum, check the following criteria: -* if the validator is the slot leader: $\mathsf{Verify}_{\pkvrf_t}(\pi', r_m||sl) \rightarrow \mathsf{valid}$ and $d' < \tau$ (verification with the VRF's verification algorithm). +* $\mathsf{Verify}_{\pksgn_t}(\sigma')\rightarrow \mathsf{valid}$ – signature verification -* if there exists a chain $C'$ with the header $H$, +* If the validator is the slot leader: $\mathsf{Verify}_{\pkvrf_t}(\pi', r_m||sl) \rightarrow \mathsf{valid}$ and $d' < \tau$ – verification using the VRF's algorithm -* if the transactions in $B$ are valid. +* There exists a chain $C'$ with header $H$, -If the validation process goes well, $V_j$ adds $B$ to $C'$. Otherwise, it ignores the block. +* The transactions in $B$ are valid. +If all checks pass, $V_j$ adds $B$ to $C'$; otherwise, it discards the block. At the end of the slot, $P_j$ selects the best chain according to the chain selection rule outlined in Section 3. -At the end of the slot, $P_j$ decides the best chain with the chain selection rule we give in Section 3. +#### 3rd: Epoch update -#### 3rd: Epoch Update +Before starting a new epoch $e_m$, validators must obtain the new epoch randomness and the updated active validator set. A new epoch begins every $R$ slots, starting from the first slot. -Starting from first slot, in every $R$ slots, the new epoch starts. -Before starting a new epoch $e_m$, validators should obtain the new epoch randomness and active validators set for the new epoch. +To ensure participation in epoch $e_m$, the validator set must be included in the relay chain by the end of the last block of epoch $e_{m-3}$. This timing enables validators to actively engage in block production for epoch $e_{m}$. Newly added validators may join block production no earlier that two epochs later after being included in the relay chain. -The validator set for the epoch $e_m$ has to be included to the relay chain until the end of the last block of the epoch $e_{m-3}$ so that they are able to actively participate the block production in epoch $e_{m}$. So, a new validator can actively join the block production at earliest two epochs later after included to relay chain. - -A fresh randomness for the epoch $e_m$ is computed as in Ouroboros Praos [2]: Concatenate all the VRF outputs of blocks in epoch $e_{m-2}$ (let us assume the concatenation is $\rho$). Then the randomness in epoch $e_{m}$: +Fresh randomness for epoch $e_m$ is computed using the Ouroboros Praos [^2] method: Specifically, all VRF outputs from blocks produced in epoch $e_{m-2}$ (denoted as $\rho$) are concatenated. The randomness for epoch $e_{m}$ is then derived as follows: $$ r_{m} = H(r_{m-2}||m||\rho) $$ -The reason of including a validator after two epochs later is to make sure that the VRF keys of the new validators added to the chain before the randomness of the epoch that they are going to be active is revealed. +Including a validator two epochs later ensures that the VRF keys of newly added validators, submitted to the chain prior to the randomness generation of their active epoch, are properly revealed. --- -## 3. Best Chain Selection +## 3. Best chain selection -Given a chain set $\mathbb{C}_j$ and the parties current local chain $C_{loc}$, the best chain algorithm eliminates all chains which do not include the finalized block $B$ by GRANDPA. Let's denote the remaining chains by the set $\mathbb{C}'_j$. If we do not have a finalized block by GRANDPA, then we use the probabilistic finality in the best chain selection algorithm (the probabilistically finalized block is the block which is $k$ block before than the last block of $C_{loc}$). +Given a chain set $\mathbb{C}_j$, and the party's current local chain $C_{loc}$, the best chain selection algorithm eliminates all chains that do not contain the finalized block $B$ determined by GRANDPA. The remaining chains form a subset denoted by $\mathbb{C}'_j$. If GRANDPA finalty is not required for a block, the algorithm resorts to probabilistic finality. In this case, the probabillistically finalized block is defined as the block that is $k$ blocks prior to the latest block in $C_{loc}$. -We do not use the chain selection rule as in Ouroboros Genesis [3] because this rule is useful for parties who become online after a period of time and do not have any information related to current valid chain (for parties always online the Genesis rule and Praos is indistinguishable with a negligible probability). Thanks to Grandpa finality, the new comers have a reference point to build their chain so we do not need the Genesis rule. +In this case, the chain selection rule does not follow Ouroboros Genesis [^3], as that rule is intended for parties that come online after a period of inactivity and lack information about the current valid chain. For parties that remain continously online, the Genesis rule and Praos are indistinguishable with negligible probability. Thanks to Grandpa finality, newcomers have a reliable reference point to build their chain, making the Genesis rule unnecessary. --- -## 4. Clock Adjustment (Relative Time Algorithm) - -It is important for parties to know the current slot for the security and completeness of BABE. For this, validators can use their computer clocks which is adjusted by the Network Time Protocol. However, in this case, we need to trust servers of NTP. If an attack happens to one of these servers than we cannot claim anymore that BABE is secure. Therefore, we show how a validator realizes the notion of slots without using NTP. Here, we assume we have a partial synchronous network meaning that any message sent by a validator arrives at most $\D$-slots later. $\D$ is an unknown parameter. +## 4. Clock adjustment (Relative Time Algorithm) - -Each party has a local clock and this clock is not updated by any external source such as NTP or GPS. When a validator receives the genesis block, it stores the arrival time as $t_0$ as a reference point of the beginning of the first slot. We are aware that the beginning of the first slot is not the same for everyone. We assume that the maximum difference of start time of the first slot between validators is at most $\delta$. Then each party divides their timeline in slots and update periodically its local clock with the following algorithm. + For the security and completeness of BABE, parties must be aware of the current slot. Typically, validators rely on system clocks sinchronized via by the Network Time Protocol (NTP). This introduces a trust assumption, and if an NTP server is compromised, BABE's security can no longer be upheld. To mitigate such a risk, validators can determine slot timing without relying on NTP. + + Let's assume a partially synchronous network scenario, where any message sent by a validator is delivered within at most $\D$-slots, an unknown parameter. Since each party relies on a local clock not sinchronized by any external source such as NTP or GPS, a validator should store the arrival time of the genesis block as $t_0$, which serves as a reference point marking the start of the first slot. This starting point varies accross validators. Assuming the maximum deviation in the first slot's start time between validators is at most $\delta$, each party should divide its timeline into slots and periodically synchronize its local clock using the following algorithm. **Median Algorithm:** -The median algorithm is run by all validators in the end of sync-epochs (we note that epoch and sync-epoch are not related). The first sync-epoch ($\varepsilon = 1$) starts just after the genesis block is released. The other sync-epochs ($\varepsilon > 1$) start when the slot number of the last (probabilistically) finalized block is $\bar{sl}_{\varepsilon}$ which is the smallest slot number such that $\bar{sl}_{\varepsilon} - \bar{sl}_{\varepsilon-1} \geq s_{cq}$ where $\bar{sl}_{\varepsilon-1}$ is the slot number of the last (probabilistically) finalized block in the sync-epoch $\varepsilon-1$. Here, $s_{cq}$ is the parameter of the chain quality (CQ) property. If the previous epoch is the first epoch then $sl_{e-1} = 0$. We define the last (probabilistically) finalized block as follows: Retrieve the best blockchain according to the best chain selection rule, prune the last $k$ blocks of the best chain, then the last (probabilistically) finalized block will be the last block of the pruned best chain. Here, $k$ is defined according to the common prefix property. +The median algorithm is executed by all validators at the end of sync-epochs [^4]. The first sync-epoch ($\varepsilon = 1$) begins once the genesis block is released. Subsequent sync-epochs ($\varepsilon > 1$) begin when the slot number of the last (probabilistically) finalized block is $\bar{sl}_{\varepsilon}$, defined as the smallest slot number such that $\bar{sl}_{\varepsilon} - \bar{sl}_{\varepsilon-1} \geq s_{cq}$. Here, $\bar{sl}_{\varepsilon-1}$ is the slot number of the last finalized block from sync-epoch $\varepsilon-1$, and $s_{cq}$ is the chain quality (CQ) parameter. If the previous epoch is the first epoch then $sl_{e-1} = 0$. + +To identify the last (probabilistically) finalized block: Retrieve the best blockchain according to the chain selection rule, prune the final $k$ blocks from this chain, and define the last finalized block as the last block of the pruned best chain, where $k$ is set according to the common prefix property. -The details of the protocol is the following: Each validator stores the arrival time $t_i$ of valid blocks constantly according to its local clock. In the end of a sync-epoch, each validator retrieves the arrival times of valid and finalized blocks which has a slot number $sl'_x$ where +The protocol details are as follows: Each validator records the arrival time $t_i$ of valid blocks using its local clock. At the end of a sync-epoch, each validator retrieves the arrival times of valid and finalized blocks with slot number $sl'_x$ where * $\bar{sl}_{\varepsilon-1} < sl_x \leq \bar{sl}_{\varepsilon}$ if $\varepsilon > 1$. * $\bar{sl}_{\varepsilon-1} \leq sl_x \leq \bar{sl}_{\varepsilon}$ if $\varepsilon = 1$. -Let's assume that there are $n$ such blocks that belong to the current sync-epoch and let us denote the stored arrival times of blocks in the current sync-epoch by $t_1,t_2,...,t_n$ whose slot numbers are $sl'_1,sl'_2,...,sl'_n$, respectively. A validator selects a slot number $sl > sl_e$ and runs the median algorithm which works as follows: +Assuming that no such $n$ blocks belong to the current sync-epoch, and denoting the stored arrival times of blocks in this sync-epoch as $t_1,t_2,...,t_n$, with corresponding slot numbers $sl'_1,sl'_2,...,sl'_n$, validators should select a slot number $sl > sl_e$ and execute the median algorithm as follows: ``` @@ -158,72 +135,72 @@ lst = sort (lst) return median(lst) ``` -In the end, the validator adjusts its clock by mapping $sl$ to the output of the median algorithm. +Ultimately, each validator adjusts its local clock by mapping slot $sl$ to the output of the median algorithm. -The following image with chains explains the algorithm with an example in the first epoch where $s_{cq} = 9$ and $k=1$: +The image below illustrates the algorithm using a chain-based example in the first epoch, where $s_{cq} = 9$ and $k=1$: ![](https://i.imgur.com/jpiuQaM.png) -**Lemma 1:** (The difference between outputs of median algorithms of validators) Asuming that $\delta\_max$ is the maximum network delay, the maximum difference between start time is at most $\delta\_max$. +**Lemma 1** or the difference between outputs of validators' median algorithms. Asuming $\delta\_max$ is the maximum network delay, the maximum difference between start time is at most $\delta\_max$. -**Proof Sketch:** Since all validators run the median algorithm with the arrival time of the same blocks, the difference between the output of the median algorithm of each validator differs at most $\delta\_max$. +**Proof Sketch.** Since all validators run the median algorithm using the arrival times of the same blocks, the difference between the output of each validator's median algorithm is bounded by at most $\delta\_max$. -**Lemma 2:** (Adjustment Value) Assuming that the maximum total drift on clocks between sync-epochs is at most $\Sigma$ and $2\delta\_max + |\Sigma| \leq \theta$, the maximum difference between the new start time of a slot $sl$ and the old start time of $sl$ is at most $\theta$. +**Lemma 2** or Adjustment Value. Assuming the maximum total drift between sync-epochs is at most $\Sigma$ and that $2\delta\_max + |\Sigma| \leq \theta$, the maximum difference between the new start time of a slot $sl$ and the old start time of $sl$ is at most $\theta$. -This lemma says that the block production may stop at most $\theta$ at the beginning of the new synch-epoch. +In simple terms, this lemma states that the block production may be delayed by at most $\theta$ at the beginning of the new sync epoch. -**Proof Sketch:** With the chain quality property, we can guarantee that more than half of arrival times of the blocks used in the median algorithm sent on time. Therefore, the output of all validators' median algorithm is the one which is sent on time. The details of the proof is in Theorem 1 our paper [Consensus on Clocks](https://eprint.iacr.org/2019/1348). +**Proof Sketch.** The chain quality property ensures that more than half of arrival times for blocks used in the median algorithm are timely. As a result, the output of each validator's median algorithm corresponds to a block delivered on time. A formal proof is provided in Theorem 1 of our paper [Consensus on Clocks](https://eprint.iacr.org/2019/1348). -Having $\theta$ small enough is important not to slow down the block production mechanism a while after a sync-epoch. For example, (a very extreme example) we do not want to end up with a new clock that says that we are in the year 2001 even if we are in 2019. In this case, honest validators may wait 18 years to execute an action that is supposed to be done in 2019. +Keeping $\theta$ small is crucial to prevent delays in block production after a sync-epoch. For example (albeit an extreme one), it is not desirable that a validator's adjusted clock indicates the year 2001 when it's actually 2019. In such a case, honest validators might have to wait 18 years before executing an action that was originally scheduled for 2019. -### Temporarily Clock Adjustment +### Temporarily clock adjustment -For validators who were offline at some point during one synch-epoch, they can adjust their clock temporarily (till the next synch epoch) with the following algorithm. +The following algorithm permits validators who were offline during part of a sync epoch to temporarily adjust their local clocks, valid until the next synch-epoch. -**1. Case:** If $V$ was online at some point of a synch-epoch and when he becomes online if his clock works well, he should continue to collect the arrival time of valid blocks and produce his block according to his clock as usual. A block is considered as valid in this case if it is not equivocated, if the block is sent by the right validator and if its slot number belong to the current synch epoch. In the end of the synch-epoch, if he has collected $n$ arrival time of valid blocks he runs the median algorithm with these blocks. -If it has less than $n$ blocks it should wait till collecting $n$ arrival time of valid blocks. We note that he does not run the median algorithm not only with the arrival time of the finalized blocks. +**Case 1:** If validator $V$ was offline at any point of a synch-epoch, and upon returning online its clock is functioning correclty, it should resume collecting the arrival times of valid blocks and produce blocks according to its local clock as usual. A block is considered valid in this case if it is not equivocated, is sent by the right validator, and its slot number falls within the current sync epoch. -**2. Case:** If $V$ was online at some point of a synch-epoch and when he becomes online if his clock does not work anymore, he should continue to collect the arrival time of valid blocks. He can adjust his clock according to e.g., the arrival time of the last finalized block in GRANDPA to continue to produce block. He can use this clock till collecting $n$ valid blocks. After collecting $n$ valid blocks he should readjust his clock according to the output of the median algorithm with these $n$ valid blocks. +At the end of the sync epoch, if $V$ has collected $n$ valid block arrival times, it should run the median algorithm using these blocks. In case it has fewer than $n$ blocks, it must wait until the required $n$ arrival times have been gathered. The validator does not run the median algorithm solely with the arrival times of finalized blocks. -With the temporary clock adjustment, we can guarantee that the difference between this new clock and an honest parties clock is at most $2\delta_{max} + |\Sigma|$. +**Case 2:** If $V$ was offline at any point during a sync epoch and, upon reconnecting, its clock is no longer functioning properly, it should continue collecting the arrival times of valid blocks. The validator may temporarily adjust its clock using, for example, the arrival time of the last finalized block in GRANDPA, and resume block production accordingly. This temporary clock can be used until $n$ valid blocks have been collected. Once this condition is met, the validator should re-adjust its clock based on the output of the median algorithm applied to these blocks. -**We note that during one sync-epoch the ratio of such offline validators should not be more than 0.05 otherwise it can affect the security of the relative time algorithm.** +With the temporary clock adjustment, it is possible to ensure that the difference between the time recorded by the adjusted clock and that of an honest party's clock is bounded by at most $2\delta_{max} + |\Sigma|$. + +**Note: During one sync epoch the ratio of such offline validators should not be more than 0.05, otherwise it can affect the security of the relative time algorithm.** --- -## 5. Security Analysis +## 5. Security analysis -(If you are interested in parameter selection and practical results based on the security analysis, you can directly go to the next section) -BABE is the same as Ouroboros Praos except for the chain selection rule and clock adjustment. Therefore, the security analysis is similar to Ouroboros Praos with few changes. +BABE functions similarly to Ouroboros Praos, with the exception of the chain selection rule and clock synchronization mechanism. From a security analysis perspective, BABE closely resembles Ouroboros Praos, albeit with a few notable differences. If you are more interested in parameter selection and practical outcomes resulting from the security analysis, feel free to skip this section and proceed directly to the next. ### Definitions -We give the definitions of security properties before jumping to proofs. +Before diving into the proofs, let’s establish some key definitions. -**Definition 1 (Chain Growth (CG)) [1,2]:** Chain growth with parameters $\tau \in (0,1]$ and $s \in \mathbb{N}$ ensures that if the best chain owned by an honest party at the onset of some slot $sl_u$ is $C_u$, and the best chain owned by an honest party at the onset of slot $sl_v \geq sl_u+s$ is $C_v$, then the difference between the length of $C_v$ and $C_u$ is greater or equal than/to $\tau s$. +**Definition 1 or Chain Growth (CG) [1,2].** Chain growth with parameters $\tau \in (0,1]$ and $s \in \mathbb{N}$ guarantees that if the best chain held by an honest party at the beginning of slot $sl_u$ is $C_u$, and the best chain at the beginning of slot $sl_v \geq sl_u+s$ is $C_v$, then the length of $C_v$ is at least $\tau s$ greater than the length of $C_u$. -The honest chain growth (HCG) property is a weaker version of CG which is the same definition with the restriction that $sl_v$ and $sl_u$ are assigned to honest validators. The parameters of HCG are $\tau_{hcg}$ and $s_{hcg}$ instead of $\tau$ and $s$ in the CG definition. +The honest chain growth (HCG) property is a relaxed version of the Chain Growth (CG) property, defined identically except for the added constraint that both $sl_v$ and $sl_u$ are assigned to honest validators. The parameters for HCG are $\tau_{hcg}$ and $s_{hcg}$, in place of $\tau$ and $s$ used in the CG definition. -**Definition 2 (Existential Chain Quality (ECQ)) [1,2]:** Consider a chain $C$ possessed by an honest party at the onset of a slot $sl$. Let $sl_1$ and $sl_2$ be two previous slots for which $sl_1 + s_{ecq} \leq sl_2 \leq sl$. Then $C[sl_1 : sl_2]$ contains at least one block generated by an honest party. +**Definition 2 or Existential Chain Quality (ECQ) [1,2].** Consider a chain $C$ held by an honest party at the beginning of slot $sl$. Let $sl_1$ and $sl_2$ be two earlier slots such that $sl_1 + s_{ecq} \leq sl_2 \leq sl$. Then, the segment $C[sl_1 : sl_2]$ contains at least one block produced by an honest party. -**Definition 2 (Chain Density (CD)):** The CD property with parameters $s_{cd} \in \mathbb{N}$ ensures that any portion $B[s_u:s_v]$ of a final blockchain $B$ spanning between rounds $s_u$ and $s_v = s_u + s_{cd}$ contains more honest blocks. +**Definition 2 or Chain Density (CD).** The CD property, with parameter $s_{cd} \in \mathbb{N}$, ensures that any segment $B[s_u:s_v]$ of the final blockchain $B$, spanning rounds $s_u$ to $s_v = s_u + s_{cd}$, contains a majority of blocks produced by honest parties. -**Definition 3 (Common Prefix)** Common prefix with parameters $k \in \mathbb{N}$ ensures that any chains $C_1, C_2$ possessed by two honest parties at the onset of the slots $sl_1 < sl_2$ are such satisfies $C_1^{\ulcorner k} \leq C_2$ where $C_1^{\ulcorner k}$ denotes the chain obtained by removing the last $k$ blocks from $C_1$, and $\leq$ denotes the prefix relation. +**Definition 3 or Common Prefix.** The Common Prefix property, with parameter $k \in \mathbb{N}$, ensures that for any chains $C_1, C_2$ held by two honest parties at the beginning of slots $sl_1$ and $sl_2$ respectively, where $sl_1 < sl_2$, it holds that $C_1^{\ulcorner k} \leq C_2$. Here, $C_1^{\ulcorner k}$ denotes the chain obtained by removing the last $k$ blocks from $C_1$, and $\leq$ represents the prefix relation. -With using these properties, we show that BABE has persistence and liveness properties. **Persistence** ensures that, if a transaction is seen in a block deep enough in the chain, it will stay there and **liveness** ensures that if a transaction is given as input to all honest players, it will eventually be inserted in a block, deep enough in the chain, of an honest player. +The use of these properties demonstrates BABE's persistence and liveness properties. **Persistence** ensures that if a transaction appears in a block sufficiently deep in the chain, it will remain there permanently. **Liveness** guaranteess that if a transaction is provided as input to all honest parties, it will eventually be included in a block, deep enough in the chain, by an honest party. -### Security Proof of BABE -We analyze BABE with the NTP protocol and with the Median algorithm. +### Security proof of BABE +The next step is to analyze BABE with the NTP protocol and with the Median algorithm. -We first prove that BABE (both versions) satisfies chain growth, existential chain quality and common prefix properties in one epoch. We also show the chain density property for the BABE with median. Then, we prove that BABE is secure by showing that BABE satisfies persistence and liveness in multiple epochs. +First, prove that both versions of BABE satisfy the chain growth, existential chain quality, and common prefix properties within a single epoch, as well as to establish the chain density property specifically for BABE with median. Then, demonstrate BABE's overall security by showing that it satisfies persistence and liveness across multiple epochs. -In Polkadot, all validators have equal stake (the same chance to be selected as slot leader), so the relative stake is $\alpha_i = 1/n$ for each validator where $n$ is the total number of validators. We assume that the ratio of honest validators is $\alpha$ and the ratio of validators sending on time is $\alpha_{timely}$. +In Polkadot, all validators have an equal chance of being selected as slot leaders due to equal stake allocation. As a result, each validator's relative stake is given by $\alpha_i = 1/n$, where $n$ is the total number of validators. Assume that the proportion of honest validators is $\alpha$, and the proportion of validators sending on time is denoted by $\alpha_{timely}$. -We use notation $p_h$ (resp. $p_m$) to show the probability of an honest validator (resp. a malicious validator) is selected. Similarly, we use $p_H$ (resp. $p_M$) to show the probability of *only* honest validators (resp. malicious validators) are selected. $p_{\bot}$ is the probability of having an empty slot (no validator selected). +Using the notation $p_h$ (resp. $p_m$), express the probability of selecting an honest (respectively malicious) validator. Similarly, $p_H$ (respectively $p_M$) denotes the probability of selecting *only* honest (respectively malicious) validators. $p_{\bot}$ represents the probability of an empty slot, where no validator is selected. $$ p_\bot=\mathsf{Pr}[sl = \bot] = \prod_{i\in \mathcal{P}}1-\phi(\alpha_i) = \prod_{i \in \mathcal{P}} (1-c)^{\alpha_i} = 1-c @@ -245,24 +222,30 @@ $$ p_m = c - p_H $$ -The probability of having timely validator is +The probability of selecting a timely validator is $$ p_H\_\mathsf{timely} = \prod_{i \in \mathcal{P_m}} 1- \phi(1/n) \sum_{i \in \mathcal{P}_h} \binom{\alpha_{timely} n}{i}\phi(1/n)^i (1- \phi(1/n))^{\alpha_{timely} n - i} $$ -and probability of having non-timely validator is $p_m\_\mathsf{timely} = c - p_H\_\mathsf{timely}$. +Meanwhile, the probability of selecting a non-timely validator is given by $p_m\_\mathsf{timely} = c - p_H\_\mathsf{timely}$. + + +Validators in BABE who use NTP are perfectly synchronized (i.e., there is no difference between the time shown on their clocks), whereas validators using the median algorithm may experience a clock discrepancy of up to $\delta\_max + |2\Sigma|$. +An honest validator in BABE with the NTP can build upon an honest block generated in slot $sl$ if the block reaches all validators before the next **non-empty** slot $sl_{\mathsf{next}}$. Such slots are referred to as good slots. In other words, a slot $sl$ is considered good if it is assigned exclusively to an honest validator and the following $\D = \lfloor \frac{\delta\_max}{T}\rfloor$ slots are empty. +For validators in BABE that rely on the median algorithm, the process diverges due to clock offsets among validators. If a slot is assigned to an honest validator whose clock runs earliest, then in order to build on top of all blocks from prior honest slots, that validator must see those blocks before generating their own. This requirement is met only if the preceding $\lfloor \frac{\delta\_max + |2 \Sigma|}{T}\rfloor$ slots are empty. -The validators in BABE with NTP are perfectly synchronized (i.e., the difference between their clocks is 0). On the other hand, the validators in BABE with the median algorithm have their clocks differ at most $\delta\_max + |2\Sigma|$. -In BABE with the NTP, any honest validator builds on top of an honest block generated in slot $sl$ for sure if the block arrives all validators before starting the next **non-empty** slot $sl_{\mathsf{next}}$. We call these slots good slots. In BABE with NTP, a slot $sl$ is good if it is assigned to only honest validators and the next $\D = \lfloor \frac{\delta\_max}{T}\rfloor$ slots are empty. However, it is different in BABE with the median algorithm because of the clock difference between validators. If a slot is assigned to an honest validator that has the earliest clock, in order to make her to build on top of blocks of all previous honest slots for sure, we should make sure that this validator sees all blocks of the previous slots before generating her block. We can guarantee this if previous $\lfloor \frac{\delta\_max + |2 \Sigma|}{T}\rfloor$ slots are empty. Also, if a slot is assigned to an honest validator that has the latest clock, we should make sure that the next honest block producers see the block of the latest validator before generating her block. We can guarantee this if the next $\lfloor \frac{2\delta\_max + |2 \Sigma|}{T}\rfloor$ slots are empty. We use $\D_m = \lfloor \frac{2\delta\_max + |2 \Sigma|}{T}\rfloor + \lfloor \frac{\delta\_max + |2 \Sigma|}{T}\rfloor$ in our analysis below. +Conversly, if a slot is assigned to an honest validator whose clock runs latest, it is crucial that subsequent honest block producers see this validator's block before producing their own. This can be ensured if the next $\lfloor \frac{2\delta\_max + |2 \Sigma|}{T}\rfloor$ slots are empty. +To accomodate both scenarios in the analysis, a parameter such as $\D_m = \lfloor \frac{2\delta\_max + |2 \Sigma|}{T}\rfloor + \lfloor \frac{\delta\_max + |2 \Sigma|}{T}\rfloor$ is required. -**Theorem 1:** BABE with NTP satisfies HCG property with parameters $\tau_{hcg} = p_hp_\bot^\D(1-\omega)$ where $0 < \omega < 1$ and $s_{hcg} > 0$ in $s_{hcg}$ slots with probability $1-\exp(-\frac{ p_h s_{hcg} \omega^2}{2})$. -**Proof:** We need to count the honest and good slots (i.e., the slot assigned to at least one honest validator and the next $\D$ slots are empty) (Def. Appendix E.5. in [Genesis](https://eprint.iacr.org/2018/378.pdf)) to show the HCG property. The best chain grows one block in honest slots. If honest slots out of $s_{hcg}$ slot are less than $s_{hcg}\tau_{hcg}$, the HCG property is violated. The probability of having an honest and good slot is $p_hp_\bot^\D$. +**Theorem 1.** BABE with NTP satisfies the HCG property with parameters $\tau_{hcg} = p_hp_\bot^\D(1-\omega)$, where $0 < \omega < 1$ and $s_{hcg} > 0$. The property holds over $s_{hcg}$ slots with probability $1-\exp(-\frac{ p_h s_{hcg} \omega^2}{2})$. -We find below the probability of less than $\tau_{hcg} s_{hcg}$ slots are honest slots. From Chernoff bound we know that +**Proof.** To demonstrate the HCG property, it is necesssary to count the *honest* and *good* slots (slots assigned to at least one honest validator, followed by $\D$ empty slots) (see Definition E.5. in [Genesis](https://eprint.iacr.org/2018/378.pdf)). The best chain grows one block during each honest slot. If the number of honest slots within $s_{hcg}$ total slots is less than $s_{hcg}\tau_{hcg}$, the HCG property no longer holds. The probability of encountering an honest and good slot is given by $p_hp_\bot^\D$. + +The probability that fewer than $\tau_{hcg} s_{hcg}$ slots are honest is given below, using the Chernoff bound: $$ \Pr[\sum honest \leq (1-\omega) p_h p_\bot s_{hcg}] \leq \exp(-\frac{p_hp_\bot^\D s_{hcg} \omega^2}{2}) @@ -274,14 +257,14 @@ $$

-BABE with median satisfies HCG property with parameters $\tau_{hcg} = p_hp_\bot^{D_m}(1-\omega)$ where $0 < \omega < 1$ and $s_{hcg} > 0$ in $s_{hcg}$ slots with probability $1-\exp(-\frac{ p_hp_\bot^{\D_m} s_{hcg} \omega^2}{2})$. +BABE with median algorithm satisfies the HCG property with parameters $\tau_{hcg} = p_hp_\bot^{D_m}(1-\omega)$, where $0 < \omega < 1$ and $s_{hcg} > 0$. The probability holds over $s_{hcg}$ slots with probability $1-\exp(-\frac{ p_hp_\bot^{\D_m} s_{hcg} \omega^2}{2})$. -**Theorem 2 (Chain Densisty)** Chain desisty property is satisfied with $s_{cd}$ in BABE with probability $1 - \exp(-\frac{p_H\_\mathsf{timely}p_\bot^{\D_m} s_{cd} \omega_H^2}{2}) - \exp(-\frac{\gamma^2s_{cd}p_m\_\mathsf{timely}}{2+\gamma}) - \exp(-\ell)$ where $\omega_H \in (0,1)$ and $\gamma > 0$. +**Theorem 2 or Chain Density.** The Chain Density (CD) property is satisfied over $s_{cd}$ slots in BABE with probability $1 - \exp(-\frac{p_H\_\mathsf{timely}p_\bot^{\D_m} s_{cd} \omega_H^2}{2}) - \exp(-\frac{\gamma^2s_{cd}p_m\_\mathsf{timely}}{2+\gamma}) - \exp(-\ell)$, where $\omega_H \in (0,1)$ and $\gamma > 0$. -**Proof:** We first find the minimum difference between the number of honest slots and the number of malicious slots in $s_{cd}$ slots belonging one synch-epoch. For this, we need to find the minimum number of honest slots $H$ and a maximum number of honest slots $m$. +**Proof.** Determine the minimum difference between the number of honest slots and the number of malicious slots within $s_{cd}$ slots of a single sync epoch. To achieve this, identify the minimum number of honest slots, denoted by $H$, and the maximum number of malicious slots, denoted by $m$. -We can show with the Chernoff bound that for all $\omega \in (0,1)$ +Using the Chernoff bound, the probability of deviation can be bounded for all $\omega \in (0,1)$: $$ \Pr[H < (1-\omega_H) p_H\_\mathsf{timely} p_\bot^{\D_m} s_{cd}] \leq \exp(-\frac{p_H\_\mathsf{timely}p_\bot^{\D_m} s_{cd} \omega^2}{2}) @@ -295,7 +278,11 @@ $$ So, $dif = h-m \geq s_{cd}((1-\omega)p_H\_\mathsf{timely}p_\bot^{\D_m} - (1+\gamma) p_m\_\mathsf{timely})$. Let's denote $dif = m + \ell$ where $\ell \geq dif - (1+\gamma) p_m\_\mathsf{timely} s_{cd}$ -Assume that the last block of the previous sync-epoch is $B$. So, we only consider the chains that are constructed on top of $B$. Consider a chain $C$ which has finalized blocks spanned in subslots $sl_u$ and $sl_v = sl_u + s_{cd}$. The longest subchain produced between $sl_u$ and $sl_v$ is $h \geq 2m + \ell$ because of the honest chain growth among the chains constructed on top $B$. The longest subchain with more malicious blocks than the honest blocks is possible with $m$ malicious blocks and $m$ honest blocks. However, this chain can never beat the longest subchain produced at the end of $sl_u$ except with probability $\frac{1}{2^\ell}$. This means that there is not any subchain that has more malicious block and can be finalized except with a negligible probability. Therefore, all finalized chains in a synch epoch has more honest slots. +Assuming the last block of the previous sync epoch is denoted by $B$, the chains under consideration are those constructed on top of $B$. Let $C$ be a chain with finalized blocks spanning subslots $sl_u$ to $sl_v$, where $sl_v = sl_u + s_{cd}$. The longest subchain produced between $sl_u$ and $sl_v$ satisfies $h \geq 2m + \ell$, due to the honest chain growth among chains built on top of $B$. + +A subchain containing more malicious blocks than honest blocks is achievable with $m$ malicious and $m$ honest blocks. However, such a chain cannot surpass the longest honest subchain, except with probability at most $\frac{1}{2^\ell}$. In other words, a subchain dominated by malicious blocks that can be finalized is possible only with negligible probability. + +Therefore, all finalized chains within a sync epoch contain a majority of honest slots. $$ \tag{$\blacksquare$} @@ -303,11 +290,11 @@ $$

-We note that we need the chain densisty property only for the BABE with the median algorithm. +The chain densisty property is required only for BABE with the median algorithm. -**Theorem 3 (Existential Chain Quality):** Let $\D \in \mathbb{N}$ and let $\frac{p_h\\p_\bot^\D}{c} > \frac{1}{2}$. Then, the probability of an adversary $\A$ violates the ECQ property with parameters $k_{cq}$ with probability at most $e^{-\Omega(k_{cq})}$ in BABE with NTP. +**Theorem 3 or Existential Chain Quality.** If $\D \in \mathbb{N}$ and $\frac{p_h\\p_\bot^\D}{c} > \frac{1}{2}$, then the probability that an adversary $\A$ violates the ECQ property with parameter $k_{cq}$ is at most $e^{-\Omega(k_{cq})}$ in BABE with NTP. -**Proof (sketch):** If $k$ proportion of a chain does not include any honest blocks, it means that the malicious slots are more than the good and honest slots between the slots that spans these $k$ blocks. Since the probability of having good and honest slots is greater than $\frac{1}{2}$, having more bad slots falls exponentially with $k_{cq}$. Therefore, the ECQ property is broken in $R$ slots at most with the probability $e^{-\Omega(k_{cq})}$. +**Proof (sketch).** If a proportion $k$ of a chain contains no honest blocks, this implies that the number of malicious slots exceeds the number of good and honest slots within the slot range spanning those $k$ blocks. Given that the probability of a slot being good and honest is greater than $\frac{1}{2}$, the likelihood of encountering more bad slots than good ones diminishes exponentially with $k_{cq}$. As a result, the ECQ property may be violated in at most $R$ slots, with probability bounded by $e^{-\Omega(k_{cq})}$. $$ \tag{$\blacksquare$} @@ -315,30 +302,32 @@ $$

-Let $\D_m \in \mathbb{N}$ and let $\frac{p_Hp_\bot^{\D_m}}{c} > \frac{1}{2}$. Then, the probability of an adversary $\A$ violates the ECQ property with parameters $k_{cq}$ with probability at most $e^{-\Omega(k_{cq})}$ in BABE with median. +In BABE with the median algorithm, if $\D_m \in \mathbb{N}$ and $\frac{p_Hp_\bot^{\D_m}}{c} > \frac{1}{2}$, then the probability that an adversary $\A$ violates the ECQ property with parameter $k_{cq}$ is at most $e^{-\Omega(k_{cq})}$. -**Theorem 4 (Common Prefix):** Let $k,\D \in \mathbb{N}$ and let $\frac{p_H p_\bot^\D}{c} > \frac{1}{2}$, the adversary violates the common prefix property with parammeter $k$ in $R$ slots with probability at most $\exp(− \Omega(k))$ in BABE with NTP. -We should have the condition $\frac{p_Hp_\bot^{\D_m}}{c} > \frac{1}{2}$ for BABE with median. +**Theorem 4 or Common Prefix.** If $k,\D \in \mathbb{N}$ and $\frac{p_H p_\bot^\D}{c} > \frac{1}{2}$, then an adversary can violate the Common Prefix property with parameter $k$ over $R$ slots with probability at most $\exp(− \Omega(k))$ in BABE with NTP. +For BABE with the median algorithm, the condition $\frac{p_Hp_\bot^{\D_m}}{c} > \frac{1}{2}$ must be considered instead. #### Overall Results: -According to Lemma 10 in [Genesis](https://eprint.iacr.org/2018/378.pdf) **chain growth** is satisfied with +According to Lemma 10 in [Genesis](https://eprint.iacr.org/2018/378.pdf), the **Chain Growth** property is satisfied with $$ s_{cg} = 2 s_{ecq} + s_{hcg} \text{ and } \tau = \tau_{hcg} \frac{s_{hcg}}{2 s_{ecq} + s_{hcg}} $$ -and **chain quality** is satisfied with +and the **Chain Quality** property is satisfied with $$ s_{cq} = 2 s_{ecq} + s_{hcq} \text{ and } \mu = \tau_{hcq}\frac{s_{hcq}}{2s_{ecq}+s_{hcq}} $$ -**Theorem 5 (Persistence and Liveness BABE with NTP):** Assuming that $\frac{p_H p_\bot^\D}{c} > \frac{1}{2}$ and given that $k_{cq}$ is the ECQ parameter, $k > 2k_{cq}$ is the CP parameter, $s_{hcg} = k/\tau_{hcg}$, $s_{ecq} = k_{cq}/\tau$, the epoch length is $R = 2s_{ecq} + s_{hcg}$ BABE with NTP is persistent and live. +**Theorem 5 or Persistence and Liveness of BABE with NTP.** Assuming $\frac{p_H p_\bot^\D}{c} > \frac{1}{2}$ and given that $k_{cq}$ is the ECQ parameter, $k > 2k_{cq}$ is the Common Prefix parameter, $s_{hcg} = k/\tau_{hcg}$ and $s_{ecq} = k_{cq}/\tau$, then the epoch length is $R = 2s_{ecq} + s_{hcg}$, and BABE with NTP is persistent and liveness. -**Proof (Sketch):** The overall result says that $\tau = \tau_{hcg}\frac{s_{hcg}}{2s_{ecq}+s_{hcg}} = \frac{k}{s_{hcg}}\frac{s_{hcg}}{2s_{ecq}+s_{hcg}} = \frac{k}{R}$. The best chain at the end of an epoch grows at least $k$ blocks in one epoch thanks to the chain growth. +**Proof (Sketch).** The overall result shows that $\tau = \tau_{hcg}\frac{s_{hcg}}{2s_{ecq}+s_{hcg}} = \frac{k}{s_{hcg}}\frac{s_{hcg}}{2s_{ecq}+s_{hcg}} = \frac{k}{R}$. So by the chain growth property, the best chain increases by at least $k$ blocks over the course of a single epoch. - Since $k > 2k_{cq}$, the last $k_{cq}$ block of includes at least one honest block. Therefore, the randomness includes one honest randomness and the adversary can have at most $s_{ecq}$ slots to change the randomness. This grinding effect can be upper-bounded by $s_{ecq}(1-\alpha)nq$ where $q$ is the hashing power [2]. The randomness generated by an epoch is finalized at latest one epoch later thanks to the common prefix property. Similary, the session key update which is going to be used in three epochs later is finalized one epoch later before a randomness of the epoch where the new key are going to be used starts to leak. + Since $k > 2k_{cq}$, the last $k_{cq}$ blocks must contain at least one honest block, while the associated randomness must include at least one honest input. This implies that the adversary has at most $s_{ecq}$ slots to attempt to manipulate the randomness. Such a grinding effect can be upper-bounded by $s_{ecq}(1-\alpha)nq$, where $q$ is the adversary's hashing power [^2]. + +By the Common Prefix property, the randomness generated during an epoch must be finalized no later than one epoch afterward. Similary, the session key update, used three epochs later, must be finalized one epoch earlier, before the randomness of the epoch in which the new key will be used begins to leak. Therefore, BABE with NTP is persistent and live. $$ @@ -347,156 +336,170 @@ $$

-**Theorem 6 (Persistence and Liveness BABE with the Median Algorithm):** Assuming that $\frac{p_H p_\bot^{\D_m}}{c} > \frac{1}{2}$ and $\tau_{hcg}-\tau_{hcg}\mu_{hcq} > p_m (1+\gamma)$ where $\tau_{hcg} = p_h p_\bot^{\D_m} (1-\omega)$, $s_{cd}$, the clock difference is between honest valdators is at most $\D_m$, BABE with median is persistent and live given that given that $k_{cq}$ is the ECQ parameter, $k > 2k_{cq}$ is the CP parameter, $s_{hcg} = k/\tau_{hcg}$, $s_{ecq} = k_{cq}/\tau$. +**Theorem 6 or Persistence and Liveness of BABE with the Median Algorithm.** Assuming that $\frac{p_H p_\bot^{\D_m}}{c} > \frac{1}{2}$ and $\tau_{hcg}-\tau_{hcg}\mu_{hcq} > p_m (1+\gamma)$, where $\tau_{hcg} = p_h p_\bot^{\D_m} (1-\omega)$, $s_{cd}$, and since the clock difference between honest valdators is at most $\D_m$, then BABE with the median algorithm satisfies persistence and liveness given that: + +* $k_{cq}$ is the ECQ parameter + +* $k > 2k_{cq}$ is the CP parameter + +* $s_{hcg} = k/\tau_{hcg}$ + +* $s_{ecq} = k_{cq}/\tau$ -**These results are valid assuming that the signature scheme with account key is EUF-CMA (Existentially Unforgible Chosen Message Attack) secure, the signature scheme with the session key is forward secure, and VRF realizing is realizing the functionality defined in [2].** +**These results hold under the following assumptions: the signature scheme using the account key is EUF-CMA (Existentially Unforgeability under Chosen Message Attack) secure, the signature scheme based on the session key is forward-secure, and the VRF correctly realizes the functionality as defined in [^2].** --- -## 6. Practical Results +## 6. Practical results -In this section, we find parameters of two versions of BABE to achieve the security in BABE. +This section specifies the parameters necessary to achieve security in both variants of the BABE protocol. -We fix the lifetime of the protocol as $\mathcal{L}=3 \text{ years} = 94670777$ seconds. We denote the slot time by $T$ (e.g., $T = 6$ seconds). -The lifetime of the protocol in terms of slots is $L = \frac{\mathcal{L}}{T}$. The maximum network delay is $\D$. +The protocol lifetime is fixed as $\mathcal{L}=3 \text{ years} = 94670777$ seconds. Let $T$ denote the slot duration (e.g., $T = 6$ seconds). The total number of slots over the lifetime is $L = \frac{\mathcal{L}}{T}$. Finally, the maximum network delay is $\D$. ### BABE with the NTP -* Define $\delta\_max$ and $T$. Let $\D = 0$ if $\delta_{\max} < T$. Otherwise, let $\D = \lceil \frac{\delta\_max - T}{T}\rceil$ -* Decide the parameter $c$ such that the condition $\frac{p_Hp_\bot^{\D}}{c} > \frac{1}{2}$ is satisfied. If there is not any such $c$, then consider to increase $\alpha$ (honest validator assumption) or decrease $\D$ (more optimistic network assumption). -* Set up a security bound $p_{attack}$ to define the probability of an adversary to break BABE in e.g., 3 years. Of course, very low $p$ is better for the security of BABE but on the other hand it may cause to have very long epochs and long probabilistic finalization. Therefore, I believe that setting $p_{attack}=0.005$ is reasonable enough in terms of security and performance. -* Set $\omega \geq 0.5$ (e.g., 0.5) and find $s_{ecq}$ and $s_{hcq}$ to set the epoch length $R = 2 s_{ecq} + s_{hcg}$ such that $p_{attack} \leq p$. For this we need an initial value $k_{cp}$ and find $s_{ecq}, s_{hcg}$ and $\tau$ that satisfies the three equations below: +* Define $\delta\_max$ and $T$. Let $\D = 0$ if $\delta_{\max} < T$; otherwise, let $\D = \lceil \frac{\delta\_max - T}{T}\rceil$ +* Choose the parameter $c$ such that $\frac{p_Hp_\bot^{\D}}{c} > \frac{1}{2}$. If no such $c$ exists, consider increasing the honest validator assumption $\alpha$, or adopting a more optimistic network assumption by decreasing $\D$. +* Define a security bound $p_{attack}$ to represent the probability that an adversary can break BABE over a fixed duration (e.g., 3 years). A lower value of $p$ improves security, but may lead to longer epochs and extended probabilistic finalization. A value of $p_{attack}=0.005$ represents a reasonable compromise between security and performance. +* Set $\omega \geq 0.5$ (e.g., 0.5), and compute $s_{ecq}$ and $s_{hcq}$ to define the epoch length $R = 2 s_{ecq} + s_{hcg}$ such that the condition $p_{attack} \leq p$ holds. To do this, select an initial value $k_{cp}$ and determine $s_{ecq}, s_{hcg}$ and $\tau$ such that they satisfy the following three equations: -From Theorem 6, we want that the best chain grows at least $k$ blocks. Therefore, we need +From Theorem 6, the goal is for the best chain to grow by at least $k$ blocks. To ensure this, the following condition must hold: $$ (2s_{ecq} + s_{hcg})\tau = k\text{ }\text{ }\text{ }\text{ }\text{ }\text{ (1)} $$ -We need $s_{ecq}$ slots to guarantee $k_{cq}$ blocks growth for the ECQ property. So, we need: +To guarantee $k_{cq}$ blocks for the ECQ property, it is required that: $$ \tau s_{ecq} = k_{cq} \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ (2)} $$ -Lastly, we need the following as given in the Overall Result: +Finally, the Overall Result gives: $$ \tau = \tau_{hcg} \frac{s_{hcg}}{2 s_{ecq} + s_{hcg}}\text{ }\text{ }\text{ }\text{ }\text{ (3)} $$ -Iterate $k_{cp}$ to find $s_{hcg}, s_{ecq}, \tau$ that satisfy above conditions until $p_{attack} \leq p$: +Iterate over $k_{cp}$ to find values for $s_{hcg}, s_{ecq}, \tau$ that satisfy the above conditions until $p_{attack} \leq p$: -1. Let $k = 4 k_{cp}$ (The CQ property parameter) We note that $4 k_{cp}$ is the optimal value that minimizes $R = 2 s_{ecq} + s_{hcg}$. -1. $t_{hcg} = p_h p_\bot^\D (1-\omega)$ (to satisfy the condition in Theorem 1) -1. $s_{hcg} = k / t_{hcg}$ (from Equation (1) and (3)) -1. $\tau = \frac{k - 2k_{cq}}{s_{hcg}}$ (from Equation (1) and (2)) -1. $s_{ecq} = k_{cq}/\tau$ -1. $p = \lceil \frac{L}{T}\rceil\frac{2^{20}(1-\alpha)n}{R}(p_{ecq} + p_{cp} + p_{cg})$ +1. Set the parameter for the Chain Quality (CQ) property at $k = 4 k_{cp}$. $4 k_{cp}$, which is the optimal value to minimize the epoch length $R = 2 s_{ecq} + s_{hcg}$. +1. Compute $t_{hcg} = p_h p_\bot^\D (1-\omega)$ to satisfy the condition in Theorem 1 +1. Calculate $s_{hcg} = k / t_{hcg}$ based on Equations (1) and (3) +1. Determine $\tau = \frac{k - 2k_{cq}}{s_{hcg}}$ by using Equations (1) and (2) +1. Compute $s_{ecq} = k_{cq}/\tau$ +1. Calculate the security parameter: $p = \lceil \frac{L}{T}\rceil\frac{2^{20}(1-\alpha)n}{R}(p_{ecq} + p_{cp} + p_{cg})$ -After finding $k_{cq}$ such that $p \leq p_{attack}$, let the epoch length $R = 2s_{ecq}+s_{hcg}$. +Once a value for $k_{cq}$ such that $p \leq p_{attack}$ is found, set the epoch length $R = 2s_{ecq}+s_{hcg}$. -The parameters below are computed with the code in https://github.com/w3f/research/blob/master/experiments/parameters/babe_NTP.py. In this code, we choose the parameter $c$ not only according to security conditions but also according to having in expectation twice more single leader than multiple leaders. +:::note Parameters +The parameters below are computed using the code available at the [GitHub entry](https://github.com/w3f/research/blob/master/experiments/parameters/babe_NTP.py). The parameter $c$ is chosen not only to satisfy security conditions, but also to ensure that, in expectation, the number of single-leader slots is at least twice the number of multi-leader slots. -################### PARAMETERS OF BABE WITH NTP $\D = 0$ ################### -c = 0.52, slot time T = 6 +c = 0.52, slot time T = 6 seconds -It is secure in 3 years with a probability 0.99523431732 +Secure over a 3-year horizon with probability 0.99523431732 -It is resistant to (6 - block generation time) second network delay +Resistant to network delays of up to 6 - block generation time seconds -~~~~~~~~~~~~~~ Common Prefix Property ~~~~~~~~~~~~~~ k = 140 -It means: Prune the last 140 blocks of the best chain. All the remaining ones are probabilistically finalized +This means that the last 140 blocks of the best chain are pruned. All preceding blocks are considered probabilistically finalized. -~~~~~~~~~~~~~~ Epoch Length ~~~~~~~~~~~~~~ -Epoch length should be at least 1440 slots,2.4 hours - +Epoch length should be at least 1440 slots (2.4 hours). -If we want more network resistance, $e.g.,\D = 1$, the parameters should be selected as follows: +If greater network resistance is desired ($e.g.,\D = 1$), the parameters should be selected as follows: -################### PARAMETERS OF BABE WITH NTP $\D = 1$ ################### -c = 0.22, slot time T = 6 +c = 0.22, slot time T = 6 seconds -It is secure in 3 years with probability 0.996701592969 +Secure over a 3-year period with probability 0.996701592969 -It is resistant to (12 - block generation time) second network delay +Resistant to network delays of up to 12 - block generation time seconds. -~~~~~~~~~~~~~~ Common Prefix Property ~~~~~~~~~~~~~~ k = 172 -It means: Prun the last 172 blocks of the best chain. All the remaining ones are probabilistically finalized +This means that the last 172 blocks of the best chain are pruned. All preceding blocks are considered probabilistically finalized. -~~~~~~~~~~~~~~ Epoch Length ~~~~~~~~~~~~~~ -Epoch length should be at least 4480 slots, 7.46666666667 hours +Epoch length should be at least 4480 slots (approximately 7.46666666667 hours) +::: +### BABE with the median algorithm -### BABE with the Median Algorithm - -* Define $\alpha_{timely} = 0.85$, $\ell = 20$, $\omega_H = 0.3$ and $\gamma = 0.5$ in Theorem 2. +* Define the following parameters for Theorem 2: $\alpha_{timely} = 0.85$, $\ell = 20$, $\omega_H = 0.3$ and $\gamma = 0.5$. * Define $\delta\_max$ and $T$. Let $\D_m = \lfloor \frac{2\delta\_max + |2 \Sigma|}{T}\rfloor + \lfloor \frac{\delta\_max + |2 \Sigma|}{T}\rfloor$ -* Decide the parameter $c$ such that the condition $\frac{p_Hp_\bot^{\D}}{c} > \frac{1}{2}$ and $\frac{p_H\_\mathsf{timely} (1- \omega_H)}{p_m\_\mathsf{timely} (1+\gamma)} > 2$ -is satisfied. If there is not any such $c$, then consider increasing $\alpha$ (honest validator assumption) or $\alpha_{timely}$ or decreasing $\D$ (more optimistic network assumption). - -* Do the rest as in BABE with NTP. - -Finding synch-epoch length +* Choose the parameter $c$ such that both of the following conditions hold: $\frac{p_Hp_\bot^{\D}}{c} > \frac{1}{2}$ and $\frac{p_H\_\mathsf{timely} (1- \omega_H)}{p_m\_\mathsf{timely} (1+\gamma)} > 2$. If no such $c$ exists, consider increasing $\alpha$ (honest validator assumption), increasing $\alpha_{timely}$, or decreasing $\D$ (adopting a more optimistic network assumption). -1. Set $s_{cd}$ with respect to Theorem 2. +* Proceed with the remaining steps as in BABE with NTP. +Next, determine the sync-epoch length and set $s_{cd}$ according to Theorem 2. -The parameters below are computed with the code in https://github.com/w3f/research/blob/master/experiments/parameters/babe_median.py +:::note Parameters +The parameters below are computed using the script available at this [GitHub entry](https://github.com/w3f/research/blob/master/experiments/parameters/babe_median.py) -############## PARAMETERS OF BABE WITH THE MEDIAN ALGORITHM ############## -c = 0.38, slot time T = 6 +c = 0.38, slot time T = 6 seconds -It is secure in 3 years with probability 0.99656794973 +Security over 3 years with probability 0.99656794973 -It is resistant to 2.79659722222 second network delay and 0.198402777778 seconds drift in one sync-epoch +Resistant to network delay of 2.79659722222 seconds and clock drift of 0.198402777778 seconds per sync epoch -~~~~~~~~~~~~~~ Common Prefix Property ~~~~~~~~~~~~~~ -k = 140 -It means: Prune the last 140 blocks of the best chain. All the remaining ones are probabilistically finalized +k = 140 + +This means that the last 140 blocks of the best chain are pruned, while all remaining blocks are probabilistically finalized -~~~~~~~~~~~~~~ Epoch Length ~~~~~~~~~~~~~~ -Sync-Epoch length should be at least 2857 slots, 4.76166666667 hours +Sync-epoch length: at least 2857 slots (~4.7617 hours) -Epoch length should be at least 2000 slots,3.33333333333 hours +Epoch length: at least 2000 slots (~3.3333 hours) -~~~~~~~~~~~~~~ Offline validators' parameters for clock adjustment ~~~~~~~~~~~~~~ -$n = 200$ for temporarily clock adjustment. +$n = 200$ for temporary clock adjustment. + +Offline validators should collect. +::: + +**Some Notes on clock drifts:** -Offline validators should collect +Computer clocks are inherently imprecise because the frequency that drives time progression is never exactly accurate. For instance, a frequency error of about 0.001% can cause a clock to drift by nearly one second per day. -**Some Notes about clock drifts:** -http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm#AEN1220 -All computer clocks are not very accurate because the frequency that makes time increase is never exactly right. For example the error about 0.001% make a clock be off by almost one second per day. -Computer clocks drift because the frequency of clocks varies over time, mostly influenced by environmental changes such as temperature, air pressure or magnetic fields, etc. Below, you can see the experiment in a non-air conditioned environment on linux computer clocks. 12 PPM correspond to one second per day roughly. I seems that in every 10000 second the change on the clocks are around 1 PPM (i.e., every 3 hours the clocks drifts 0.08 seconds.). We can roughly say that the clock drifts around 1 second per day. If we have sync epoch around 12 hours it means that we have 0.5 second drift and +Clock drift occurs because the oscillation frequency varies over time, primarily due to environmental factors such as temperature, air pressure, and magnetic fields. Experiments conducted on Linux systems in non-air conditioned environments show that a drift of 12 PPM (parts per million) corresponds to roughly one second per day. + +Observation suggets that over every 10,000 seconds, the clock frequency changes by 1 PPM, resulting in a drift of approximately 0.08 seconds every three hours. Thus, a rough estimate of one second of drift per day is reasonable. If the sync epoch spans 12 hours, this implies a clock drift of approximately 0.5 seconds over that period. For further details, refer to the [NTP Clock Quality FAQ](http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm#AEN1220) + +
[![](https://i.imgur.com/Slspcg6.png)](http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm#AEN1220) -**Figure. Frequency Correction within a Week** +Frequency Correction within a Week +
-## References +**For inquieries or questions, please contact** [Bhargav Nagajara Bhatt](/team_members/JBhargav.md) [1] Kiayias, Aggelos, et al. "Ouroboros: A provably secure proof-of-stake blockchain protocol." Annual International Cryptology Conference. Springer, Cham, 2017. -[2] David, Bernardo, et al. "Ouroboros praos: An adaptively-secure, semi-synchronous proof-of-stake blockchain." Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, Cham, 2018. +[^2] David, Bernardo, et al. "Ouroboros praos: An adaptively-secure, semi-synchronous proof-of-stake blockchain." Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, Cham, 2018. + +[^3] Badertscher, Christian, et al. "Ouroboros genesis: Composable proof-of-stake blockchains with dynamic availability." Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2018. -[3] Badertscher, Christian, et al. "Ouroboros genesis: Composable proof-of-stake blockchains with dynamic availability." Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2018. +[^4] An epoch and a sync-epoch are distinct concepts -[4] Aggelos Kiayias and Giorgos Panagiotakos. Speed-security tradeoffs in blockchain protocols. Cryptology ePrint Archive, Report 2015/1019, 2015. http://eprint.iacr.org/2015/1019 +[5] Aggelos Kiayias and Giorgos Panagiotakos. Speed-security tradeoffs in blockchain protocols. Cryptology ePrint Archive, Report 2015/1019, 2015. http://eprint.iacr.org/2015/1019 diff --git a/docs/Polkadot/protocols/block-production/Block-production2.png b/docs/Polkadot/protocols/block-production/Block-production2.png new file mode 100644 index 00000000..b980e39e Binary files /dev/null and b/docs/Polkadot/protocols/block-production/Block-production2.png differ diff --git a/docs/Polkadot/protocols/block-production/SASSAFRAS.md b/docs/Polkadot/protocols/block-production/SASSAFRAS.md index 46553234..ca42dd62 100644 --- a/docs/Polkadot/protocols/block-production/SASSAFRAS.md +++ b/docs/Polkadot/protocols/block-production/SASSAFRAS.md @@ -1,153 +1,184 @@ --- -title: SASSAFRAS +title: Sassafras --- -**Authors**: [Jeff Burdges](/team_members/jeff.md), Fatemeh Shirazi, [Alistair Stewart](/team_members/alistair.md), [Sergey Vasilyev](/team_members/Sergey.md) + -BADASS BABE is a constant-time block production protocol. It intends to ensure that there is exactly one block produced with constant-time intervals rather than multiple or none. It extends on BABE to address this shortcoming of BABE. While [Jeff's Write-up](https://github.com/w3f/research/tree/master/docs/papers/habe) describes the whole design space of constant-time block production, here we describe a practical instantiation of the protocol using zk-SNARKs to construct a ring-VRF. +BADASS BABE is a constant-time block production protocol designed to ensure that exactly one block is produced at constant-time intervals, thereby avoiding multiple block production and empty slots. It builds upon BABE to address this limitation in the original protocol. While [Jeff's write-up](https://github.com/w3f/research/tree/master/docs/papers/habe) explores the full design space of constant-time block production, the focus here is on a practical instantiation using zk-SNARKs to construct a ring-VRF. ## Layman overview -We want to run a lottery to distribute the block production slots in an epoch, to fix the order validators produce blocks by the beginning of an epoch. Each validator signs the same on-chain randomness (VRF input) and publishes this signature (VRF output=[value, proof]). This value is their lottery ticket, that can be validated against their public key. The problem with this approach is that the lottery-winners are publicly known in advance and risk becoming targets of attacks. We aim to keep the block production order anonymous. The assignment of the validators to the slots should be fixed for the whole epoch, but noone besides the assigned validator should know whose a slot is. However, we can't validate the tickets prior the lottery using their public keys as it would deanonymize the validators. If tickets were not validated prior to the lottery then instead we can validate them after the lottery by an honest validator claiming their slots when producing blocks. +Simply put, the aim of the protocol is twofold: to run a lottery that distributes the block production slots in an epoch, and to fix the order in which validators produce blocks at the start of that epoch. Each validator signs the same on-chain randomness (VRF input) and publishes the resulting signature (VRF output=[value, proof]). This value serves as their lottery ticket, which can be validated against their public key. -However, the problem is that anyone can submit fake tickets, and though they won't be able to produce a block, slots would be preassigned to them. Effectively, it results in empty slots, which defeats the goal of the protocol. To address this problem, we need a privacy-preserving way of validating a ticket. So an honest validator when submitting their ticket accompanies it with a SNARK of the statement: "Here's my VRF output that has been generated using the given VRF input and my secret key. I'm not telling you my keys, but my public key is among those of the nominated validators", that is validated before the lottery. +The approach reveals the lottery winners in advance, making them potential targets for attacks. The goal is then to keep the block production order anonymous. While the assignment of the validators to slots should remain fixed throughout the epoch, no one other than the assigned validator should know which slot is assigned to whom. -Now we have a way of making the ticket itself anonymous, we need a way to anonymously publish it to the chain. All ways of doing this with full anonymity are expensive. Fortunately, one of the simplest schemes is good enough for our purposes: a validator just sends each of their tickets to a random validator who later puts it on-chain as a transaction. +Validating tickets before the lottery using public keys can compromise anonymity. Instead, tickets can be validated after the lottery, when an honest validator claims their slot by producing a block. + +The main issue with this approach is that anyone can submit fake tickets. Although these entities wouldn't be able to produce blocks, slots could still be preassigned to them, resulting in empty slots, which undermines the goal of the protocol. What's needed then is a privacy-preserving method for validating tickets. + +Relying on such a method, an honest validator could submit their ticket along with a SNARK, validated before the lottery, proving the statement: "This is my VRF output, generated using the given VRF input and my secret key. I won't reveal my keys, but my public key is among those of the nominated validators." + + +Once the ticket is made anonymous, the next step is to publish it to the chain without revealing its origin. While fully anonymous methods tend to be costly, a simple scheme suffices to achieve the core objectives: each validator can send their ticket to a randomly chosen peer, who then submits it on-chain as a transaction. ## Plan -In an epoch $e_m$ we use BABE randomness $r_m$ for the epoch as ring VRF inputs to produce a number of outputs and publish them on-chain. After they get finalized we sort them and their order defines the order of block production for the epoch $e_{m+2}$. +In an epoch $e_m$, BABE randomness $r_m$ is used as the ring VRF input to generate a set of outputs, which are then published on-chain. Once finalized, these outputs are sorted, and their order determines the block production sequence for epoch $e_{m+2}$. ## Parameters -$V$ - the set of nominated validators -$s$ - number of slots per epoch, for an hour-long epoch with 6 second slots $s=600$ -$x$ - redundancy factor, for an epoch of $s$ slots we want to have $xs$ tickets in expectation for block production. We set $x=2$. -$a$ - attempts number of tickets generated per validator in epoch -$L$ - a bound on a number of tickets that can be gossiped, used for DoS resistance +* $V$: The set of nominated validators +* $s$: Number of slots per epoch. For an hour-long epoch with 6-second slots, $s=600$ +* $x$: Redundancy factor. For an epoch with $s$ slots, the aim is to generate $xs$ tickets in expectation for block production. In this analysis, we set $x=2$. +* $a$: Number of ticket-generation attempts per validator in an epoch +* $L$: A bound on the number of tickets that can be gossiped, used for DoS resistance ## Keys -In addition to their regular keys, we introduce for each validator a keypair on a SNARK-friendly curve [Jubjub](https://z.cash/technology/jubjub/). We must ensure that the keys are generated before the randomness for an epoch they are used in is determined. +In addition to their regular keys, each validator must posssess a keypair on a SNARK-friendly curve such as [Jubjub](https://z.cash/technology/jubjub/). It is essential to generate these keys before the randomness for the epoch is derived or finalized. + +Given a security parameter $\lambda$ and randomness $r$, generate a key pair using the RVRF key generation function +$$ +\texttt{KeyGen}_{RVRF}:\lambda,r\mapsto sk, pk +$$ -Given the security parameter $\lambda$ and some randomness $r$ generate a key pair $\texttt{KeyGen}_{RVRF}:\lambda,r\mapsto sk, pk$ +To optimize the process, an aggregate public key $apk$, referred to as a commitment in Jeff's write-up, is introduced for the full set of validators. This key is essentially a Merkle root derived from the list of individual public keys. -As an optimization we introduce an aggregate public key $apk$ (called a commitment in Jeff's writeup) for the whole set of validators, that is basically a Merkle root built upon the list of individual public keys. In conjuction to that we use the copath $ask_v$ to identify a public key in the tree as a private input to a SNARK. -$\texttt{Aggregate}_{RVRF}: v, \{pk_v\}_{v\in V}\mapsto apk, ask_v$ +$$ +\texttt{Aggregate}_{RVRF}: v, \{pk_v\}_{v\in V}\mapsto apk, ask_v +$$ +The copath $ask_v$ serves to identify a specific public key within the tree as a private input to a SNARK. ## Phases -Here we describe the regular operation of the protocol starting from a new set of validators being nominated. Bootstrapping the protocol from the genesis or soft-forking Kusama is not described here. +Bootstrapping the protocol from genesis or through a soft fork of Kusama is beyond the scope of this description. The regular operation of the protocol thus begins with the nomination of a new set of validators. ### 1) Setup -Once per era, as a new set of validators $V$ gets nominated or some other parameter changes, we reinitialize the protocol with new values for the threshold $T$ and the aggregated public key $apk$. +As a new set of validators $V$ is nominated, or another protocol parameter changes, the protocol reinitializes once per era with updated values for the threshold $T$ and the aggregated public key $apk$. Each validator $v \in V$ -1. Calculates the threshold $T = \frac{xs}{a\mid V\mid}$ that prevents the adversary to predict how many more blocks a block producer is going to produce. +1. Calculates the threshold $T = \frac{xs}{a\mid V\mid}$. This value prevents adversaries to predicting how many additional blocks a block producer will generate. 2. Computes the aggregated public key and copath of $v$s public key $$ apk, spk_v = \texttt{Aggregate}_{RVRF}(v, \{pk_v\}_{v\in V}) $$ -3. Obtains the SNARK CRS and checks for subversion if it has changed or $v$ hasn't done it earlier. +3. Obtains the SNARK CRS and verifies whether the subversion status has changed or if $v$ has not previously performed this step. -### 2) VRF generation Phase +### 2) VRF generation phase -We aim to have at least $s$ VRF outputs (tickets) published on-chain (we can't really guarantee that, but the expected value will be $xS$). +The objective is to have at least $s$ VRF outputs (tickets) published on-chain. Although this cannot be strictly guaranteed, the expected value is $xS$. #### Randomness -At the epoch $e_m$ we use the randomness $r_m$ as provided by [BABE](polkadot/protocols/block-production/Babe), namely +At the epoch $e_m$, [BABE](polkadot/protocols/block-production/Babe) provides randomness $r_m$, defined as follows: $$ r_m=H(r_{m-1}, m, \rho) $$ -We use $r_m$ to create inputs to the ring-VRF, and the corresponding tickets will be consumed in $e_{m+2}$. +Here, $r_m$ creates inputs to the ring-VRF, with resulting tickets consumed in epoch $e_{m+2}$. -It's critical that $\rho$ is still the concatenation of regular BABE VRF outputs. It follows that we run regular VRFs and ring VRFs in parallel. This is because ring VRF outputs will be revealed in epoch $e_m$ and hence if we use ring VRF outputs for randomness $r_{m+1}$ would be revealed too early. Thus we use VRFs that are unrevealed until the corresponding blocks are produced. +It's critical that $\rho$ remains the concatenation of regular BABE VRF outputs. Standard VRFs and ring VRFs are then excecuted regularly and concurrently, with ring-VRF outputs revealed in epoch $e_m$. If VRF outputs are used prematurely for randomness, $r_{m+1}$ would be exposed too early. Thus, only unrevealed VRFs are used until their corresponding blocks have been produced. -If we have a VDF, then all this would need to be determined an epoch prior i.e. +In the case of a VDF, randomness would need to be determined one epoch earlier, i.e., $$ r_m=VDF(H(r_{m-2}, m, \rho)) $$ -with $\rho$ being the concatenation of BABE VRFs from $e_{m-2}$. The VDF would be run at the start of $e_{m-1}$ so that the output would be on-chain before $e_{m}$ starts. +where $\rho$ is the concatenation of BABE VRFs from epoch $e_{m-2}$. The VDF would be excecuted at the start of $e_{m-1}$, ensuring that its output is available on-chain before $e_{m}$ begins. #### VRF production -Each validator $v \in V$ +Each validator $v \in V$ performs the following steps: -1. Given the randomness $r_{m}$, computes a bunch of $a$ VRF outputs for the inputs $in_{m,i}=(r_m, i)$, $i = 1,\ldots,a$: +1. Computes $a$ VRF outputs using the randomness $r_{m}$, for inputs $in_{m,i}=(r_m, i)$, where $i = 1,\ldots,a$: $$ out_{m,v,i}=\texttt{Compute}_{RVRF}(sk_v, in_{m, i}) $$ -2. Selects the "winning" outputs that are below the threshold $T$: $\texttt{bake}(out_{m,v,i}) < T$ -where $\texttt{bake()}$ is a function that effectively maps VRF outputs to the interval $[0,1]$. We call the set of $i$ corresponding to winning outputs $I_{win}$. +2. Selects "winning" outputs below the threshold $T$: $\texttt{bake}(out_{m,v,i}) < T$, +where $\texttt{bake()}$ maps VRF outputs to the interval $[0,1]$. The indices correponding to winning outputs form the set $I_{win}$. -3. Uses its copath $ask_v$ generate proofs for the selected outputs $i \in I_{win}$, +3. Generates proofs using its copath $ask_v$ for each winning output $i \in I_{win}$, $$ \pi_{m,v,i} = \texttt{Prove}_{RVRF}(sk_v, spk_v, in_{m,i} ) $$ -where $\texttt{Prove}_{RVRF}(sk_v, spk_v, in_{m,j} )$ consists of the SNARK and its public inputs $cpk,i$. +where $\texttt{Prove}_{RVRF}(sk_v, spk_v, in_{m,j} )$ includes the SNARK and associated public inputs $cpk,i$. -As the result of this phase every validator obtains a number, possibly 0, of winning tickets together with proofs of their validity $(j, out_{m, v,j}, \pi_{m,v,j})$ that need to be published on-chain. +Once this phase concludes, each validator holds zero or more winning tickets and corresponding validity proofs $(j, out_{m, v,j}, \pi_{m,v,j})$. These must later be published on-chain. -### 3) Publishing Phase -We want block producers for at least a large fraction of slots unknown in advance. Thus well-behaved validators should keep their tickets private. To this end validators dont publish their winning VRF outputs themselves immediately, but instead relay them to another randomly selected validator (proxy) who then publishes it. +### 3) Publishing phase +The goal is to identify block producers for a large portion of slots unknown in advance. Well-behaved validators should keep their tickets private. To achieve this, validators do not publish their winning VRF outputs immediately; instead, they relay them to another randomly selected validator (a proxy), who is responsible for publishing them on-chain. -Concretely, $v$ chooses another validator $v'$, based on the output $out_{m,v,i}$ for $i \in I_{win}$. To this end, the validator takes $k=out_{m,v,i} \textrm{mod} |V|$ and sends its winning ticket to the $k$th validator in a fixed ordering. Then the validator signs the message: $(v, l, enc_v'(out_{m,v,i}, \pi_{m,v,i}))$ where $end_{v'}$ refers to encrypted to a public key of $v'$. We number the winning outputs using $l$ ranging from $0$ up to $L-1$ and gossip them. If we have more than $L$ outputs below $T$, we gossip only the lowest $L$. This limitation is so that it is impossible for a validator to spam the network. +Concretely, validator $v$ selects another validator $v'$ based on the output $out_{m,v,i}$ for $i \in I_{win}$. The validator computes $k=out_{m,v,i} \textrm{mod} |V|$ and sends its winning ticket to the $k$th validator according to a fixed ordering. Then, validator $v$ signs the message: $(v, l, enc_v'(out_{m,v,i}, \pi_{m,v,i}))$ where $end_{v'}$ denotes encryption using the public key of $v'$. Winning outputs are indexed using $l$ ranging from $0$ to $L-1$, and are gossiped through the network. If there are more than $L$ outputs below the threshold $T$, only the lowest $L$ are disseminated. This limitation helps prevent validator from spamming the network. -Once a valiaotr receives a messages it checks whether it has received a message with the same $v$ and $l$ and if so it discards the new message. Otherwise, the validator forwards (gossips) the message and decrypts it to find out whether the validator is the intended proxy. Validators gossip messages that are intended for them further to be secure against traffic correlation. +Once a validator receives a message, it checks whether it has already received a message with the same $v$ and $l$; if so, it discards the new message. Otherwise, it decrypts the message to determine whether it is the intended proxy and forwards (gossips) the message. Validators further gossip messages addressed to themselves to mitigate traffic correlation risks. -Once a validator decrypts a message with their private key they verify that they were the correct proxy, i.e. that $out_{m,v,i} \textrm{mod} |V|$ corresponds to them. If so, then at some fixed block number, they send a transaction including $(out_{m,v,i}, \pi_{m,v,i}))$ for inclusion on-chain. Note that the validator might have been proxy for a number of tickets, in that case, it sends a number of transaction on designated block number. +When a validator decrypts a message using its private key, it verifies whether it was the correct proxy by checking out that $out_{m,v,i} \textrm{mod} |V|$ corresponds to its index. If confirmed, it then broadcasts a transaction at a designated block number, containing $(out_{m,v,i}, \pi_{m,v,i}))$ for inclusion on-chain. If the validator serves as a proxy for multiple tickets, it submits multiple transactions at the appointed block. -If a validators $v$ ticket is not included on-chain before some later block number, either because the proxy is misbehaving or because they havent sent the winning ticket to any proxies, then $v$ publishes the transaction $(out_{m,v,i}, \pi_{m,v,i})$ themselves. The reason why a validator would not send a winning ticket to any proxy is that it has more than $L$ winning tickets. +If validator $v$'s ticket is not included on-chain before a certain block number, either due to proxy misbehavior or because it did not forward the ticket to any proxy, then $v$ submits the transaction $(out_{m,v,i}, \pi_{m,v,i})$ independently. A validator might refrain from selecting a proxy when it holds more than $L$ winning tickets. ### 4) Verification -A transaction of this sort is valid for inclusion in a block if it can be verified as follows. - -To verify the published transactions $(out_{m, v,i}, \pi_{m,v,i})$, we need to verify the SNARK. For this we need -- the corresponding input $in_{m,i}$, which we can calculate from $i$ and $r_m$, +A transaction of this type is valid for block inclusion if it can be verified. To check published transactions $(out_{m, v,i}, \pi_{m,v,i})$, the corresponding SNARK proof must hold. This verification requires: +- the input $in_{m,i}$, which can be computed from $i$ and $r_m$, - the published output $out_{m,v,i}$ -- the aggregate public key $apk$. +- the aggregate public key, denoted as $apk$. -All of these are the public inputs in SNARK verification: +These values constitute the public inputs to the SNARK verifier: $$ Verify(\pi_{m,v,i}, apk, out_{m,v,i}, in_{m,i}) $$ ### 5) Sorting -In the epoch $e_{m+2}$ we have the list $\{out_{m,k}\}_{k=1}^{K}$ of $K$ verified VRF outputs generated during the epoch $e_m$ which are finalized on-chain. For each of these outputs, we combine the ouput with the randomness $r'$, with either $r'=r_{m+1}$ if we do not have a VDF or $r'=r_{m+2}$ if we do have a VDF. Then we compute $out'_{m,k}=H(out_{m,k} || r')$. +Epoch $e_{m+2}$ contains the list $\{out_{m,k}\}_{k=1}^{K}$ of $K$ verified VRF outputs generated during the epoch $e_m$, which are finalized on-chain. Each output is combined with a source of randomness $r'$, where: +- $r'=r_{m+1}$ if no VDF is used, or +- $r'=r_{m+2}$ if a VDF is used. + +The resulting hash is computed as: $out'_{m,k}=H(out_{m,k} || r')$ -To determine the block production order for the epoch $e_{m+2}$, each validator sorts the list of $out'_{m,k}$ in ascending order and drops the largest $s-K$ values if any: $out'_{m,1},\ldots, out'_{m,l}$, where $l\leq s$ and $out'_{m,p}\leq out'_{m,q}$ for $1\leq p(2,4,5,3,1) +**Slot assignment via "Outside-in" sorting.** Ticket values $out'_{m,k}$ are assigned to slots using an outside-in ordering: -In the unlikely event that $K < s$, there will be some unassigned slots in the middle of the epoch, and for these we use AuRa. +- The lowest value $out'_{m,1}$ maps the last slot +- The second lowest $out'_{m,2}$ maps the first slot +- The third $out'_{m,3}$ maps the penultimate slot +- The fourth $out'_{m,4}$ maps to the second slot, and so on. -Concretely, for the algorithm for assiging lots that uses outside-in sorting, we take lists of even and odd numbered elements, reverse the list of odd elements, then concatenate the list of even elements, the list of aura slots and the reversed list of odd elements. +Example of outside-in ordering: Given the input (1,2,3,4,5), the resulting output is (2,4,5,3,1). + +In the unlikely event that $K < s$, some slots will remain unassigned in the middle of the epoch. These gaps are filled using the AuRa protocol. + +To assign slots using outside-in sorting, split the list of outputs into even- and odd-numbered elements, reverse the list of odd elements, then concatenate the even elements, followed by the Aura-assigned slots, and finally the reversed odd elements. ### 6) Claiming the slots -To produce a block in the assigned slot, the validator needs to include the ticket, a VRF output $out_{m,v,i}$, that corresponds to the slot together with a non-anonymous proof that this is the output of their VRF. +To produce a block in the assigned slot, a validator needs to include a ticket, specifically a VRF output $out_{m,v,i}$, corresponding to the slot, along with a non-anonymous proof that this output is the result of their VRF. + +Introducing the following functions facilitates this: -Thus we introduce $\texttt{Reveal}_{RVRF}: sk_v, out\mapsto \tau$ -and the corresponding + $\texttt{Check}_{RVRF}: \tau, out\mapsto true/false$ -calls that are basically Schnorr knowledge of exponent proofs (PoKE). -When validating the block nodes verify these proofs. -The validator must also include a never before seen VRF output, called the BABE VRF above. This may be done with the existing (non-jubjub) key on the same input (r_m || i). +These are esssentially Schnorr-style proofs of knowledge of exponent (PoKE). +When validating a block, nodes must verify these proofs. Additionally, the validator must include a previously unseen VRF output-referred to as the BABE VRF above, which can be generated using the existing (non-jubjub) key on the same input (r_m || i). -## Probabilities and parameters. +## Probabilities and parameters -The first parameter we consider is $x$. We need that there is a very small probability of their being less than $s$ winning tickets, even if up to $1/3$ of validators are offline. The probability of a ticket winning is $T=xs/a|V|$. -Let $n$ be the number of validators who actually participate and so $2|V|/3 \leq n \leq |V|$. These $n$ validators make $a$ attempts each for a total of $an$ attempts. -Let $X$ be the nimber of winning tickets. +The first parameter under consideration is $x$. The goal is to ensure a very low probability of having fewer than $s$ winning tickets, even if up to $1/3$ of validators are offline. The probability that any given attempt yields a winning ticket is $T=xs/a|V|$. +Let $n$ be the number of validators who actually participate such that $2|V|/3 \leq n \leq |V|$. Each of the $n$ validators makes $a$ attempts, resulting in a total of $an$ attempts. +Let $X$ be the number of winning tickets. Its expected value is -Then it's expectation has $E[X] = Tan = xsn/|V|$. If we set $x=2$, this is $\geq 4s/3$. In this case, $Var[X] = anT(1-T) \leq anT = xsn/|V| = 2sn/|V| \leq 2s$. +$$ +E[X] = Tan = xsn/|V| +$$ + +Setting $x=2$ yields $\geq 4s/3$. In this case, the variance is + +$$ +Var[X] = anT(1-T) \leq anT = xsn/|V| = 2sn/|V| \leq 2s +$$ Using Bernstein's inequality: $$ @@ -159,9 +190,11 @@ $$ \end{align*} $$ -For $s=600$, this gives under $4 * 10^{-13}$, which is certainly small enough. We only need the Aura fallback to deal with censorship. On the other hand, we couldn't make $x$ smaller than $3/2$ and still have tolerance against validators going offline. So $x=2$ is a sensible choice, and we should never need the Aura fallback. +For $s=600$, this yields a probability below $4 * 10^{-13}$, which is sufficiently small. The Aura fallback mechanism is needed only as a safeguard against censorship. It is not feasible to reduce $x$ below $3/2$ while retaining tolerance for offline validators, making $x=2$ a prudent choice. Under this configuration, the Aura fallback should remain unused. + +The next parameter to configure is $a$. A challenge arises in that if a validator $v$ receives $a$ winning tickets during an epoch, an adversary observing this will deduce that no additional blocks will be produced by $v$. -The next parameter we should set is $a$. The problem here is that if a validator $v$ gets $a$ winning tickets in an epoch, then when the adversary sees these, they now know that there will be no more blocks from $v$. +**For inquieries or questions, please contact** [Jeff Burdges](/team_members/JBurdges.md) diff --git a/docs/Polkadot/protocols/block-production/Sassafras-Part-1.md b/docs/Polkadot/protocols/block-production/Sassafras-Part-1.md deleted file mode 100644 index c8957508..00000000 --- a/docs/Polkadot/protocols/block-production/Sassafras-Part-1.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: 'Sassafras Part 1: A Novel Single Secret Leader Election Protocol' - ---- - -# Sassafras Part 1: A Novel Single Secret Leader Election Protocol - -Authors: Armando Caracheo, Elizabeth Crites, and Fatemeh Shirazi - -Polkadot is set to replace the [BABE](https://wiki.polkadot.network/docs/learn-consensus#block-production-babe)+[Aura](https://openethereum.github.io/Aura.html) consensus protocol with a new one: *Sassafras*. Sassafras will be used to generate blocks on Polkadot's relay chain, but can be used in other proof-of-stake (PoS) blockchains as well. So, what key advantages does this new protocol bring to the blockchain ecosystem? - -Imagine a bread factory where loaves are produced at random time intervals and multiple conveyor belts may release bread simultaneously, but only one loaf can be packaged at once. Clearly, this is not the most efficient way to process as much bread as possible. In a way, what happens in this factory resembles the current state of block production in Polkadot, as well as in other PoS blockchains. - -Classical proof-of-stake protocols for block generation create new blocks at unpredictable intervals, due to having multiple eligible block producers at once, or none at all. [BABE](https://wiki.polkadot.network/docs/learn-consensus#block-production-babe) similarly relies on randomized block generation and therefore inherits these traits. - -Just like in the bread factory, where the system could benefit from fixed-time production and a single conveyor belt, allowing loaves to be packaged one after another, an optimized block generation mechanism should rely on constant-time intervals and a single block producer per block. These improvements increase the number of loaves that can be produced, and analogously the number of blocks added to the chain. - -Production changes often introduce new challenges, and shifting to constant-time intervals within blockchain systems is no exception. Randomized block generation helps protect block producers from attacks, as adversaries are unable to predict the producer of the next block. For this reason, randomized block production has been considered a viable solution. - -But what if a protocol could ensure secrecy of block producers, protecting them from a looming adversary, while realizing non-randomized block generation to improve efficiency? Centered on the concept of *single secret leader election* (SSLE) (see [Part 2](https://hackmd.io/@W3F64sDIRkudVylsBHxi4Q/Bkr59i7ekg)), the Sassafras block producer selection protocol achieves exactly that. - -Curious to know how Sassafras works? Then keep reading. - -## Randomized block production: the root of multiple producers and empty blocks - -On Polkadot, [BABE](https://wiki.polkadot.network/docs/learn-consensus#block-production-babe) selects block producers through a process known as *probabilistic leader election* (PLE), which is common in many modern PoS blockchains. Since leader selection is based on a randomized algorithm, the following outcomes may occur: multiple block producers may be eligible to add a block to the chain, or no block producer may be eligible when the block is expected. - -Both outcomes have a negative impact on throughput (data processed per unit time), which is crucial to the scalability of blockchain consensus protocols. The absence of eligible block producers results in random time intervals between blocks. This can lead to a decrease in the overall throughput (i.e., fewer blocks in the final chain) and longer time intervals that lower the average block production rate. - -One way to mitigate low throughput is to simply reduce the time between produced blocks. Such a reduction, however, introduces the likelihood of certain risks. For instance, block producers who generate new blocks without first seeing the previous one may create forks. - -## Consistent timing from the core - -Constant-time block generation helps eliminate these potential risks and can be achieved by assigning a single block producer whenever a new block is expected. For example, in a fixed block producer rotation using a round-robin style, each validator takes turns and generates a block in a predefined order. [Aura](https://openethereum.github.io/Aura.html) is a good example of this block production mechanism. - -Unfortunately, this simple approach using a non-randomized algorithm for leader selection leaves the door open to attacks. If block producers are designated non-randomly and at fixed time intervals, the chain becomes vulnerable to forking and denial-of-service attacks. - -One way to counter these vulnerabilities is to implement an SSLE protocol. In the literature, there are several proposed SSLE protocols for block producer elections (see [Part 3](https://hackmd.io/I8VSv8c6Rfizi9JWmzX25w)). SSLE boasts many desirable features, which make it an attractive option for integration into PoS blockchains. However, existing protocols have various drawbacks, particularly in terms of efficiency. Indeed, none have been deployed to date. - -## Sassafras’s innovative approach - -Sassafras is a consensus protocol designed to randomly select the next block producer. Its main aim is to efficiently select a unique block producer and release blocks at constant time intervals. To achieve definite finality of blocks, the protocol may be combined with another protocol, such as [Grandpa](https://docs.polkadot.com/polkadot-protocol/architecture/polkadot-chain/pos-consensus/#). - -Sassafras operates as an SSLE protocol rather than a PLE protocol. Its novelty lies in using a [ring verifiable random function (VRF)](https://eprint.iacr.org/2023/002) to select a single block producer per block, while maintaining sufficient block producer secrecy within a “ring” of all participants. This design reduces on-chain communication and computation, enhances block production as well as leader selection efficiency, and conceals the identities of a sufficient number of honest block producers (enough to stay secure) until the moment they create blocks. - -We are now ready to describe the Sassafras protocol. - -## An overview of how Sassafras works, step-by-step - -At a high level, Sassafras works as follows. First, every validator generates a ticket, which they publish on-chain in a private manner. These tickets are sorted into a list. The first validator in that list reveals their identity and produces the next block. Tickets are kept private until the moment blocks are produced to protect validators from attacks. - -In terms of timing, during an epoch of the blockchain, blocks are generated according to the election results of the previous epoch, and new leaders are elected for the next epoch. - -The figure below illustrates the protocol in more detail. - -![sassafrass-diagram](https://hackmd.io/_uploads/BJaO6HF4ee.png) - -**Phase A)** Each validator generates a ticket $(y_i, \sigma_i)$ consisting of the ring VRF outputs, and encrypts it using the encryption key of a randomly chosen validator. This validator, called a repeater, acts as an identity guard. Each validator then sends their ciphertext to all validators. Both the ticket and its repeater remain hidden. - -**Phase B)** Each repeater receives all tickets and decrypts those for which it holds the decryption key. Repeaters then publish the tickets they have received on-chain. - -**Phase C)** All tickets are sorted and recorded on-chain. - -**Phase D)** When validator identities leak in Phase B, adversarial repeaters can decrypt their tickets (orange dots) and launch attacks (orange dots with a cross). The figure illustrates the worst-case scenario, in which all validators with leaked identities have been attacked. Even in this situation, the first honest validator on the sorted list (one who has not been attacked) is able to submit proof that they hold the winning ticket and become the next block producer. - -**Phase E)** The selected validator then generates the next block. - -With these five phases in place, and since all block producers for an epoch are determined in the previous epoch, blocks can be generated seamlessly in constant time (e.g., every 6 seconds on Polkadot). Moreover, Sassafras achieves the highest efficiency among SSLE protocols while maintaining sufficient anonymity to ensure the security of a blockchain that deploys it. - - -Striking this balance is what makes Sassafras an ideal candidate for real-world deployment. In fact, a specification of Sassafras, called Safrole, has already been integrated into the architecture of the upcoming [JAM protocol](https://graypaper.com/). - -## Eager to learn more about Sassafras? - -So far, we have presented a concise introduction to Sassafras, accessible to readers with a basic understanding of blockchains. We have also outlined the motivation behind the protocol and provided a clear overview of how it works. But this is only the beginning. The Web3 Foundation team has prepared two additional blog posts that explore Sassafras in greater detail. - -Our next blog post, [Part 2 - Deep Dive](https://hackmd.io/@W3F64sDIRkudVylsBHxi4Q/Bkr59i7ekg), will explain the concept of an SSLE protocol and delve into the technical details of Sassafras. The final chapter, [Part 3 - Compare and Convince](https://hackmd.io/I8VSv8c6Rfizi9JWmzX25w), will demonstrate how Sassafras achieves unparalleled efficiency in block production and provide readers with a comparison of similar protocols, highlighting its value. The upcoming blog posts aim to describe how Sassafras offers a practical solution for achieving better throughput while maintaining security. - -So stay tuned, brave reader. There's much more to discover in our upcoming Sassafras series, which is packed with valuable insights! diff --git a/docs/Polkadot/protocols/block-production/Sassafras.png b/docs/Polkadot/protocols/block-production/Sassafras.png new file mode 100644 index 00000000..3e8f6f94 Binary files /dev/null and b/docs/Polkadot/protocols/block-production/Sassafras.png differ diff --git a/docs/Polkadot/protocols/block-production/index.md b/docs/Polkadot/protocols/block-production/index.md index 046b1718..69a7d7ea 100644 --- a/docs/Polkadot/protocols/block-production/index.md +++ b/docs/Polkadot/protocols/block-production/index.md @@ -4,10 +4,19 @@ title: Block production import DocCardList from '@theme/DocCardList'; -The relay chain in Polkadot is built with the underlying proof-of-stake (POS) block production mechanism by validators. The currently deployed mechanism is a hybrid of BABE and Aura. We plan to replace BABE+Aura with Sassafras in the future. -**BABE:** A PoS protocol provides a way to elect validators to produce a block in the corresponding time slot. BABE's election is based on verifiable random function (VRF) of validators invented by David et al. for [Ouroboros Praos](https://eprint.iacr.org/2017/573.pdf) i.e., if a VRF output of a validator is less than a pre-defined threshold, then the validator is legitimate to produce a block. So, one validator or more than one validator or no validator can be elected. This election mechanism is completely private. In other words, no one can guess who is elected until the elected validator publishes a block. The privacy property is very critical for the blockchain security because it is indispensable for achieving security against an adaptive adversary who can corrupt any validator at any time. The drawback of this election mechanism is that no validator will be elected in a significant amount of time. So, validators waste these times by doing nothing which causes slightly worse (and uneven) throughput. Therefore, we fill the empty slots with blocks generated by validators who are deterministically selected by [Aura](https://eprint.iacr.org/2018/1079.pdf). Aura's election mechanism is not private so it is not secure against an adaptive adversary. For example, the adversary can prepare a DDOS attack on the elected validator by Aura to prevent him to publish his block because the adversary knows who is elected beforehand. Therefore, filling the empty slots with Aura blocks is not a solution in the adaptive adversarial model to prevent empty slots. Nevertheless we note that BABE+Aura is secure (safe and live) in the adaptive adversarial model - the security reduces to the BABE's security. It just does not prevent theoretically to have empty slots that we need to have a better throughput in the adaptive adversarial model. +The Polkadot relay chain, built by validators through a Proof-of-Stake (PoS) block production mechanism, operates with a hybrid system that combines BABE and Aura. The plan is to eventually replace this with Sassafras. -**Sassafras:** We construct Sassafras to obtain both security and non-empty slot property in the relay chain in the adaptive adversarial model. The election mechanism is based on the new primitive 'ring-VRF' that we define. Ring-VRF has the same properties as VRF e.g. randomness, verifiability of the output but also has verifiability property without knowing the public key of the validator who generated the output. In short, all validators generate a good amount of ring VRF outputs. Then, these outputs are sorted after verification by all validators to determine the order of the validators in the block production. Since a ring-VRF output does not reveal the identity of the validators for the verification, the privacy is preserved. Another good property of Sassafras is that there is only **one** validator is selected for a specified time interval. This property is useful to have fewer forks which is better for the parachain performance. +![](Block-production2.png) + +**BABE:** A Proof-of-Stake (PoS) protocol provides a mechanism for electing validators to produce blocks in designated time slots. BABE's election is based on a verifiable random function (VRF), originally introduced by David et al. in [Ouroboros Praos](https://eprint.iacr.org/2017/573.pdf). Specifically, a validator is elligible to produce a block if its VRF output falls below a pre-defined threshold. As a result, a round may yield one, multiple, or no elected validators. This election mechanism is completely private. Until an elected validator publishes a block, no one can know who was selected. Such privacy property is crucial for blockchain security, as it provides resilience against adaptive adversaries capable of corrupting validators at any time. + +One limitation of BABE's approach is that blocks may not be produced for extended periods if no validator meets the threshold. These empty slots degrade throughput and create uneven performance. To mitigate this, empty slots are filled using blocks generated by validators deterministically selected by [Aura](https://eprint.iacr.org/2018/1079.pdf). This ensures that the chain continues producing blocks during otherwise idle slots, though it comes with a tradeoff. + +Aura's selection process is not private, making it vulnerable to adaptive adversaries. For instance, knowing in advance which validator will be chosen allows an attacker to launch targeted denial-of-service (DDoS) attacks to block publication. Relying solely on Aura is unsuitable under adaptive adversarial conditions. Nevertheless, the BABE+Aura hybrid protocol remains secure and live under such model, since overall security relies on BABE. The only drawback is that empty slots are not prevented in theory, meaning throughput improvements are limited in the adaptive adversarial setting. + +**Sassafras:** Sassafras aims to achieve both security and non-empty slot property on the relay chain under an adaptive adversarial model. Its election mechanism is built on a novel primitive called 'ring-VRF.' Like standard VRFs, ring-VRF provides randomness and output verifiability, but it also allows verification without requiring knowledge of the public key of the validator who generated the output. + +In essence, all validators generate a sufficient number of ring VRF outputs. These outputs are then verified and sorted to determine the validator order for block production. Because ring-VRF outputs do not disclose the validator's identity during verification, the mechanism preserves privacy. Sassafras limits block production to a **single** validator per time interval. This approach reduces the likelihood of forks, thereby improving parachain performance. diff --git a/docs/Polkadot/protocols/finality.md b/docs/Polkadot/protocols/finality.md index eecb7c2f..ade6a332 100644 --- a/docs/Polkadot/protocols/finality.md +++ b/docs/Polkadot/protocols/finality.md @@ -4,20 +4,11 @@ title: Finality import useBaseUrl from '@docusaurus/useBaseUrl'; -Owner: [Alistair Stewart](/team_members/alistair.md) +![](Grandpa.png) -GRANDPA is the finality (consensus) algorithm for Polkadot. Here we first -present a high-level overview, as an "extended abstract". Details are presented -in the full paper directly below that. +GRANDPA is the finality (consensus) algorithm used in Polkadot. To get started, you can read our "extended abstract," which provides a high-level overview. If you're eager to dive deeper into the technical details, feel free to skip ahead to the full paper just below. And as a bonus, there is a more polished and slightly shorter version of the full paper available on [arxiv](https://arxiv.org/abs/2007.01560). -We also have an [alternative version](https://arxiv.org/abs/2007.01560) of the -full paper available on arxiv, which is more polished and a bit shorter. - -What is implemented in the Polkadot software and deployed in practise, we refer -to as "Polite GRANDPA" which includes optimisations required for efficient -real-world performance in practise. These are not covered in the papers below -for brevity, but we go into [the details](#polite-grandpa) later here on this -page. The high-level concepts and design principles remain the same as GRANDPA. +"Polite GRANDPA" is the implementation of GRANDPA used in the Polkadot software and deployed in practice. It includes optimizations tailored for efficient real-world performance in practice, which are not covered in the papers below for the sake of brevity, You can find the [details](#polite-grandpa) later on this page. The high-level concepts and design principles remain consistent with GRANDPA. ## GRANDPA Abstract paper