Skip to content

PeerDAS: Implement core. #15192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
May 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
cecfdf0
Fulu: Implement params.
nalepae Apr 18, 2025
9b0d9f6
KZG tests: Re-implement `getRandBlob` to avoid tests cyclical depende…
nalepae Apr 18, 2025
4d4abdc
Fulu testing util: Implement `GenerateCellsAndProofs`.
nalepae Apr 18, 2025
531234c
Create `RODataColumn`.
nalepae Apr 18, 2025
bb440e6
Implement `MerkleProofKZGCommitments`.
nalepae Apr 18, 2025
8469aca
Export `leavesFromCommitments`.
nalepae Apr 18, 2025
de65d87
Implement peerDAS core.
nalepae Apr 18, 2025
d7154e1
Add changelog.
nalepae Apr 18, 2025
6967090
Update beacon-chain/core/peerdas/das_core.go
nalepae Apr 26, 2025
43c0982
Fix Terence's comment: Use `IsNil`.
nalepae Apr 26, 2025
9507b1d
Fix Terence's comment: Avoid useless `filteredIndices`.
nalepae Apr 26, 2025
4447c0f
Fix Terence's comment: Simplify odd/even cases.
nalepae Apr 26, 2025
702293d
Fix Terence's comment: Use `IsNil`.
nalepae Apr 26, 2025
c08366d
Spectests: Add Fulu networking
nalepae Apr 27, 2025
48f6ef0
Fix Terence's comment: `CustodyGroups`: Stick to the spec by returnin…
nalepae Apr 27, 2025
e3ea07e
Fix Terence's comment: `CustodyGroups`: Handle correctly the `maxUint…
nalepae Apr 27, 2025
9980462
Update beacon-chain/core/peerdas/das_core.go
nalepae Apr 27, 2025
6fcda3d
Fix Terence's comment: `ComputeColumnsForCustodyGroup`: Add test if `…
nalepae Apr 27, 2025
b6900b6
`CustodyGroups`: Test if `custodyGroupCount > numberOfCustodyGroup`.
nalepae Apr 27, 2025
6058f19
`CustodyGroups`: Add a shortcut if all custody groups are needed.
nalepae Apr 27, 2025
648e14b
`ComputeCystodyGroupForColumn`: Move from `p2p_interface.go` to `das_…
nalepae Apr 28, 2025
6bc9776
Fix Terence's comment: Fix `ComputeCustodyGroupForColumn`.
nalepae Apr 28, 2025
6d423cb
Fix Terence's comment: Remove `constructCellsAndProofs` function.
nalepae Apr 28, 2025
afe2652
Fix Terence's comment: `ValidatorsCustodyRequirement`: Use effective …
nalepae Apr 28, 2025
0cce37e
`MerkleProofKZGCommitments`: Add tests
nalepae Apr 28, 2025
2543120
Remove peer sampling.
nalepae Apr 29, 2025
1d8fb51
`DataColumnSidecars`: Add missing tests.
nalepae May 5, 2025
ea5e0e8
Fix Jame's comment.
nalepae May 5, 2025
8dac003
Fix James' comment.
nalepae May 5, 2025
c87701e
Fix James' comment.
nalepae May 5, 2025
2d4d8d1
Fix James' coment.
nalepae May 5, 2025
182b5d7
Fix James' comment.
nalepae May 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion beacon-chain/blockchain/kzg/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ go_test(
deps = [
"//consensus-types/blocks:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_consensys_gnark_crypto//ecc/bls12-381/fr:go_default_library",
"@com_github_crate_crypto_go_kzg_4844//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)
41 changes: 39 additions & 2 deletions beacon-chain/blockchain/kzg/validation_test.go
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
package kzg

import (
"bytes"
"crypto/sha256"
"encoding/binary"
"testing"

"github.com/OffchainLabs/prysm/v6/consensus-types/blocks"
"github.com/OffchainLabs/prysm/v6/testing/require"
"github.com/OffchainLabs/prysm/v6/testing/util"
"github.com/consensys/gnark-crypto/ecc/bls12-381/fr"
GoKZG "github.com/crate-crypto/go-kzg-4844"
"github.com/sirupsen/logrus"
)

func GenerateCommitmentAndProof(blob GoKZG.Blob) (GoKZG.KZGCommitment, GoKZG.KZGProof, error) {
Expand Down Expand Up @@ -37,11 +41,44 @@ func TestBytesToAny(t *testing.T) {
}

func TestGenerateCommitmentAndProof(t *testing.T) {
blob := util.GetRandBlob(123)
blob := getRandBlob(123)
commitment, proof, err := GenerateCommitmentAndProof(blob)
require.NoError(t, err)
expectedCommitment := GoKZG.KZGCommitment{180, 218, 156, 194, 59, 20, 10, 189, 186, 254, 132, 93, 7, 127, 104, 172, 238, 240, 237, 70, 83, 89, 1, 152, 99, 0, 165, 65, 143, 62, 20, 215, 230, 14, 205, 95, 28, 245, 54, 25, 160, 16, 178, 31, 232, 207, 38, 85}
expectedProof := GoKZG.KZGProof{128, 110, 116, 170, 56, 111, 126, 87, 229, 234, 211, 42, 110, 150, 129, 206, 73, 142, 167, 243, 90, 149, 240, 240, 236, 204, 143, 182, 229, 249, 81, 27, 153, 171, 83, 70, 144, 250, 42, 1, 188, 215, 71, 235, 30, 7, 175, 86}
require.Equal(t, expectedCommitment, commitment)
require.Equal(t, expectedProof, proof)
}

func deterministicRandomness(seed int64) [32]byte {
// Converts an int64 to a byte slice
buf := new(bytes.Buffer)
err := binary.Write(buf, binary.BigEndian, seed)
if err != nil {
logrus.WithError(err).Error("Failed to write int64 to bytes buffer")
return [32]byte{}
}
bytes := buf.Bytes()

return sha256.Sum256(bytes)
}

// Returns a serialized random field element in big-endian
func getRandFieldElement(seed int64) [32]byte {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two identical implementations, could we reuse?

func GetRandFieldElement(seed int64) [32]byte {

func getRandFieldElement(seed int64) [32]byte {

Copy link
Contributor Author

@nalepae nalepae Apr 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is explained in the corresponding commit message: 972a7a6

Aka, for this test only code, duplicating, while not ideal, is, IMO, better than trying to solve the cyclical dependencies.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm I think we might want to fix this dependency. We should be able to import testing/util anywhere so there is a bad design issue outside of this PR. Someone else will probably run into a similar problem. It is not clear to me what the exact cause is. We can postpone this one.

compilepkg: dependency cycle detected between "github.com/OffchainLabs/prysm/v6/beacon-chain/blockchain/kzg" and "github.com/OffchainLabs/prysm/v6/testing/util" in file "/private/var/tmp/_bazel_t/3a219d5e47df5947c1056b3e139abb7c/sandbox/darwin-sandbox/627/execroot/_main/beacon-chain/blockchain/kzg/validation_test.go"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I created this ticket: #15231 for book keeping.

bytes := deterministicRandomness(seed)
var r fr.Element
r.SetBytes(bytes[:])

return GoKZG.SerializeScalar(r)
}

// Returns a random blob using the passed seed as entropy
func getRandBlob(seed int64) GoKZG.Blob {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, the implementations are duplicated in deneb.go and util_test.go. We should find a way to consolidate them to avoid copy-pasting each time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is explained in the corresponding commit message: 972a7a6

Aka, for this test only code, duplicating, while not ideal, is, IMO, better than trying to solve the cyclical dependencies.

var blob GoKZG.Blob
bytesPerBlob := GoKZG.ScalarsPerBlob * GoKZG.SerializedScalarSize
for i := 0; i < bytesPerBlob; i += GoKZG.SerializedScalarSize {
fieldElementBytes := getRandFieldElement(seed + int64(i))
copy(blob[i:i+GoKZG.SerializedScalarSize], fieldElementBytes[:])
}
return blob
}
71 changes: 71 additions & 0 deletions beacon-chain/core/peerdas/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = [
"das_core.go",
"info.go",
"metrics.go",
"p2p_interface.go",
"reconstruction.go",
"util.go",
"validator.go",
],
importpath = "github.com/OffchainLabs/prysm/v6/beacon-chain/core/peerdas",
visibility = ["//visibility:public"],
deps = [
"//beacon-chain/blockchain/kzg:go_default_library",
"//beacon-chain/state:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//container/trie:go_default_library",
"//crypto/hash:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
"@com_github_ethereum_go_ethereum//p2p/enr:go_default_library",
"@com_github_hashicorp_golang_lru//:go_default_library",
"@com_github_holiman_uint256//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prometheus_client_golang//prometheus:go_default_library",
"@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
"@org_golang_x_sync//errgroup:go_default_library",
],
)

go_test(
name = "go_default_test",
srcs = [
"das_core_test.go",
"info_test.go",
"p2p_interface_test.go",
"reconstruction_test.go",
"utils_test.go",
"validator_test.go",
],
deps = [
":go_default_library",
"//beacon-chain/blockchain/kzg:go_default_library",
"//beacon-chain/state/state-native:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/primitives:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"@com_github_consensys_gnark_crypto//ecc/bls12-381/fr:go_default_library",
"@com_github_crate_crypto_go_kzg_4844//:go_default_library",
"@com_github_ethereum_go_ethereum//p2p/enode:go_default_library",
"@com_github_ethereum_go_ethereum//p2p/enr:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)
Loading
Loading