[WIP] Optimize Peras weight calculation #1613

amesgen · 2025-07-29T16:52:26Z

Not a priority for now, but opening a PR to have a reference

Relies on the fact that boosted points all lie on the same chain (which is not true across cooldowns, so this requires special care)

amesgen · 2025-07-29T16:53:05Z

  weightBoostOfFragment of length 0:    OK
    20.0 ns ± 2.0 ns,  31 B  allocated,   0 B  copied,  39 MB peak memory,       same as baseline
  weightBoostOfFragment of length 100:  OK
    4.46 μs ± 118 ns, 9.5 KB allocated,   0 B  copied,  39 MB peak memory, 56% less than baseline
  weightBoostOfFragment of length 200:  OK
    4.54 μs ± 184 ns, 9.6 KB allocated,   0 B  copied,  39 MB peak memory, 79% less than baseline
  weightBoostOfFragment of length 300:  OK
    4.32 μs ± 357 ns, 8.0 KB allocated,   0 B  copied,  39 MB peak memory, 88% less than baseline
  weightBoostOfFragment of length 400:  OK
    4.74 μs ± 473 ns, 8.9 KB allocated,   0 B  copied,  39 MB peak memory, 92% less than baseline
  weightBoostOfFragment of length 500:  OK
    4.19 μs ± 2.6 μs, 9.1 KB allocated,   0 B  copied,  39 MB peak memory, 95% less than baseline
  weightBoostOfFragment of length 600:  OK
    3.27 μs ± 502 ns, 8.3 KB allocated,   0 B  copied,  39 MB peak memory, 97% less than baseline
  weightBoostOfFragment of length 700:  OK
    4.03 μs ± 185 ns, 9.2 KB allocated,   0 B  copied,  39 MB peak memory, 97% less than baseline
  weightBoostOfFragment of length 800:  OK
    4.25 μs ± 377 ns, 9.4 KB allocated,   0 B  copied,  39 MB peak memory, 97% less than baseline
  weightBoostOfFragment of length 900:  OK
    4.02 μs ± 378 ns, 8.6 KB allocated,   0 B  copied,  39 MB peak memory, 97% less than baseline
  weightBoostOfFragment of length 1000: OK
    4.08 μs ± 403 ns, 9.5 KB allocated,   0 B  copied,  39 MB peak memory, 97% less than baseline
  weightBoostOfFragment of length 1100: OK
    3.97 μs ± 376 ns, 9.6 KB allocated,   0 B  copied,  39 MB peak memory, 97% less than baseline
  weightBoostOfFragment of length 1200: OK
    3.12 μs ± 291 ns, 8.0 KB allocated,   0 B  copied,  39 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1300: OK
    2.95 μs ± 292 ns, 8.9 KB allocated,   0 B  copied,  39 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1400: OK
    2.95 μs ± 255 ns, 9.1 KB allocated,   0 B  copied,  39 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1500: OK
    2.89 μs ± 170 ns, 8.3 KB allocated,   0 B  copied,  39 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1600: OK
    3.31 μs ± 284 ns, 9.2 KB allocated,   0 B  copied,  39 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1700: OK
    3.24 μs ± 145 ns, 9.4 KB allocated,   0 B  copied,  40 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1800: OK
    3.11 μs ± 304 ns, 8.6 KB allocated,   0 B  copied,  40 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 1900: OK
    3.37 μs ± 189 ns, 9.5 KB allocated,   0 B  copied,  40 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 2000: OK
    3.49 μs ± 338 ns, 9.7 KB allocated,   0 B  copied,  40 MB peak memory, 98% less than baseline
  weightBoostOfFragment of length 2100: OK
    2.74 μs ± 131 ns, 8.0 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2200: OK
    2.93 μs ±  86 ns, 8.9 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2300: OK
    3.09 μs ± 266 ns, 9.1 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2400: OK
    3.02 μs ± 223 ns, 8.3 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2500: OK
    3.28 μs ± 150 ns, 9.2 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2600: OK
    3.29 μs ± 200 ns, 9.4 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2700: OK
    3.13 μs ± 183 ns, 8.6 KB allocated,   0 B  copied,  40 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2800: OK
    3.24 μs ± 239 ns, 9.5 KB allocated,   0 B  copied,  41 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 2900: OK
    3.28 μs ± 246 ns, 9.6 KB allocated,   0 B  copied,  41 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3000: OK
    2.70 μs ± 197 ns, 8.0 KB allocated,   0 B  copied,  41 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3100: OK
    2.97 μs ± 271 ns, 8.9 KB allocated,   0 B  copied,  41 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3200: OK
    3.00 μs ± 204 ns, 9.1 KB allocated,   0 B  copied,  41 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3300: OK
    2.91 μs ± 242 ns, 8.3 KB allocated,   0 B  copied,  41 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3400: OK
    3.18 μs ± 179 ns, 9.2 KB allocated,   0 B  copied,  42 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3500: OK
    3.16 μs ± 209 ns, 9.4 KB allocated,   0 B  copied,  42 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3600: OK
    3.01 μs ± 165 ns, 8.6 KB allocated,   0 B  copied,  42 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3700: OK
    3.21 μs ±  82 ns, 9.5 KB allocated,   0 B  copied,  42 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3800: OK
    3.35 μs ± 262 ns, 9.6 KB allocated,   0 B  copied,  42 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 3900: OK
    2.75 μs ± 254 ns, 8.0 KB allocated,   0 B  copied,  42 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4000: OK
    3.32 μs ± 307 ns, 8.9 KB allocated,   0 B  copied,  43 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4100: OK
    3.63 μs ± 194 ns, 9.1 KB allocated,   0 B  copied,  43 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4200: OK
    3.40 μs ± 606 ns, 8.3 KB allocated,   0 B  copied,  43 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4300: OK
    4.17 μs ± 416 ns, 9.2 KB allocated,   0 B  copied,  43 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4400: OK
    4.50 μs ± 418 ns, 9.4 KB allocated,   0 B  copied,  43 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4500: OK
    4.35 μs ± 426 ns, 8.6 KB allocated,   0 B  copied,  43 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4600: OK
    4.97 μs ± 343 ns, 9.5 KB allocated,   0 B  copied,  44 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4700: OK
    4.08 μs ± 1.3 μs, 9.6 KB allocated,   0 B  copied,  44 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4800: OK
    3.47 μs ± 210 ns, 8.0 KB allocated,   0 B  copied,  44 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 4900: OK
    3.96 μs ± 356 ns, 8.9 KB allocated,   0 B  copied,  44 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5000: OK
    4.20 μs ± 722 ns, 9.1 KB allocated,   0 B  copied,  45 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5100: OK
    3.38 μs ± 210 ns, 8.3 KB allocated,   0 B  copied,  45 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5200: OK
    3.18 μs ± 381 ns, 9.2 KB allocated,   0 B  copied,  45 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5300: OK
    3.13 μs ± 214 ns, 9.4 KB allocated,   0 B  copied,  45 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5400: OK
    3.02 μs ± 181 ns, 8.6 KB allocated,   0 B  copied,  46 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5500: OK
    3.23 μs ± 168 ns, 9.5 KB allocated,   0 B  copied,  46 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5600: OK
    3.32 μs ± 266 ns, 9.6 KB allocated,   0 B  copied,  46 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5700: OK
    2.67 μs ± 145 ns, 8.0 KB allocated,   0 B  copied,  46 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5800: OK
    2.93 μs ± 237 ns, 8.9 KB allocated,   0 B  copied,  47 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 5900: OK
    3.53 μs ± 234 ns, 9.1 KB allocated,   0 B  copied,  47 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6000: OK
    2.77 μs ± 322 ns, 8.3 KB allocated,   0 B  copied,  47 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6100: OK
    3.03 μs ± 214 ns, 9.2 KB allocated,   0 B  copied,  47 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6200: OK
    3.04 μs ± 241 ns, 9.4 KB allocated,   0 B  copied,  48 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6300: OK
    3.02 μs ± 264 ns, 8.6 KB allocated,   0 B  copied,  48 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6400: OK
    3.17 μs ± 256 ns, 9.5 KB allocated,   0 B  copied,  48 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6500: OK
    3.33 μs ± 203 ns, 9.6 KB allocated,   0 B  copied,  49 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6600: OK
    2.69 μs ± 195 ns, 8.0 KB allocated,   0 B  copied,  50 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6700: OK
    1.99 μs ± 172 ns, 8.9 KB allocated,   0 B  copied,  50 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6800: OK
    2.01 μs ± 189 ns, 9.1 KB allocated,   0 B  copied,  51 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 6900: OK
    2.01 μs ± 188 ns, 8.3 KB allocated,   0 B  copied,  51 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7000: OK
    2.17 μs ± 208 ns, 9.2 KB allocated,   0 B  copied,  51 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7100: OK
    2.13 μs ± 206 ns, 9.4 KB allocated,   0 B  copied,  52 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7200: OK
    2.10 μs ± 168 ns, 8.6 KB allocated,   0 B  copied,  52 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7300: OK
    2.28 μs ± 167 ns, 9.5 KB allocated,   0 B  copied,  52 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7400: OK
    2.34 μs ± 182 ns, 9.6 KB allocated,   0 B  copied,  53 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7500: OK
    1.90 μs ± 174 ns, 8.0 KB allocated,   0 B  copied,  53 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7600: OK
    2.01 μs ± 297 ns, 8.9 KB allocated,   0 B  copied,  53 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7700: OK
    2.31 μs ±  92 ns, 9.1 KB allocated,   0 B  copied,  54 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7800: OK
    2.37 μs ± 125 ns, 8.3 KB allocated,   0 B  copied,  54 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 7900: OK
    2.62 μs ± 174 ns, 9.2 KB allocated,   0 B  copied,  54 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8000: OK
    2.32 μs ± 232 ns, 9.4 KB allocated,   0 B  copied,  55 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8100: OK
    2.14 μs ± 145 ns, 8.6 KB allocated,   0 B  copied,  55 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8200: OK
    2.27 μs ± 125 ns, 9.5 KB allocated,   0 B  copied,  55 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8300: OK
    2.34 μs ±  95 ns, 9.6 KB allocated,   0 B  copied,  56 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8400: OK
    1.90 μs ± 135 ns, 8.0 KB allocated,   0 B  copied,  56 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8500: OK
    2.17 μs ± 190 ns, 8.9 KB allocated,   0 B  copied,  57 MB peak memory, 99% less than baseline
  weightBoostOfFragment of length 8600: OK
    2.13 μs ± 175 ns, 9.1 KB allocated,   0 B  copied,  57 MB peak memory, 99% less than baseline

For the largest fragment, we have a reduction from 1.25 ms to 2.13 μs.

This is preparing for Peras, which will introduce a refined immutability criterion. Due to this change, we defer the immutability criterion to the implementation of `getCurrentChain`, reducing the impact of Peras.

This is a preparatory change for Peras, which uses a different immutability criterion (based on weight of chains, instead of (just) length). The only remaining use of the `SecurityParam` is for the snapshot policy 🙃

This requires some knot tying as we initialize the LedgerDB before the ChainDB is fully initialized. This is preparing for Peras, which will introduce a refined immutability criterion. Due to this change, the LedgerDB will automatically use the same criterion as the ChainDB (via `getCurrentChain`).

In case of data loss in the VolatileDB and a node restart, this condition can be violated until we received `k` headers again. Additionally, Ouroboros Peras will use a different immutability criterion where it is perfectly fine to have less than `k` headers on our chain (as long as they have sufficient weight).

- Avoid orphans - We actually can't check that the open states are consistent directly as we would need to statefully get that info from the SUT.

It makes sense to use this without using an entire PerasCertDB, so decouple these. It might be nice to rename PerasCertDB-bench, but doesn't seem like a priority.

using cabal-docspec (not yet nixified)

This is purely for concise QuickCheck counterexample output.

This is in preparation for weighted chain comparisons.

Also remove the version for `ValidatedChainDiff` as it is unused.

amesgen added the Peras label Jul 29, 2025

amesgen force-pushed the amesgen/opt-weight-snapshot branch 4 times, most recently from c13b020 to 9544349 Compare July 30, 2025 09:08

amesgen mentioned this pull request Aug 5, 2025

O.C.Peras.Weight: add totalWeightForFragment/takeVolatileSuffix #1621

Merged

amesgen force-pushed the amesgen/opt-weight-snapshot branch from 9544349 to 4f232eb Compare August 8, 2025 08:13

amesgen and others added 22 commits August 13, 2025 17:28

ChainSel: olderThanK -> olderThanImmTip

7977e6d

ChainDB.Background: avoid hardcoding immutability criterion

7b95734

This is preparing for Peras, which will introduce a refined immutability criterion. Due to this change, we defer the immutability criterion to the implementation of `getCurrentChain`, reducing the impact of Peras.

LedgerDB: abstract out immutability criterion

c66b0aa

This is a preparatory change for Peras, which uses a different immutability criterion (based on weight of chains, instead of (just) length). The only remaining use of the `SecurityParam` is for the snapshot policy 🙃

Scaffolding for Peras certs and PerasCertDB

2ce3053

[WIP] set structure for model-based testing for PerasCertDB

5246a54

Fix missing instances

aa94e92

Pairing

5fc441d

Minor polishing

b5cea47

- Avoid orphans - We actually can't check that the open states are consistent directly as we would need to statefully get that info from the SUT.

PerasCertDB: implement garbage collection

5a58067

Mention ChainSync Client benchmark

45b3add

ouroboros-consensus: add Peras chain weight benchmark

046e0ff

ChainDB: expose PerasCertDB functionality

1380783

ChainDB: invoke PerasCertDB GC

5eca085

Move PerasWeightSnapshot to separate module

f727c6c

It makes sense to use this without using an entire PerasCertDB, so decouple these. It might be nice to rename PerasCertDB-bench, but doesn't seem like a priority.

PerasCertDB.getWeightSnapshot: add Fingerprint

9231e68

PerasCertDB.addCert: return whether we added the cert

395381d

Peras: minor tweaks

04f48fb

Make PerasWeightSnapshot opaque

f919246

Nomenclature: "weight boost" instead of "boosted weight"

bee6981

Glossary: add Peras weight-related terms

a5ad9ed

amesgen added 17 commits August 13, 2025 17:39

PerasWeightSnapshot: minimize API

1cfcd95

PerasRoundNo/PerasWeight: terse output

0a217b9

O.C.Peras.Weight: add haddocks

e975d0d

using cabal-docspec (not yet nixified)

SecurityParam: mention weighted nature

c1df482

PerasRoundNo/PerasWeight: add Condense instances

6b44e97

This is purely for concise QuickCheck counterexample output.

O.C.Peras.Weight: add totalWeightForFragment/takeVolatileSuffix

6fd217e

Add test for PerasWeightSnapshot

dd53c36

ChainDB.StateMachine: check immutable tip monotonicity

3fbf549

ChainDB: define getCurrentChain in terms of weight

2e865cd

GSM: allow candidateOverSelection to be stateful

f8a0c06

This is in preparation for weighted chain comparisons.

Add WeightedSelectView

25512c2

ChainSel: make rollbackExceedsSuffix weight-aware

f4f27b6

Also remove the version for `ValidatedChainDiff` as it is unused.

Introduce weighted chain comparisons

f95a485

Integrate weighted BlockFetch decision logic

6c4caed

ChainDB: implement chain selection for certificates

7b76810

MockChainSel: switch to weighted chain selection

7df2c92

ChainDB q-s-m: test weighted chain selection

8a299a9

amesgen force-pushed the peras-staging branch from f994dc8 to 8a299a9 Compare August 13, 2025 15:44

amesgen added 3 commits August 13, 2025 18:53

Fix cabal-docspec

c971d39

PerasCertDB.StateMachine: generate chain-like boosted points

3244b19

Optimize PerasWeightSnapshot

18c93d4

amesgen force-pushed the amesgen/opt-weight-snapshot branch from 4f232eb to 18c93d4 Compare August 13, 2025 16:53

amesgen changed the title ~~Optimize Peras weight calculation~~ [WIP] Optimize Peras weight calculation Sep 1, 2025

amesgen force-pushed the peras-staging branch from 5ba50d1 to 36d8f01 Compare September 9, 2025 11:27

dnadales mentioned this pull request Sep 19, 2025

[Peras #1] Introduce basic types to support Peras #1673

Open

amesgen mentioned this pull request Sep 23, 2025

Choose "realistic" boosted blocks in tests tweag/cardano-peras#124

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] Optimize Peras weight calculation #1613

[WIP] Optimize Peras weight calculation #1613

Uh oh!

amesgen commented Jul 29, 2025 •

edited

Loading

Uh oh!

amesgen commented Jul 29, 2025

Uh oh!

Uh oh!

[WIP] Optimize Peras weight calculation #1613

Are you sure you want to change the base?

[WIP] Optimize Peras weight calculation #1613

Uh oh!

Conversation

amesgen commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amesgen commented Jul 29, 2025

Uh oh!

Uh oh!

amesgen commented Jul 29, 2025 •

edited

Loading