Testnet difficulty adjustment: future-dated minimum-difficulty blocks crash difficulty (~12.7 s block time)

[daira]

Summary

On Testnet, the ZIP 205 minimum-difficulty rule interacts with difficulty adjustment so that difficulty is pinned far below equilibrium: measured mean block time is around 12.7 s against the 75 s target (around 5.9× too fast). Two things combine:

(a) the minimum-difficulty trigger is driven almost entirely by future-dated timestamps, not real pauses; and
(b) because the averaging is over targets, each such block crashes difficulty rather than nudging it.

This makes Testnet block production erratic, and the estimation of NU activation heights hard.

Testnet-only (the minimum-difficulty rule does not apply to Mainnet), but it degrades Testnet usability.

Measured evidence (last 48 h of Testnet, 13650 blocks, heights 4105654–4119303)

• mean block time 12.67 s (median gap 7 s; p90 = 22 s; p99 = 124 s) vs 75 s target → 5.92× too fast.
• 41 minimum-difficulty blocks (0.30%). Classified: 40 future-dated timestamp artifacts, 0 real pauses, 1 ambiguous. The artifacts have gap_prev ≈ 451 s (just over the 450 s threshold), gap_next negative (~−130 s, i.e. the next block's nTime steps back), and sit ~475–950 s ahead of median-time-past.
• The near-constant gap_prev = 451 s suggests these blocks are future-dated just past the threshold (for an unknown reason), resulting in minimum-difficulty (cheap) blocks.
• 223 blocks are timestamped > 450 s ahead of median-time-past (max future-skew 949 s; validity bound is MTP + 90 min = 5400 s) — each such block can create a spurious trigger.
• Difficulty sawtooths: min 1, median 242, max 2.28e6 (equilibrium for 75 s would be ~1400), pinned at 1 for the 41 minimum-difficulty blocks.

Note the count (41) is only ~1.2× the naive Poisson-tail estimate e^(−6)·13650 ≈ 34, so a count comparison alone looks unremarkable — but the classification shows these are not steady-state tail events at all; they are timestamp artifacts. That is why "the tail of the Poisson distribution" does not explain the behaviour.

Mechanism

From the Protocol Specification "Difficulty adjustment", with ZIP 205 and ZIP 208 (post-Blossom threshold):

• A block N is a minimum-difficulty block iff nTime(N) − nTime(N−1) > 6 × PoWTargetSpacing (post-Blossom = 450 s), using raw consecutive timestamps (not median-time-past), which need only satisfy nTime > MTP and nTime ≤ MTP + 90 min and need not be monotonic. Its nBits is set to ToCompact(PoWLimit) (Testnet PoWLimit = 2^251 − 1).
• MeanTarget(N) is the arithmetic mean of ToTarget(nBits(i)) over the trailing PoWAveragingWindow = 17 blocks, including minimum-difficulty blocks.

Because the average is over targets, one minimum-difficulty block dominates the mean. With 16 normal targets R and one PoWLimit:

MeanTarget ≈ (16·R + PoWLimit) / 17 ≈ PoWLimit / 17     (since PoWLimit ≫ R)

so the next target jumps from ≈ R to ≈ PoWLimit/17, and since difficulty ∝ 1/target, difficulty collapses from D to ≈ 17 (≈ PoWAveragingWindow) in one block — a factor (16 + D)/17 ≈ D/17. This is specific to averaging targets: because PoWLimit is ~D× a normal target, a single minimum-difficulty entry dominates the arithmetic mean. Averaging over difficulties or log-targets, or bounding the per-block change of MeanTarget, would avoid it.

The per-block ±16% / −32% limit does not prevent this: it bounds only the ActualTimespanBounded (median-time) factor, not the MeanTarget change. The minimum-difficulty block then persists in the window for 17 blocks, and recovery is bounded by the 16%-per-block factor, so returning to equilibrium D takes ~17 + log₁.₁₉(D/17) ≈ tens of uninterrupted blocks — during which the chain runs fast.

Proposed direction

To be specified in draft-valargroup-blocktime-reduction.md:

• Trigger side: the trigger on raw consecutive nTime, with a 90-minute future allowance and no monotonicity requirement, is trivially exploitable (or can be triggered accidentally) to cause minimum-difficulty blocks. Consider triggering on median-time-past rather than raw nTime, and/or tightening the future-timestamp allowance, and/or requiring the gap to be robust to non-monotonic timestamps.
• Averaging side: so that a trigger does not crash difficulty — exclude minimum-difficulty blocks' nBits from MeanTarget (e.g. carry forward the last non-minimum-difficulty target, cf. Bitcoin testnet's reset-to-prior behaviour), and/or average over difficulties / log-targets, and/or bound the per-block MeanTarget change.

🤖 Claude Opus 4.8

[linear]

COR-1473

[daira]

A note on where in block construction the timestamps come to land just past the 450 s threshold (as opposed to the consensus check, which only reads nTime):

• Consensus check (matches the spec / ZIP 205 / ZIP 208): a block is minimum-difficulty iff its raw nTime exceeds prevBlock.nTime + 6 × PoWTargetSpacing. In zcashd this is in pow.cpp GetNextWorkRequired: pblock->GetBlockTime() > pindexLast->GetBlockTime() + PoWTargetSpacing(...) * 6.
• Block construction in zcashd sets nTime = max(medianTimePast + 1, GetTime()) (real local time) in UpdateTime, then recomputes nBits. So a stock zcashd miner only produces a minimum-difficulty block when real time has actually advanced > 450 s since the previous block — in which case the following block's timestamp would step forward, not back. I did not find any zcashd path that pushes nTime forward to just clear the threshold.
• In the measured data the following block's nTime steps back (gap_next < 0), i.e. the minimum-difficulty block is timestamped ahead of its real arrival. So the timestamp is being advanced by the block producer, not by a genuine gap.

The dominant miner on Testnet runs zebra, whose block-construction / minimum-difficulty timestamp handling may differ from zcashd's, so the exact source of the forward-dating should be confirmed there rather than assumed from zcashd.

Either way this does not change the consensus-level conclusion: the rule reads raw consecutive nTime (which need only exceed median-time-past, may be up to MTP + 90 min ahead, and need not be monotonic), so it can be satisfied by a forward-dated timestamp regardless of real block timing. That is why the fix directions above — trigger on median-time-past / tighten the future allowance, and prevent a single trigger from crashing the average — are the right level to address it.

🤖 Claude Opus 4.8

[daira]

Followed this into zebra (the dominant Testnet miner runs zebra). The "unknown reason" is a Zebra-specific getblocktemplate rule, not present in zcashd.

In zebra-state/src/service/read/difficulty.rs, when building a Testnet block template Zebra applies a Zebra-specific rule (code comment: "If a miner is likely to find a block with the cur_time and standard difficulty within a target block interval or two, keep the original difficulty. Otherwise, try to use the minimum difficulty."):

• EXTRA_TIME_TO_MINE_A_BLOCK = 2 × PoWTargetSpacing = 150 s
• std_difficulty_max_time = previous_block_time + 6 × PoWTargetSpacing = previous_block_time + 450 s
• local_std_difficulty_limit = std_difficulty_max_time − 150 = previous_block_time + 300 s
• if cur_time ≤ previous_block_time + 300 → standard difficulty (max_time clamped down to prev + 450, keeping the template standard-difficulty);
• else (cur_time > previous_block_time + 300) → minimum difficulty: min_time is set to previous_block_time + 451, cur_time is clamped up to min_time, and the template difficulty is recomputed at that time.

So once the current time reaches ~300 s after the previous block's timestamp — 150 s before the 450 s consensus threshold — Zebra proactively hands out a minimum-difficulty template with cur_time = min_time = previous_block_time + 451. Since min_time is a floor on the header nTime, any conforming miner is forced to stamp nTime ≥ prev + 451, ahead of the real time at which the switch happened. That is exactly the future-dated, gap_prev ≈ 451, gap_next < 0 pattern in the data.

zcashd has no equivalent: it only marks a block minimum-difficulty when the real timestamp already exceeds the threshold (UpdateTime sets nTime = max(MTP + 1, real time)), so a stock zcashd miner would only do this after a genuine > 450 s gap.

This closes the loop: Zebra's template heuristic generates the future-dated minimum-difficulty blocks, and each one then crashes the difficulty via the target-space averaging described above. Two independent levers therefore help:

• the averaging fix (so a minimum-difficulty block does not crash the mean) — the robust consensus-level change; and
• separately, Zebra's 150 s "give up on standard difficulty" margin makes it emit these blocks quite eagerly, which is worth revisiting on its own.

🤖 Claude Opus 4.8

[upbqdn]

Two additions not yet in the thread.

1. Consensus-level, and identical in zcashd, Zebra, and the spec.

Let a block's target be the expanded nBits, and its difficulty be D := PoWLimit / target (so a minimum-difficulty block, target = PoWLimit, has D = 1). All three implementations compute MeanTarget as the arithmetic mean of the targets of the 17 preceding blocks — not of their difficulties — and include minimum-difficulty blocks in that mean, with no Bitcoin-style walk-back:

• spec §7.7.3 MeanTarget; constants §5.3 (PoWAveragingWindow = 17, PoWDampingFactor = 4, PoWMaxAdjustUp = 16%, PoWMaxAdjustDown = 32%);
• zcashd — the loop in GetNextWorkRequired (src/pow.cpp) sums SetCompact(nBits) over the 17-block window, then CalculateNextWorkRequired rescales it; every block is added unconditionally;
• Zebra — mean_target_difficulty() (zebra-state/src/service/check/difficulty.rs) sums to_expanded() over the window with no filter.

So this is a spec defect both nodes inherit; the remedy is a (testnet) consensus change, not an implementation patch.

The per-block clamp acts only on the timespan factor, so it cannot arrest the mean collapse. Writing AWT := PoWAveragingWindow · PoWTargetSpacing = 17 · 75 = 1275 s (post-Blossom) for the averaging-window timespan, the next threshold is

$$\text{Threshold} = \min!\Big(\text{PoWLimit},\ \tfrac{\text{MeanTarget}}{\text{AWT}} \cdot S\Big),$$

where S is the damped, clamped median-time span of the window, bounded to S ∈ [1 − 0.16, 1 + 0.32] · AWT = [0.84, 1.32] · AWT. Since S merely multiplies the already-collapsed mean, per block it can lower the next difficulty by a further factor ≤ 1.32 (an easier target) or raise it by at most 1 / 0.84 ≈ 1.19. That ×1.19/block ceiling is exactly your ~17 + log₁.₁₉(D/17) recovery bound, and the 1.32 head-room means the realised one-block drop can exceed the mean factor (16 + D)/17 by up to ×1.32.

2. The averaging defect is independent of the timestamp trigger.

A minimum-difficulty block sets nBits = ToCompact(PoWLimit) regardless of why it triggered, so an honest gap ≥ 450 s collapses MeanTarget exactly as a future-dated artifact does. Even with the trigger fixed (MTP-based, no future-dating), legitimate minimum-difficulty blocks still crash difficulty — and not rarely.

Model block discovery under fixed difficulty and hash rate as a homogeneous Poisson process; then the inter-block gap T (seconds) is exponential and memoryless, P(T > t) = e^(−t/μ), where μ is the mean gap and λ := 1/μ the discovery rate. The trigger threshold τ := 6 · PoWTargetSpacing = 450 s is a fixed consensus constant, independent of the realised rate. At difficulty equilibrium the retarget holds μ = PoWTargetSpacing = 75 s, so a genuine gap trips the rule with probability

$$p = P(T > \tau) = e^{-\tau/\mu} = e^{-450/75} = e^{-6} \approx 2.5\times10^{-3},$$

i.e. one trip per 1/p = e⁶ ≈ 403 blocks, or ≈ 403 · 75 s ≈ 8.4 h of chain time.

Off equilibrium, let m denote the realised mean gap — the actual average block spacing, which need not equal the target μ = 75 s (e.g. when hash rate falls before difficulty adjusts, blocks genuinely take longer, so m > 75). The threshold stays fixed at 450 s while only the mean in the exponent changes, so

$$p(m) = e^{-450/m}.$$

This climbs steeply as blocks slow: m = 150 s → e^(−3) ≈ 0.05, m = 225 s → e^(−2) ≈ 0.14. So precisely during the real hashrate droughts the valve is meant to smooth, minimum-difficulty blocks — and hence difficulty crashes — become common.

Conclusion: an averaging-side fix (exclude min-diff from MeanTarget, or average log-targets) is necessary on its own; a trigger-only fix leaves the collapse reachable.

[daira]

[Edited 2026-07-01 13:17 UTC] Corrected the validation section — the mean-block-time match is the calibration target, not independent confirmation of the model (the incorrect wording is struck through below rather than deleted) — and softened the Conclusions to attribute them to the simulation and to the algorithm's structure rather than stating them as established facts about the live chain. The fix-direction comparison (the table) is unchanged.

---

To pin down how much each of the two fix directions above actually helps, Claude wrote an event-driven simulation of the coupled system — the target-space difficulty averaging, the 11-block median-time-past (MTP) rule, and the block-template timestamp heuristic — with the DAA arithmetic and constants transcribed from Zebra (ThresholdBits/MeanTarget/ActualTimespanBounded, the min-difficulty gap, and the Testnet PoWLimit). Hashrate is modelled as constant (Testnet is essentially one operator's ASICs): the expected real time to find a block at difficulty D is D / R, with R chosen so that, absent the pathology, the DAA would settle at the 75 s target.

The model reproduces the observed chain What the simulation does and does not establish

With the single free parameter (equilibrium difficulty) calibrated so the mean block time matches, the simulation independently reproduces the other measured signatures:

Corrected framing: the one free parameter (equilibrium difficulty) was calibrated so that the current configuration reproduces the observed ~13 s mean block time. That match is therefore by construction and is not evidence for the model — for essentially any constant hashrate there is a calibration that reproduces the observed mean.

Calibration is unavoidable here because the chain does not tell us the Testnet ASIC operator's hash rate, and hence not the equilibrium difficulty the DAA would reach absent the pathology: the observed difficulty series is the depressed sawtooth, never that equilibrium, and the per-block timestamps that could otherwise reveal a real block-finding rate at a given difficulty are exactly what the future-dating corrupts. The one robust aggregate the chain does give is the long-run mean block time (~13 s) — robust because over a long window each future-dated spike is compensated by the following step-back, so mean header-time spacing equals mean real spacing. Spending that single reliable observable to fix the single unknown (hash rate, equivalently equilibrium difficulty) is precisely why the resulting mean-block-time match cannot then also count as confirmation.

The narrower, defensible reading:

• mean block time ~13 s (measured: 12.6–12.9 s); — this is the calibration target, not a prediction.
• gap_next negative (~ −151 s) and minimum-difficulty blocks isolated (never consecutive): these were not used in the calibration and are genuine qualitative consequences of the coupled dynamics, so their agreement with the observed chain is (weak) support for the mechanism. gap_prev = 451 s agrees too, but that only confirms Zebra's template logic was transcribed correctly, not the mechanism itself.
• difficulty pinned ~100x below its equilibrium — a statement about the model, since the real chain's equilibrium difficulty is not directly observed.
• the minimum-difficulty fraction — the one genuinely independent quantitative prediction — comes out only order-of-magnitude right: ~0.16% predicted vs 0.30% measured (the real chain presumably also sees occasional genuine hashrate dips).

So this should be read as a mechanistic model that reproduces the pathology given the known code, not as independent empirical confirmation of the mechanism. Its load-bearing use here is the relative comparison of the fix directions below: all four configurations share one hashrate and one calibration, so the differences between them are robust to the absolute equilibrium difficulty even where the absolute numbers are not.

Quantifying the two fix directions

Four configurations, same hashrate and seed (equilibrium difficulty calibrated to the observed ~13 s under the current behaviour):

configuration	trigger margin	averaging	mean block time	min-diff blocks	future-dating
current	prev + 300 s (eager)	over targets	13.1 s (5.7x too fast)	0.16%	151 s
trigger-side only	prev + 450 s	over targets	29.8 s (2.5x too fast)	0.12%	~1 s
averaging-side only	prev + 300 s (eager)	exclude min-diff	75.2 s (on target)	2.11%	151 s
both	prev + 450 s	exclude min-diff	75.3 s (on target)	0.33%	~1 s

("Trigger margin" is the block-template heuristic's switch point; the current implementation switches 2 * PoWTargetSpacing = 150 s early, at prev + 300 s. "Exclude min-diff" is the averaging-side direction: minimum-difficulty blocks' PoWLimit targets are left out of MeanTarget.)

Conclusions

These are results of the simulation together with structural reasoning about the algorithm, not independent empirical proofs about the live chain.

1. The averaging-side change addresses the dominant mechanism, and in the simulation it alone returns block time to target. With minimum-difficulty blocks excluded from MeanTarget, the simulated chain sits at ~75 s even with the eager template margin and the 151 s future-dating still in place. The reason is structural and does not depend on the simulation: only the target-space averaging produces an unbounded effect — a single PoWLimit entry dominates the arithmetic mean of targets and pins difficulty low for the whole 17-block window — whereas the other perturbations here are bounded by the per-block +16%/−32% clamp. One caveat: the DAA is by construction self-correcting toward 75 s absent a crash, so the simulated return to 75 s is partly self-consistency; what it adds is that the residual timestamp pollution on its own does not prevent that return.

2. The trigger-side change helps but, in the simulation, is not a complete fix. Switching to a minimum-difficulty template only once cur_time reaches the consensus threshold (rather than 150 s early) moves the simulated chain from ~13 s to ~30 s and removes the future-dating (151 s → ~1 s), which also stops the future-dated timestamps polluting the DAA's own time measurements. But simulated difficulty stays well below equilibrium, because each trigger that still occurs continues to crash the target-mean. It is a worthwhile, non-consensus mitigation (it affects only template construction, not block validity), not a cure.

3. Both the simulation and the algorithm's structure point to the target-space averaging crash as primary and the timestamp pollution as secondary. The "averaging-side only" run keeps the 151 s future-dating yet returns to target, consistent with the median-time-past being robust to a minority of future-dated blocks and the timespan factor being clamped. So the pollution appears to be a bounded perturbation — enough to make "current" deeper than "trigger-side only", but not by itself able to crash difficulty.

4. As an internal consistency check: once difficulty is no longer being crashed, the simulated minimum-difficulty frequency matches the plain Poisson tail e^{-gap / 75} (2.11% ≈ e^{-4} at a 300 s margin, 0.33% ≈ e^{-6} at 450 s), and with the averaging-side fix in place these triggers no longer move difficulty, so their frequency is only cosmetic.

Net: in the simulation, the averaging-side change returns block time to target on its own, and adding the trigger-side change makes minimum-difficulty blocks rare and removes the future-dating. The direction to rely on is the averaging-side one, because the argument for it — an unbounded target-mean crash versus bounded, clamped perturbations — does not depend on the simulation. The simulation script (and the exact parameters used for the table above) is in the following comment.

🤖 Claude Opus 4.8

[upbqdn]

A live cross-check on Testnet over the last 48 h (heights 4106744 → 4120261, ending 2026-07-01 13:02 UTC):

Mean block spacing = 12.78 s (5.87× the 75 s target), across the 13 517 block intervals in the window.

Method: I divided the elapsed span by the height delta (13 517 intervals), not by the number of records my pool holds (12 992 — it found 96.1 % of blocks).

This independently reproduces the simulation's calibrated ~13 s. It is also consistent with the ~100× depression: the analytic floor D_M → PoWAveragingWindow = 17 gives a steady depression ≈ D_eq / 17, so an equilibrium D_eq ≈ 1.4k implies ≈ 80×, the same order as the simulation.

[daira]

All three implementations compute MeanTarget as the arithmetic mean of the targets of the 17 preceding blocks — not of their difficulties — and include minimum-difficulty blocks in that mean, with no Bitcoin-style walk-back ...

Yes I know; that just hadn't been written down in the thread yet. The simulation takes it into account, and confirms strongly suggests that it's indeed most of the problem.

On the other hand, it would be very disruptive to make a consensus change on testnet before NU6.3. We should defer that to draft-valargroup-blocktime-reduction IMHO.

[upbqdn]

Yes I know; that just hadn't been written down in the thread yet.

Yup, I had my own investigation going on, and I told Claude to help me post the relevant stuff that was missing here.

[upbqdn]

I also wanted to point out that there is no diff among the implementations and spec.

[daira]

Right; there originally was, but it was fixed in #417 (in 2020).

[daira]

The simulation script used for the table above is below.

Parameters used for the table. The four rows were produced with:

python3 testnet-difficulty-sim.py --d-eq 300000 --blocks 80000 --warmup 3000

• --d-eq 300000 — the equilibrium difficulty (the one calibrated free parameter). It was chosen because, under the current configuration, it reproduces the observed ~13 s mean block time; see the calibration discussion in the previous comment for why this has to be fit rather than read off the chain. Hash rate is derived as R = d_eq / 75 (difficulty-1 blocks per real second), so that absent the pathology the DAA would settle at the 75 s target.
• --blocks 80000 --warmup 3000 — 80 000 blocks simulated, the first 3 000 discarded so the statistics are gathered from steady state rather than the bootstrap.
• --seed 12345 (default) — fixed RNG seed, so the table is reproducible. Running with no --d-eq instead performs the D_eq sweep that located 300000.

Everything else is a fixed consensus/DAA constant transcribed from Zebra, not a tunable: Testnet PoWLimit = 2^251 − 1, PoWTargetSpacing = 75 s, PoWAveragingWindow = 17, PoWMedianBlockSpan = 11, PoWDampingFactor = 4, PoWMaxAdjustUp/Down = 16% / 32%, minimum-difficulty gap = 6 × 75 = 450 s, and the block-time-vs-median bound of 90 minutes. The four scenarios differ only in the two levers under discussion: the template switch point (prev + 300 s vs prev + 450 s) and whether minimum-difficulty blocks are excluded from MeanTarget.

testnet-difficulty-sim.py (Python 3, standard library only)

#!/usr/bin/env python3
"""
Event-driven simulation of the Zcash *testnet* difficulty-adjustment / min-difficulty
feedback loop, faithful to zebra's implementation.

Purpose: reproduce and quantify the testnet misbehaviour analysed in zcash/zips#1321 --
mean block time pinned ~6x below the 75s target -- which arises from the coupling of
three mechanisms that resist closed-form analysis:

  (1) target-space difficulty averaging (a single PoWLimit entry pins MeanTarget, hence
      difficulty <= ~17, for the whole 17-block averaging window);
  (2) the 11-block median-time-past (MTP) rule, which lets the header-time series run
      ahead of real time and only walk back down incrementally; and
  (3) zebra's getblocktemplate heuristic, which future-dates a block's nTime to prev+451
      (making it min-difficulty) once clamp(now, MTP+1, MTP+5400) > prev + LOCAL_LIMIT.

The future-dated nTimes then *poison* the DAA's own time measurements (ActualTimespan is
a difference of medians of nTime), so the DAA reads ~75s spacing while blocks really
arrive every ~13s, and holds difficulty low instead of raising it.

DAA constants and arithmetic are transcribed from
  zebra-state/src/service/check/difficulty.rs  (ThresholdBits / MeanTarget / ActualTimespanBounded)
  zebra-chain/src/parameters/network_upgrade.rs (spacing, min-diff gap, PoWLimit)
  zebra-state/src/service/read/difficulty.rs    (adjust_difficulty_and_time_for_testnet)

Scenarios compared:
  current       : LOCAL_LIMIT = prev+300 (EXTRA_TIME_TO_MINE_A_BLOCK = 150), target-space avg
  pr            : LOCAL_LIMIT = prev+450 (EXTRA removed -- the zebra PR), target-space avg
  avg_fix       : LOCAL_LIMIT = prev+300, but MeanTarget excludes min-diff blocks (#1321)
  both          : LOCAL_LIMIT = prev+450 AND MeanTarget excludes min-diff blocks

Hashrate is modelled as constant (testnet is ~one developer's ASICs): expected real
time to find a block at difficulty D is D / R seconds, with R = D_eq / target_spacing so
that absent the pathology the DAA would settle at spacing = target_spacing.
"""

import argparse
import random
import statistics
from dataclasses import dataclass

# ---- consensus / DAA constants (post-Blossom testnet) ----
TARGET_SPACING = 75
POW_AVERAGING_WINDOW = 17
POW_MEDIAN_BLOCK_SPAN = 11
POW_ADJUSTMENT_BLOCK_SPAN = POW_AVERAGING_WINDOW + POW_MEDIAN_BLOCK_SPAN  # 28
POW_DAMPING_FACTOR = 4
POW_MAX_ADJUST_UP_PERCENT = 16
POW_MAX_ADJUST_DOWN_PERCENT = 32
AVERAGING_WINDOW_TIMESPAN = TARGET_SPACING * POW_AVERAGING_WINDOW  # 1275
MIN_ACTUAL_TIMESPAN = AVERAGING_WINDOW_TIMESPAN * (100 - POW_MAX_ADJUST_UP_PERCENT) // 100   # 1071
MAX_ACTUAL_TIMESPAN = AVERAGING_WINDOW_TIMESPAN * (100 + POW_MAX_ADJUST_DOWN_PERCENT) // 100  # 1683
BLOCK_MAX_TIME_SINCE_MEDIAN = 90 * 60  # 5400
MIN_DIFFICULTY_GAP = 6 * TARGET_SPACING  # 450

POW_LIMIT_TESTNET = (1 << 251) - 1


def median(xs):
    # zebra: sorted(S)[len(S)//2]  (matches ceiling((n+1)/2) for the sizes used here)
    s = sorted(xs)
    return s[len(s) // 2]


@dataclass
class Block:
    ntime: int          # header time (may be future-dated)
    real: float         # real wall-clock arrival time
    target: int         # expanded difficulty threshold (target)
    min_diff: bool      # was this a min-difficulty block?


def mean_target(window_targets):
    """MeanTarget over the last 17 block targets (target-space arithmetic mean)."""
    return sum(window_targets) // POW_AVERAGING_WINDOW


def mean_target_excluding_mindiff(window_blocks, fallback_target):
    """#1321 variant: average only non-min-difficulty targets in the window.

    Excludes PoWLimit (min-diff) entries so one trigger cannot crash the mean.
    If every block in the window is min-diff, fall back to PoWLimit."""
    tgts = [b.target for b in window_blocks if not b.min_diff]
    if not tgts:
        return fallback_target
    return sum(tgts) // len(tgts)


def actual_timespan_bounded(blocks):
    """ActualTimespanBounded: damped, clamped difference of medians of nTime."""
    newer = [b.ntime for b in blocks[-POW_MEDIAN_BLOCK_SPAN:]]                       # h-1 .. h-11
    older = [b.ntime for b in blocks[-POW_ADJUSTMENT_BLOCK_SPAN:-POW_AVERAGING_WINDOW]]  # h-18 .. h-28
    actual = median(newer) - median(older)
    # damped_variance truncated toward zero (int() truncates toward zero in Python)
    damped_variance = int((actual - AVERAGING_WINDOW_TIMESPAN) / POW_DAMPING_FACTOR)
    damped = AVERAGING_WINDOW_TIMESPAN + damped_variance
    return max(MIN_ACTUAL_TIMESPAN, min(MAX_ACTUAL_TIMESPAN, damped))


def threshold_bits(blocks, pow_limit, exclude_mindiff):
    """ThresholdBits: the standard (non-min-difficulty) target for the next block.

    Independent of the candidate block's own time (the candidate time only enters
    the separate min-difficulty test), matching zebra's threshold_bits()."""
    window = blocks[-POW_AVERAGING_WINDOW:]
    if exclude_mindiff:
        mt = mean_target_excluding_mindiff(window, pow_limit)
    else:
        mt = mean_target([b.target for b in window])
    tb = (mt // AVERAGING_WINDOW_TIMESPAN) * actual_timespan_bounded(blocks)
    return min(pow_limit, tb)


def simulate(scenario, d_eq, n_blocks, warmup, pow_limit, seed):
    rng = random.Random(seed)
    R = d_eq / TARGET_SPACING          # difficulty-1 blocks found per real second
    eq_target = pow_limit // d_eq      # target corresponding to equilibrium difficulty

    local_limit = 300 if scenario in ("current", "avg_fix") else 450
    exclude_mindiff = scenario in ("avg_fix", "both")

    # Bootstrap: 28 blocks at equilibrium, nTime == real, 75s apart.
    blocks = []
    for i in range(POW_ADJUSTMENT_BLOCK_SPAN):
        t = i * TARGET_SPACING
        blocks.append(Block(ntime=t, real=float(t), target=eq_target, min_diff=False))

    records = []  # per (post-warmup) block: dict of stats
    for h in range(POW_ADJUSTMENT_BLOCK_SPAN, n_blocks):
        prev = blocks[-1]
        mtp = median([b.ntime for b in blocks[-POW_MEDIAN_BLOCK_SPAN:]])
        min_time = mtp + 1
        max_time = mtp + BLOCK_MAX_TIME_SINCE_MEDIAN

        # Standard-difficulty target for this height (time-independent).
        std_target = threshold_bits(blocks, pow_limit, exclude_mindiff)
        std_diff = pow_limit / std_target
        mean_std_find = std_diff / R          # expected real seconds at standard difficulty
        mean_min_find = 1.0 / R               # min-difficulty (D=1) find time

        # Earliest real time at which clamp(now, min_time, max_time) > prev.ntime + local_limit.
        limit = prev.ntime + local_limit
        if min_time > limit:
            # MTP floor already exceeds the limit: min-difficulty from the start of mining.
            t_flip = prev.real
        else:
            # cur_time == now once now >= min_time (< limit here); flips when now > limit.
            t_flip = max(float(limit), prev.real)

        tau_std = rng.expovariate(1.0 / mean_std_find)
        if prev.real + tau_std <= t_flip:
            # ---- standard-difficulty block found before the template flips ----
            real = prev.real + tau_std
            target = std_target
            is_min = False
            # standard branch: max_time clamped down to prev+450, cur_time into range
            smax = min(max(prev.ntime + MIN_DIFFICULTY_GAP, min_time), max_time)
            cur = min(max(int(round(real)), min_time), smax)
            ntime = cur
        else:
            # ---- template flips to min-difficulty; block found ~instantly at D=1 ----
            real = t_flip + rng.expovariate(1.0 / mean_min_find)
            target = pow_limit
            is_min = True
            smin = min(max(prev.ntime + MIN_DIFFICULTY_GAP + 1, min_time), max_time)  # prev+451
            cur = min(max(int(round(t_flip)), min_time), max_time)
            ntime = min(max(cur, smin), max_time)

        # Consensus consistency: nTime gap > 450 <=> min difficulty.
        gap_prev = ntime - prev.ntime
        if gap_prev > MIN_DIFFICULTY_GAP and not is_min:
            # Shouldn't happen given the clamps, but enforce the consensus rule if it does.
            target = pow_limit
            is_min = True
        blocks.append(Block(ntime=ntime, real=real, target=target, min_diff=is_min))

        if h >= warmup:
            records.append(dict(
                ntime=ntime, real=real, gap_prev=gap_prev,
                real_gap=real - prev.real, min_diff=is_min,
                diff=pow_limit / target, ahead=ntime - real,
            ))
        # keep memory bounded
        if len(blocks) > 4 * POW_ADJUSTMENT_BLOCK_SPAN:
            blocks = blocks[-2 * POW_ADJUSTMENT_BLOCK_SPAN:]

    return records, eq_target


def summarize(name, records, d_eq):
    n = len(records)
    ntime_gaps = [r["gap_prev"] for r in records]
    real_gaps = [r["real_gap"] for r in records]
    mins = [r for r in records if r["min_diff"]]
    diffs = [r["diff"] for r in records]
    aheads = [r["ahead"] for r in records]
    mean_ntime = statistics.fmean(ntime_gaps)
    mean_real = statistics.fmean(real_gaps)
    # gap_next for min-diff blocks: the nTime step of the following block
    gap_next = []
    for i, r in enumerate(records):
        if r["min_diff"] and i + 1 < n:
            gap_next.append(records[i + 1]["gap_prev"])
    print(f"=== {name}  (D_eq={d_eq:g}) ===")
    print(f"  blocks:                 {n}")
    print(f"  mean nTime gap:         {mean_ntime:6.2f} s   (ratio to 75s: {75/mean_ntime:.2f}x too fast)")
    print(f"  mean REAL gap:          {mean_real:6.2f} s   (ratio to 75s: {75/mean_real:.2f}x too fast)")
    print(f"  min-diff blocks:        {len(mins)} / {n} = {100*len(mins)/n:.3f}%")
    if mins:
        gp = [r['gap_prev'] for r in mins]
        print(f"    gap_prev  min/median/max: {min(gp)} / {int(median(gp))} / {max(gp)}")
        if gap_next:
            print(f"    gap_next  min/median/max: {min(gap_next)} / {int(median(gap_next))} / {max(gap_next)}")
        fd = [r['ahead'] for r in mins]  # future-dating = nTime - real arrival
        print(f"    future-dating (nTime-real) median: {median(fd):.0f} s")
    print(f"  difficulty min/median/max: {min(diffs):.0f} / {statistics.median(diffs):.0f} / {max(diffs):.0f}"
          f"   (equilibrium {d_eq:g})")
    print(f"  header-ahead-of-real median: {statistics.median(aheads):.0f} s"
          f"   (frac blocks with nTime>real: {100*sum(1 for a in aheads if a>0)/n:.1f}%)")
    print()
    return dict(mean_ntime=mean_ntime, mean_real=mean_real, minfrac=len(mins)/n)


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--d-eq", type=float, default=None,
                    help="equilibrium difficulty; if omitted, sweep to match observed ~12.67s")
    ap.add_argument("--blocks", type=int, default=60000)
    ap.add_argument("--warmup", type=int, default=2000)
    ap.add_argument("--seed", type=int, default=12345)
    ap.add_argument("--scenarios", default="current,pr,avg_fix,both")
    args = ap.parse_args()

    pow_limit = POW_LIMIT_TESTNET

    if args.d_eq is None:
        print("### D_eq sweep (scenario=current) -- find equilibrium difficulty reproducing ~12.67s\n")
        for d_eq in [1e3, 3e3, 1e4, 3e4, 1e5, 3e5, 1e6, 3e6, 1e7]:
            recs, _ = simulate("current", d_eq, args.blocks, args.warmup, pow_limit, args.seed)
            gaps = [r["gap_prev"] for r in recs]
            rgaps = [r["real_gap"] for r in recs]
            minf = sum(1 for r in recs if r["min_diff"]) / len(recs)
            print(f"  D_eq={d_eq:>9.0f}:  mean_nTime={statistics.fmean(gaps):6.2f}s  "
                  f"mean_real={statistics.fmean(rgaps):6.2f}s  min-diff={100*minf:.3f}%")
        print("\n(pick --d-eq to run the full scenario comparison)")
        return

    for sc in args.scenarios.split(","):
        recs, _ = simulate(sc, args.d_eq, args.blocks, args.warmup, pow_limit, args.seed)
        summarize(sc, recs, args.d_eq)


if __name__ == "__main__":
    main()

🤖 Claude Opus 4.8