Skip to content

docs(utxo/aerospike): document the create-first transaction flow - #1679

Open
ordishs wants to merge 1 commit into
bsv-blockchain:mainfrom
ordishs:docs/aerospike-utxo-create-first
Open

docs(utxo/aerospike): document the create-first transaction flow#1679
ordishs wants to merge 1 commit into
bsv-blockchain:mainfrom
ordishs:docs/aerospike-utxo-create-first

Conversation

@ordishs

@ordishs ordishs commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Documents the create-first ordering in the Aerospike UTXO store README. The behaviour landed in 8bba344cb (PR 1518); the README did not describe it.

What changed

stores/utxo/aerospike/README.md only. No code changes.

  • New Create-First Transaction Flow section: the create / process / commit phases, the three recovery paths (validator roll-forward on ErrTxExists + creating, the pruner sweep that rolls stale records forward instead of deleting them, and setMined), and why the ordering prevents orphaned spends.
  • Added the creating bin to the normal-transaction field list.
  • Restored the trailing newline at end of file.

Accuracy notes

The creating bin description matches teranode.lua:

  • teranode.lua:23local BIN_CREATING = "creating"
  • teranode.lua:297 — the spend guard tests == true explicitly, because nil or absent means not creating
  • teranode.lua:653-654setMined sets the bin to nil, deleting it, rather than setting it to false

The doc therefore says the bin is deleted on commit, not set to false.

Testing

Documentation only, no build or runtime impact.

Comment on lines +116 to +119
2. **Pruner Sweep**:
- The pruner identifies stale 'creating' transactions
- Instead of deleting them, the pruner rolls them forward through processing
- This prevents accumulation of incomplete transaction states

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Major] Documentation appears correct but misleads — implementation differs.

This "Pruner Sweep" recovery path does not exist in the code. The pruner never inspects the creating flag:

  • stores/utxo/aerospike/pruner/pruner_service.go and pruner/prune_policies.go contain zero references to creating/Creating. The pruner selects records by delete_at_height <= blockHeight and deletes them; it has no logic to detect stale creating records or "roll them forward."

The actual auto-recovery of stale creating records happens in subtree validation, not the pruner:

  • services/subtreevalidation/processTxMetaUsingStore.go:152-162 — when a decorated tx has Creating == true, it is treated as missing to trigger re-processing.
  • services/subtreevalidation/processTxMetaUsingStore.go:224-225 — the GetMeta path does the same (if !txMeta.Creating before using the meta).

Suggest replacing this bullet with the subtree-validation re-processing path (or the create-side clearCreatingFlag retry at create.go:1228-1236), which is what the code actually does.

Comment on lines +111 to +114
1. **Validator Roll-Forward** (on ErrTxExists + Creating):
- When a transaction is detected in creating state during validation
- Validator performs roll-forward to complete the transaction
- This handles cases where the initial creation was interrupted

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] The attribution and trigger here are imprecise.

  • Trigger: the heading says "on ErrTxExists + Creating," but the two real triggers are different errors from different layers. The validator retry loop fires on ErrTxCreating (parent still being written), not ErrTxExists — see services/validator/Validator.go:498-525, which retries validateInternal with exponential backoff. ErrTxExists is a create-side outcome (create.go:1232), unrelated to this retry.

  • "Validator performs roll-forward to complete the transaction" overstates what the validator does: it simply retries the child with backoff and gives up after maxRetries (Validator.go:520-524). It does not complete/roll forward the parent. The genuine roll-forward (re-processing the incomplete tx) lives in subtree validation, as noted in the comment on the Pruner Sweep bullet.

Consider reframing as: "Validator retry — on ErrTxCreating, the validator retries validation with backoff while the parent finishes; subtree validation re-processes txs still flagged creating."

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete

Current Review:

Docs-only PR adding a "Create-First Transaction Flow" section to the Aerospike UTXO store README. The creating-bin field description, the commit-phase deletion of the bin (via clearCreatingFlag, also cleared defensively in setMined, teranode.lua:653-654), and the spend-guard rationale all check out against the code.

Two accuracy issues remain in the Recovery Paths section (both flagged inline, still unresolved — the code confirms them):

  • Pruner Sweep bullet describes a path that does not exist. The Aerospike pruner never inspects the creating flag; the real auto-recovery lives in subtree validation (services/subtreevalidation/processTxMetaUsingStore.go:154,224-225), which treats Creating == true as missing to trigger re-processing.
  • Validator Roll-Forward bullet misattributes the trigger and behavior. The validator retry loop fires on ErrTxCreating (services/validator/Validator.go:503), not ErrTxExists, and it retries with exponential backoff then gives up — it does not "roll forward" or complete the parent.

Both build false confidence (docs read correctly but describe different logic), so they're worth addressing before merge. Everything else is accurate.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-1679 (614d6d0)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 139
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.600µ 1.640µ ~ 0.100
Block_ValidOrderAndBlessed_DiskVsMemory/leaves=1024/memory-4 12.30m 12.45m ~ 0.400
Block_ValidOrderAndBlessed_DiskVsMemory/leaves=1024/disk_1-4 12.89m 12.84m ~ 0.200
Block_ValidOrderAndBlessed_DiskVsMemory/leaves=1024/disk_2-4 12.85m 12.84m ~ 0.400
Block_ValidOrderAndBlessed_DiskVsMemory/leaves=16384/memo... 32.57m 31.40m ~ 0.100
Block_ValidOrderAndBlessed_DiskVsMemory/leaves=16384/disk... 40.35m 39.27m ~ 0.100
Block_ValidOrderAndBlessed_DiskVsMemory/leaves=16384/disk... 40.27m 40.61m ~ 0.700
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 70.98n 70.92n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 71.05n 71.08n ~ 1.000
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 71.18n 71.06n ~ 0.100
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 32.72n 32.64n ~ 0.400
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 56.98n 55.51n ~ 0.400
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 128.7n 127.6n ~ 0.400
MiningCandidate_Stringify_Short-4 199.5n 199.0n ~ 0.600
MiningCandidate_Stringify_Long-4 1.351µ 1.373µ ~ 0.200
MiningSolution_Stringify-4 689.4n 707.9n ~ 0.400
BlockInfo_MarshalJSON-4 1.544µ 1.517µ ~ 0.100
NewFromBytes-4 94.70n 94.21n ~ 0.100
AddTxBatchColumnar_Validation-4 1.545µ 1.458µ ~ 0.200
OffsetValidationLoop-4 555.5n 561.9n ~ 0.200
Mine_EasyDifficulty-4 60.88µ 60.60µ ~ 0.100
Mine_WithAddress-4 6.904µ 6.838µ ~ 0.200
BlockAssembler_AddTx-4 0.02896n 0.02644n ~ 0.400
AddNode-4 10.30 10.36 ~ 1.000
AddNodeWithMap-4 10.88 10.76 ~ 0.700
DiskTxMap_SetIfNotExists-4 3.807µ 3.591µ ~ 0.100
DiskTxMap_SetIfNotExists_Parallel-4 3.390µ 3.421µ ~ 1.000
DiskTxMap_ExistenceOnly-4 323.1n 328.6n ~ 0.700
Queue-4 190.8n 190.5n ~ 1.000
AtomicPointer-4 4.447n 4.290n ~ 0.400
TxMapSetIfNotExists-4 62.13n 62.39n ~ 1.000
TxMapSetIfNotExistsDuplicate-4 46.13n 46.29n ~ 1.000
ChannelSendReceive-4 579.6n 576.6n ~ 0.400
DirectSubtreeAdd/4_per_subtree-4 46.67n 44.13n ~ 0.400
DirectSubtreeAdd/64_per_subtree-4 24.08n 24.44n ~ 0.200
DirectSubtreeAdd/256_per_subtree-4 23.48n 23.63n ~ 0.200
DirectSubtreeAdd/1024_per_subtree-4 22.63n 22.60n ~ 1.000
DirectSubtreeAdd/2048_per_subtree-4 22.25n 22.30n ~ 0.800
SubtreeProcessorAdd/4_per_subtree-4 138.6n 138.9n ~ 0.400
SubtreeProcessorAdd/64_per_subtree-4 137.2n 138.8n ~ 0.400
SubtreeProcessorAdd/256_per_subtree-4 135.7n 137.6n ~ 0.200
SubtreeProcessorAdd/1024_per_subtree-4 140.7n 140.2n ~ 0.400
SubtreeProcessorAdd/2048_per_subtree-4 142.1n 140.7n ~ 0.800
SubtreeProcessorRotate/4_per_subtree-4 138.6n 137.1n ~ 0.200
SubtreeProcessorRotate/64_per_subtree-4 138.2n 137.6n ~ 0.800
SubtreeProcessorRotate/256_per_subtree-4 138.1n 137.9n ~ 0.800
SubtreeProcessorRotate/1024_per_subtree-4 137.7n 138.7n ~ 0.100
SubtreeNodeAddOnly/4_per_subtree-4 55.16n 56.54n ~ 0.100
SubtreeNodeAddOnly/64_per_subtree-4 43.47n 43.85n ~ 0.200
SubtreeNodeAddOnly/256_per_subtree-4 42.59n 42.81n ~ 0.100
SubtreeNodeAddOnly/1024_per_subtree-4 41.97n 42.17n ~ 0.200
SubtreeCreationOnly/4_per_subtree-4 77.33n 87.10n ~ 0.100
SubtreeCreationOnly/64_per_subtree-4 281.0n 283.8n ~ 1.000
SubtreeCreationOnly/256_per_subtree-4 819.7n 848.7n ~ 0.100
SubtreeCreationOnly/1024_per_subtree-4 2.455µ 2.490µ ~ 0.400
SubtreeCreationOnly/2048_per_subtree-4 4.378µ 4.263µ ~ 1.000
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 137.5n 132.6n ~ 0.100
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 138.6n 136.5n ~ 0.100
ParallelGetAndSetIfNotExists/1k_nodes-4 6.665m 8.744m ~ 0.100
ParallelGetAndSetIfNotExists/10k_nodes-4 8.934m 10.099m ~ 0.200
ParallelGetAndSetIfNotExists/50k_nodes-4 10.96m 11.08m ~ 0.100
ParallelGetAndSetIfNotExists/100k_nodes-4 12.69m 13.09m ~ 0.100
SequentialGetAndSetIfNotExists/1k_nodes-4 7.828m 7.806m ~ 1.000
SequentialGetAndSetIfNotExists/10k_nodes-4 9.061m 11.655m ~ 0.100
SequentialGetAndSetIfNotExists/50k_nodes-4 14.00m 15.84m ~ 0.100
SequentialGetAndSetIfNotExists/100k_nodes-4 17.39m 22.07m ~ 0.100
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 7.278m 7.906m ~ 0.400
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 10.56m 10.72m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 14.15m 14.31m ~ 1.000
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 7.102m 32.732m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 10.96m 13.20m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 30.08m 30.02m ~ 1.000
CalcBlockWork-4 351.0n 338.9n ~ 1.000
CalculateWork-4 500.3n 462.2n ~ 0.700
CheckOldBlockIDs/on-chain-prefetch/1000-4 35.83µ 36.48µ ~ 0.700
CheckOldBlockIDs/in-memory-chain-check/1000-4 996.6µ 962.4µ ~ 0.100
CheckOldBlockIDs/on-chain-prefetch/10000-4 272.3µ 274.1µ ~ 1.000
CheckOldBlockIDs/in-memory-chain-check/10000-4 1.372m 1.359m ~ 0.200
BuildBlockLocatorString_Helpers/Size_10-4 1004.0n 984.1n ~ 0.400
BuildBlockLocatorString_Helpers/Size_100-4 9.231µ 9.285µ ~ 0.400
BuildBlockLocatorString_Helpers/Size_1000-4 91.43µ 91.51µ ~ 1.000
CatchupWithHeaderCache-4 105.1m 104.6m ~ 0.100
SubtreeSizes/10k_tx_4_per_subtree-4 1.377m 1.395m ~ 0.100
SubtreeSizes/10k_tx_16_per_subtree-4 325.6µ 329.1µ ~ 0.400
SubtreeSizes/10k_tx_64_per_subtree-4 77.43µ 78.39µ ~ 0.700
SubtreeSizes/10k_tx_256_per_subtree-4 19.46µ 19.51µ ~ 0.700
SubtreeSizes/10k_tx_512_per_subtree-4 9.600µ 9.695µ ~ 0.200
SubtreeSizes/10k_tx_1024_per_subtree-4 4.803µ 4.802µ ~ 1.000
SubtreeSizes/10k_tx_2k_per_subtree-4 2.374µ 2.384µ ~ 0.200
BlockSizeScaling/10k_tx_64_per_subtree-4 76.00µ 76.87µ ~ 0.700
BlockSizeScaling/10k_tx_256_per_subtree-4 19.00µ 19.26µ ~ 0.400
BlockSizeScaling/10k_tx_1024_per_subtree-4 4.798µ 4.744µ ~ 0.200
BlockSizeScaling/50k_tx_64_per_subtree-4 402.9µ 404.1µ ~ 0.700
BlockSizeScaling/50k_tx_256_per_subtree-4 95.28µ 95.20µ ~ 0.700
BlockSizeScaling/50k_tx_1024_per_subtree-4 23.60µ 23.51µ ~ 0.400
SubtreeAllocations/small_subtrees_exists_check-4 158.7µ 159.1µ ~ 1.000
SubtreeAllocations/small_subtrees_data_fetch-4 165.5µ 168.7µ ~ 0.100
SubtreeAllocations/small_subtrees_full_validation-4 332.4µ 331.4µ ~ 1.000
SubtreeAllocations/medium_subtrees_exists_check-4 9.487µ 9.583µ ~ 0.200
SubtreeAllocations/medium_subtrees_data_fetch-4 9.939µ 9.933µ ~ 1.000
SubtreeAllocations/medium_subtrees_full_validation-4 19.44µ 19.38µ ~ 0.400
SubtreeAllocations/large_subtrees_exists_check-4 2.290µ 2.267µ ~ 1.000
SubtreeAllocations/large_subtrees_data_fetch-4 2.403µ 2.386µ ~ 0.400
SubtreeAllocations/large_subtrees_full_validation-4 4.850µ 4.842µ ~ 1.000
_BufferPoolAllocation/16KB-4 4.204µ 2.884µ ~ 0.200
_BufferPoolAllocation/32KB-4 7.076µ 7.259µ ~ 0.700
_BufferPoolAllocation/64KB-4 14.01µ 14.39µ ~ 0.700
_BufferPoolAllocation/128KB-4 27.27µ 23.70µ ~ 0.100
_BufferPoolAllocation/512KB-4 91.01µ 103.04µ ~ 0.200
_BufferPoolConcurrent/32KB-4 17.12µ 17.36µ ~ 0.400
_BufferPoolConcurrent/64KB-4 24.56µ 27.25µ ~ 0.100
_BufferPoolConcurrent/512KB-4 126.3µ 126.5µ ~ 1.000
_SubtreeDeserializationWithBufferSizes/16KB-4 452.6µ 480.1µ ~ 0.200
_SubtreeDeserializationWithBufferSizes/32KB-4 455.2µ 439.8µ ~ 0.400
_SubtreeDeserializationWithBufferSizes/64KB-4 450.7µ 430.3µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/128KB-4 461.7µ 419.0µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/512KB-4 458.5µ 421.2µ ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/16KB-4 25.43m 25.79m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/32KB-4 25.27m 25.45m ~ 0.700
_SubtreeDataDeserializationWithBufferSizes/64KB-4 25.93m 25.21m ~ 0.200
_SubtreeDataDeserializationWithBufferSizes/128KB-4 25.11m 24.99m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/512KB-4 25.74m 26.15m ~ 1.000
_PooledVsNonPooled/Pooled-4 758.8n 764.6n ~ 0.200
_PooledVsNonPooled/NonPooled-4 5.972µ 5.938µ ~ 1.000
_MemoryFootprint/Current_512KB_32concurrent-4 6.421µ 5.857µ ~ 0.700
_MemoryFootprint/Proposed_32KB_32concurrent-4 7.724µ 8.335µ ~ 0.100
_MemoryFootprint/Alternative_64KB_32concurrent-4 7.566µ 7.856µ ~ 0.400
_prepareTxsPerLevel-4 322.7m 313.8m ~ 0.400
_prepareTxsPerLevelOrdered-4 3.022m 3.338m ~ 0.200
_prepareTxsPerLevel_Comparison/Original-4 314.6m 316.1m ~ 0.700
_prepareTxsPerLevel_Comparison/Optimized-4 3.218m 3.178m ~ 1.000
StoreBlock_Sequential/BelowCSVHeight-4 342.8µ 353.6µ ~ 0.100
StoreBlock_Sequential/AboveCSVHeight-4 343.7µ 294.3µ ~ 0.400
GetUtxoHashes-4 270.8n 266.4n ~ 0.100
GetUtxoHashes_ManyOutputs-4 45.53µ 43.60µ ~ 0.400
MetaBytes-4 87.90n 88.58n ~ 0.100
_NewMetaDataFromBytes-4 279.3n 278.4n ~ 1.000
_Bytes-4 390.6n 397.7n ~ 0.100
_MetaBytes-4 137.1n 138.6n ~ 0.100

Threshold: >10% with p < 0.05 | Generated: 2026-09-07 08:19 UTC

Describe the create-first ordering that the Aerospike UTXO store now uses:
the tentative 'creating' state, the three recovery paths (validator
roll-forward on ErrTxExists, the pruner sweep that rolls forward instead
of deleting, and setMined), and why the ordering prevents orphaned spends.

Also add the creating bin to the record field list. The bin is set to true
during creation and removed on finalisation, since absence means the
transaction is not creating.
@ordishs
ordishs force-pushed the docs/aerospike-utxo-create-first branch from 0156f4f to 99e5d7d Compare September 7, 2026 08:04
@sonarqubecloud

sonarqubecloud Bot commented Sep 7, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant