Feature Request: Teranode Auto-Tuning Performance Profiles

[KeyvanJS]

I used Claude.ai to summarize and format my proposal.

Feature Request: Auto-Tuning Performance Profiles

📋 Summary

Include a performance profiling system that automatically detects system hardware and generates optimized configuration values for all microservices. Currently, achieving optimal performance requires deep knowledge of 14+ services, manual calculation of values based on hardware, and extensive trial-and-error tuning. This creates a high barrier for new node operators.

---

💡 Proposed Solution

A teranode-profiler CLI tool with three core functions:

Function	Description
Hardware Detection	Automatically reads CPU cores, RAM, disk type (HDD/SSD/NVMe), available storage, and network speed
Preset Profiles	Four standard profiles for common deployment scenarios
Config Generator	Outputs optimized YAML configs for all modules based on detected hardware or selected profile

---

✅ Benefits

• Lower barrier to entry — New operators get optimal configs instantly
• Better hardware utilization — No wasted or overcommitted resources
• Reduced misconfiguration — Consistent, tested configurations
• Mode-specific optimization — Mining, listen-mode, and explorer configs
• Self-documenting — Generated configs explain the reasoning

---

🔮 Additional Features (Future)

• Docker/Kubernetes resource limit generation
• Interactive tuning wizard
• Config comparison against recommendations
• Runtime monitoring with adjustment suggestions

---

🙋 Contribution

I am willing to help implement this feature or provide additional specifications as needed.

[KeyvanJS]

Full proposal

🎯 Motivation

Currently, configuring Teranode for optimal performance requires:

1. Deep knowledge of each microservice's configuration options
2. Manual calculation of appropriate values based on available hardware
3. Trial-and-error tuning across 14+ services
4. Understanding the interdependencies between services (e.g., Kafka buffer sizes affecting validator throughput)

This creates a high barrier to entry for new node operators and makes it difficult to achieve optimal performance without extensive experimentation.

---

💡 Proposed Solution

1. Hardware Detection Script

A script/tool that reads system resources and outputs recommended configurations:

teranode-profiler --detect

# Output:
# System Detection Results:
# ========================
# CPU Cores:      64 (AMD EPYC 7763)
# RAM:            256 GB
# Disk Type:      NVMe SSD
# Disk Space:     4 TB available
# Network:        10 Gbps
# 
# Recommended Profile: high-performance
# Run: teranode-profiler --generate --profile=high-performance

2. Preset Performance Profiles

Profile	Target Hardware	Use Case
minimal	4 cores, 16GB RAM, HDD	Testing, development, Teratestnet
standard	8-16 cores, 64GB RAM, SSD	Small-scale production, listen-mode nodes
high-performance	32+ cores, 128GB+ RAM, NVMe	Production mining nodes
enterprise	64+ cores, 256GB+ RAM, NVMe RAID	High-throughput transaction processors
custom	Auto-detected	Generated based on actual hardware

3. Configuration Generator

# Generate config based on detected hardware
teranode-profiler --generate --output=./config/

# Generate specific profile
teranode-profiler --generate --profile=high-performance --output=./config/

# Generate with overrides
teranode-profiler --generate --profile=standard \
  --network=mainnet \
  --direct=true \
  --override="validator.workers=32"

4. Output Structure

./config/
├── generated-settings.yaml       # Main unified config
├── profiles/
│   ├── propagation.yaml
│   ├── validator.yaml
│   ├── block-assembly.yaml
│   ├── block-validation.yaml
│   ├── subtree-validation.yaml
│   ├── blockchain.yaml
│   ├── block-persister.yaml
│   ├── utxo-persister.yaml
│   ├── p2p.yaml
│   ├── asset-server.yaml
│   ├── kafka.yaml
│   ├── aerospike.yaml
│   └── blob-store.yaml
└── profile-report.txt            # Human-readable explanation of choices

---

⚙️ Modules Requiring Fine-Tuning

Core Services

1. Propagation Service

Parameter	Description	Tuning Factor
propagation.workers	Parallel propagation workers	CPU cores
propagation.buffer_size	Incoming transaction buffer	RAM
propagation.max_batch_size	Batch size for forwarding	Network bandwidth
propagation.rate_limit	Rate limiting for peers	Network bandwidth

2. Validator Service

Parameter	Description	Tuning Factor
validator.workers	Parallel validation workers	CPU cores
validator.script_workers	Script execution parallelism	CPU cores
validator.cache_size	Script cache size	RAM
validator.utxo_prefetch_batch	UTXO prefetch batch size	RAM, Disk I/O
validator.max_script_ops	Max script operations	CPU

3. Block Assembly Service

Parameter	Description	Tuning Factor
blockassembly.workers	Assembly workers	CPU cores
blockassembly.max_block_size	Maximum block size	RAM, Disk
blockassembly.subtree_size	Target subtree size	RAM
blockassembly.merkle_workers	Merkle tree calculation workers	CPU cores
blockassembly.candidate_refresh_interval	Block template refresh rate	CPU

4. Block Validation Service

Parameter	Description	Tuning Factor
blockvalidation.workers	Parallel block validators	CPU cores
blockvalidation.parallel_subtrees	Concurrent subtree validation	CPU cores, RAM
blockvalidation.cache_size	Validation cache	RAM

5. Subtree Validation Service

Parameter	Description	Tuning Factor
subtreevalidation.workers	Subtree validation workers	CPU cores
subtreevalidation.batch_size	Transaction batch size	RAM
subtreevalidation.prefetch_depth	UTXO prefetch depth	RAM, Disk I/O

6. Blockchain Service

Parameter	Description	Tuning Factor
blockchain.cache_size	Block header cache	RAM
blockchain.index_cache	Chain index cache	RAM
blockchain.reorg_depth	Max reorg depth to track	RAM

---

Overlay Services

7. Block Persister Service

Parameter	Description	Tuning Factor
blockpersister.workers	Persistence workers	Disk I/O, CPU
blockpersister.buffer_size	Write buffer size	RAM
blockpersister.batch_size	Batch write size	Disk I/O
blockpersister.compression	Compression level	CPU vs Disk tradeoff
blockpersister.skipUTXODelete	Skip UTXO deletion (for explorers)	Disk space
blockpersister.flush_interval	Flush to disk interval	Disk I/O

8. UTXO Persister Service

Parameter	Description	Tuning Factor
utxopersister.workers	Persistence workers	Disk I/O
utxopersister.buffer_size	Write buffer (e.g., 1024KB)	RAM
utxopersister.batch_size	Batch size for Aerospike writes	RAM, Network
utxopersister.flush_interval	Flush interval	Disk I/O

9. P2P Service

Parameter	Description	Tuning Factor
p2p.max_peers	Maximum peer connections	RAM, Network
p2p.connection_timeout	Connection timeout	Network latency
p2p.buffer_size	Per-connection buffer	RAM
p2p.max_message_size	Maximum P2P message size	RAM
p2p.ban_threshold	Peer ban threshold	—

10. Asset Server

Parameter	Description	Tuning Factor
asset.http_workers	HTTP handler workers	CPU cores
asset.cache_size	Response cache size	RAM
asset.rate_limit	API rate limiting	CPU, Network
asset.centrifuge_buffer	WebSocket buffer	RAM

---

Infrastructure Components

11. Kafka Settings

Parameter	Description	Tuning Factor
kafka.num_partitions	Topic partitions	CPU cores
kafka.replication_factor	Replication factor	Disk, Network
kafka.batch_size	Producer batch size	RAM, Network
kafka.buffer_memory	Producer buffer	RAM
kafka.linger_ms	Batch linger time	Latency vs throughput
kafka.fetch_min_bytes	Consumer min fetch	Network
kafka.fetch_max_bytes	Consumer max fetch	RAM
kafka.max_poll_records	Max records per poll	RAM

12. Aerospike / UTXO Store

Parameter	Description	Tuning Factor
aerospike.namespace_memory	Memory allocation	RAM
aerospike.write_block_size	Write block size	Disk I/O
aerospike.defrag_threshold	Defrag threshold	Disk I/O
aerospike.ConnectionQueueSize	Connection pool size	CPU cores
aerospike.MinConnectionsPerNode	Min connections	CPU cores
aerospike.externalStore.hashPrefix	External store sharding	Disk I/O

13. Blob Store

Parameter	Description	Tuning Factor
blobstore.cache_size	In-memory cache	RAM
blobstore.write_buffer	Write buffer size	RAM
blobstore.compression	Compression algorithm	CPU vs Disk
blobstore.sharding_prefix	Directory sharding depth	Disk I/O

---

📄 Example: Auto-Generated Profile

Input: System Detection

CPU:    AMD EPYC 7763 (64 cores / 128 threads)
RAM:    256 GB DDR4
Disk:   2x 2TB NVMe (RAID 0)
Network: 10 Gbps

Output: generated-settings.yaml

# Auto-generated by teranode-profiler
# Profile: enterprise (auto-detected)
# Generated: 2026-01-18T12:00:00Z
# Hardware: 64 cores, 256GB RAM, NVMe RAID

global:
  network: mainnet
  log_level: info
  direct: true

# Core Services
propagation:
  workers: 16
  buffer_size: 512MB
  max_batch_size: 10000
  rate_limit: 100000/s

validator:
  workers: 32                    # 50% of cores for validation
  script_workers: 16             # 25% of cores for script execution
  cache_size: 8GB
  utxo_prefetch_batch: 5000
  max_script_ops: 500000000

blockassembly:
  workers: 8
  max_block_size: 4GB
  subtree_size: 256MB
  merkle_workers: 8
  candidate_refresh_interval: 500ms

blockvalidation:
  workers: 16
  parallel_subtrees: 8
  cache_size: 4GB

subtreevalidation:
  workers: 16
  batch_size: 10000
  prefetch_depth: 1000

blockchain:
  cache_size: 2GB
  index_cache: 1GB
  reorg_depth: 100

# Overlay Services
blockpersister:
  workers: 8
  buffer_size: 1GB
  batch_size: 1000
  compression: lz4
  skipUTXODelete: false          # Set true for block explorers
  flush_interval: 1s

utxopersister:
  workers: 8
  buffer_size: 1GB              # 1024KB * 1024
  batch_size: 10000
  flush_interval: 500ms

p2p:
  max_peers: 125
  connection_timeout: 30s
  buffer_size: 64MB
  max_message_size: 256MB

asset:
  http_workers: 16
  cache_size: 4GB
  rate_limit: 10000/s
  centrifuge_buffer: 256MB

# Infrastructure
kafka:
  num_partitions: 64            # Match CPU cores
  replication_factor: 1         # Increase for HA
  batch_size: 1MB
  buffer_memory: 2GB
  linger_ms: 5
  fetch_min_bytes: 1MB
  fetch_max_bytes: 50MB
  max_poll_records: 10000

aerospike:
  namespace_memory: 128GB       # 50% of RAM for UTXO store
  write_block_size: 1MB
  defrag_threshold: 50
  ConnectionQueueSize: 512
  MinConnectionsPerNode: 128
  LimitConnectionsToQueueSize: true
  externalStore:
    path: /data/external
    hashPrefix: 2

blobstore:
  cache_size: 16GB
  write_buffer: 1GB
  compression: lz4
  sharding_prefix: 2

---

🛠️ Implementation Suggestions

Phase 1: Hardware Detection

type SystemProfile struct {
    CPUCores         int
    CPUModel         string
    RAMBytes         int64
    DiskType         string  // "hdd", "ssd", "nvme"
    DiskSpaceBytes   int64
    NetworkSpeedMbps int
}

func DetectSystem() (*SystemProfile, error) {
    // Use runtime, /proc/cpuinfo, /proc/meminfo, lsblk, etc.
}

Phase 2: Profile Calculator

type ProfileCalculator struct {
    system  *SystemProfile
    network string  // mainnet, testnet, teratestnet
    mode    string  // mining, listen, explorer
}

func (p *ProfileCalculator) CalculateValidatorWorkers() int {
    // Reserve 50% of cores for validation
    return max(4, p.system.CPUCores / 2)
}

func (p *ProfileCalculator) CalculateKafkaPartitions() int {
    // Match partitions to available parallelism
    return max(8, p.system.CPUCores)
}

func (p *ProfileCalculator) CalculateAerospikeMemory() int64 {
    // Allocate 50% of RAM for UTXO store
    return p.system.RAMBytes / 2
}

Phase 3: Profile Templates

# profiles/minimal.yaml.tmpl
validator:
  workers: {{ max 2 (div .CPUCores 4) }}
  cache_size: {{ max "512MB" (div .RAMBytes 16) }}

---

🖥️ CLI Interface

# Detect and show recommendations
teranode-profiler detect

# Generate config with auto-detection
teranode-profiler generate --output ./config/

# Generate specific profile
teranode-profiler generate --profile minimal --output ./config/

# Generate for specific use case
teranode-profiler generate --mode explorer --network mainnet

# Validate existing config against hardware
teranode-profiler validate --config ./config/settings.yaml

# Compare current config to recommended
teranode-profiler diff --config ./config/settings.yaml

# Interactive tuning wizard
teranode-profiler wizard

---

📌 Additional Considerations

1. Mode-Specific Profiles

Mode	Optimization Focus
Mining Mode	Block assembly and validation throughput
Listen Mode	Sync speed and minimal resource usage
Explorer Mode	Enable skipUTXODelete, larger caches for historical queries

2. Docker/Kubernetes Integration

# docker-compose.override.yaml generated by profiler
services:
  validator:
    deploy:
      resources:
        limits:
          cpus: '32'
          memory: 64G
    environment:
      - VALIDATOR_WORKERS=32

3. Runtime Monitoring & Adjustment

Future enhancement: Monitor actual resource usage and suggest config adjustments:

teranode-profiler monitor --duration 1h

# Output: 
# "Validator workers underutilized (avg 45% CPU). 
#  Consider reducing from 32 to 24."

---

✅ Benefits

• Lower barrier to entry — New operators get optimal configs instantly
• Consistent performance — Reduce misconfiguration issues
• Hardware utilization — Ensure resources aren't wasted or overcommitted
• Documentation — Generated configs serve as learning material
• Reproducibility — Same hardware = same recommended config

---

🔗 Related Issues

• Configuration documentation improvements
• Performance benchmarking suite
• Resource monitoring dashboard

---

🙋 Willingness to Contribute

I am willing to help implement this feature or provide additional specifications as needed.

---

📝 Environment

• Teranode Version: latest
• Deployment: Docker / Kubernetes
• Hardware: Various (AMD EPYC, Intel Xeon, etc.)

[icellan]

@KeyvanJS Experience tells us that this is very hard to automate, especially in a complex cluster where the settings for a pod are dependent on how other pods are configured. For example, the setTxMined batch size and concurrency are dependent on the expected throughput of the network and the Aerospike cluster setup being used.

This is definitely something that needs more work. If you would like to propose a solution for lowering the threshold, that would be very welcome.