Organizing the Loading and Checking of Input Parameters to stacks-node

[gregorycoppola]

Overview

This discussion:

• outlines how crucial input parameters for stacks-node are loaded at disparate places
• discusses the implications of this
• proposes a flow for communication of constraints between Stacks engineers

Background

Parameters that govern how stacks-node will run are mostly specified in the object Config. Config is created from the user-supplied object ConfigFile in the factory method

pub fn from_config_file(config_file: ConfigFile) -> Result<Config, String>

Problems Identified

• not all parameters are loaded at this time
  • some parameters are loaded at arbitrary times, including at runtime
  • no canonical place to find where parameters are loaded
• not all parameters are checked at this time
  • some incorrect inputs are not detected until runtime, including after a lot of runtime has passed
  • results in wasted runtime

Drawbacks

When parameters enter execution at arbitrary points:

• no canonical place to put constraints
• no canonical place to find constraints
• no canonical place to print and inspect the parameters

Proposal

Observation:

• all parameters can, in principle, be loaded as early as Config::from_config_file
• there is no need or reason to wait

Therefore:

• determine all parameters before Config::from_config_file is done
• check all parameters in this time as well

Proposed Communication Flow Between Stacks Engineers

• developer who introduces constraints
  • has the responsibility to check them in from_config_file
• tester whose input must respect the constraints
  • can assume that if their inputs pass from_config_file they are good
  • if inputs pass from_config_file

Examples of Disparate Parameters Load Sites

In this section, we look at the disparate load sites of:

• BurnchainParameters
• PoxConstants
• get_bitcoin_stacks_epochs::Vec<StacksEpoch>

Note that, in order to check constraints for #3407, we had to compare these three structures.

Config::from_config_file

INFO [1669044084.718603] [testnet/stacks-node/src/config.rs:522] [main] Config::from_config_file
   1: stacks_node::config::Config::from_config_file
             at src/config.rs:521:18
   2: stacks_node::main
             at src/main.rs:184:22

BurnchainParameters::from_params

INFO [1669044084.734057] [src/burnchains/mod.rs:109] [main] BurnchainParameters::from_params
   1: blockstack_lib::burnchains::BurnchainParameters::from_params
             at /Users/greg/main1/src/burnchains/mod.rs:108:18
   2: stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::with_burnchain
             at src/burnchains/bitcoin_regtest_controller.rs:249:32
   3: stacks_node::run_loop::neon::RunLoop::instantiate_burnchain_state
             at src/run_loop/neon.rs:362:40
      stacks_node::run_loop::neon::RunLoop::start
             at src/run_loop/neon.rs:592:13
   4: stacks_node::main
             at src/main.rs:209:9

PoxConstants::new

trace 1

INFO [1669044084.735594] [src/burnchains/mod.rs:332] [main] PoxConstants::new
   1: blockstack_lib::burnchains::PoxConstants::new
             at /Users/greg/main1/src/burnchains/mod.rs:331:18
   2: blockstack_lib::burnchains::PoxConstants::regtest_default
             at /Users/greg/main1/src/burnchains/mod.rs:410:9
      blockstack_lib::burnchains::burnchain::<impl blockstack_lib::burnchains::Burnchain>::new
             at /Users/greg/main1/src/burnchains/burnchain.rs:419:17
   3: stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::default_burnchain
             at src/burnchains/bitcoin_regtest_controller.rs:364:23
      stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::get_burnchain
             at src/burnchains/bitcoin_regtest_controller.rs:388:21
   4: stacks_node::run_loop::neon::RunLoop::instantiate_burnchain_state
             at src/run_loop/neon.rs:391:32
      stacks_node::run_loop::neon::RunLoop::start
             at src/run_loop/neon.rs:592:13
   5: stacks_node::main
             at src/main.rs:209:9

trace 2

INFO [1669044084.736168] [src/burnchains/mod.rs:332] [main] PoxConstants::new
   1: blockstack_lib::burnchains::PoxConstants::new
             at /Users/greg/main1/src/burnchains/mod.rs:331:18
   2: blockstack_lib::burnchains::PoxConstants::regtest_default
             at /Users/greg/main1/src/burnchains/mod.rs:410:9
      blockstack_lib::burnchains::burnchain::<impl blockstack_lib::burnchains::Burnchain>::new
             at /Users/greg/main1/src/burnchains/burnchain.rs:419:17
   3: stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::default_burnchain
             at src/burnchains/bitcoin_regtest_controller.rs:364:23
      stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::get_burnchain
             at src/burnchains/bitcoin_regtest_controller.rs:388:21
   4: stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::receive_blocks
             at src/burnchains/bitcoin_regtest_controller.rs:466:29
   5: <stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController as stacks_node::burnchains::BurnchainController>::start
             at src/burnchains/bitcoin_regtest_controller.rs:1779:9
      stacks_node::run_loop::neon::RunLoop::instantiate_burnchain_state
             at src/run_loop/neon.rs:410:15
      stacks_node::run_loop::neon::RunLoop::start
             at src/run_loop/neon.rs:592:13

get_bitcoin_stacks_epochs

INFO [1669044547.140982] [src/burnchains/bitcoin/indexer.rs:105] [main] get_bitcoin_stacks_epochs
   1: blockstack_lib::burnchains::bitcoin::indexer::get_bitcoin_stacks_epochs
             at /Users/greg/main1/src/burnchains/bitcoin/indexer.rs:104:14
      <blockstack_lib::burnchains::bitcoin::indexer::BitcoinIndexer as blockstack_lib::burnchains::indexer::BurnchainIndexer>::get_stacks_epochs
             at /Users/greg/main1/src/burnchains/bitcoin/indexer.rs:1037:21
   2: blockstack_lib::burnchains::burnchain::<impl blockstack_lib::burnchains::Burnchain>::sync_with_indexer
             at /Users/greg/main1/src/burnchains/burnchain.rs:1368:13
      stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::receive_blocks
             at src/burnchains/bitcoin_regtest_controller.rs:472:19
   3: <stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController as stacks_node::burnchains::BurnchainController>::sync
             at src/burnchains/bitcoin_regtest_controller.rs:1795:13
   4: stacks_node::run_loop::neon::RunLoop::start
             at src/run_loop/neon.rs:726:27
   5: stacks_node::main
             at src/main.rs:209:9

[jcnelson]

This is already being handled here: #3409

[gregorycoppola]

Yes. Thanks. That is a great PR.

But, also, can we?:

1. make into a convention the fact that constraints should be i) checked, and ii) checked by the end of Config::from_config_file
2. move the loading of parameters like BurnchainParameters and PoxConstants (and any other relevant parameters) to be a part of the creation of this Config, e.g. so that they can be loaded from the toml file, cf. Make BurnchainParameters TOML-configurable.. possibly even PoxConstants #3417