Introducing Broker Scenarios #446

JacobCallahan · 2025-12-15T02:24:59Z

This pull request introduces the brand new Broker scenarios, providing a powerful way to define and execute complex, chained workflows (scenarios) as YAML files, and follows up with significant feature additions, fixes, and comprehensive documentation.

This feature allows users to automate sophisticated infrastructure and cloud operations by leveraging all existing Broker actions within a structured, dynamic, and testable workflow.

Key Features Introduced and Enhanced

The following features were implemented across the three commits:

1. Core Scenario Workflow Management

New broker scenarios CLI Group:
- list: Display available scenarios.
- execute <scenario>: Run a specified scenario with support for overrides and background execution.
- info <scenario>: Get a summary of a scenario's configuration and steps.
- validate <scenario>: Structural validation against the formal schema.
Core Scenario Capabilities:
- Templating: Integrated Jinja2 templating for dynamic values.
- Conditional Logic: Step execution control via when clauses.
- Variable Management: Capture step outputs into new scenario variables for subsequent use.
- Nesting: Support for running other scenarios as steps.
- Inventory: Management of a scenario-specific inventory for resource checkout/checkin.

2. Advanced Workflow Control and Data Handling

Enhanced Loop Capabilities:
- Allow the iterable to be a Jinja2 expression, variable, or inventory filter (@inv, @scenario_inv).
- Support tuple unpacking for iter_var (e.g., key, value for dict iteration).
- Implement capture.key for dynamic variable key assignment within loops.
- Evaluate when conditions on a per-iteration basis.
Flexible Error Handling:
- on_error can now be a list of recovery steps or the string "continue" to ignore the failure and proceed.
Provider Data Querying:
- Introduce provider_info action to programmatically query configured providers for structured resource data (e.g., available images, workflows) for use within the scenario.
- Provider provider_help methods now return structured data in addition to printing it.
Output Action:
- New output action to write content to stdout, stderr, or files, with automatic format detection.
Type Preservation:
- Improve Jinja2 templating to preserve native Python types (int, bool, list, dict) for simple variable references, preventing unwanted string conversions.

3. Configuration and Observability

Custom Scenario Logging:
- Implement custom logging, allowing users to configure a dedicated log file for each scenario with flexible path resolution.
Formal Schema Validation:
- Introduction of a comprehensive scenario_schema.json for validating scenario files, formalizing the structure of all steps, actions, arguments, and control flow elements.

Key Fixes and Refactoring

merge_dicts Fix: Corrected helpers.merge_dicts to ensure values from the second dictionary argument consistently take precedence, and updated all call sites to reflect the desired precedence behavior (e.g., ensuring CLI arguments correctly override defaults).
Hostname Resolution: Improved container hostname resolution with robust fallbacks to container name or short ID if standard attributes are missing.
Inventory Isolation: Added _clear_scenario_inventory to ensure each scenario starts with a fresh inventory.
Action Result Format: Refined _action_ssh, _action_scp, and _action_sftp to consistently return a single Result object for single-host operations and a dictionary for multi-host operations, simplifying subsequent templating.

Documentation and Testing

New Tutorial: Added an in-depth scenarios_tutorial.md covering all aspects of scenario creation, execution, templating, loops, error handling, and examples.
Functional Tests: Introduced new functional tests for scenarios, including a comprehensive test for the Container provider and specific workflow tests, integrated via test_scenarios.py.

Features: - Introduce the new 'scenarios' feature, enabling the definition and execution of complex workflows as chained Broker actions within YAML files. - Add a new `broker scenarios` CLI group to manage these workflows: - `list`: Displays all available scenario files found in the scenarios directory. - `execute <scenario>`: Runs a specified scenario, supporting command-line variable and configuration overrides, and background execution. - `info <scenario>`: Provides a summary of a scenario's configuration, variables, and steps. - `validate <scenario>`: Checks a scenario file against its defined JSON schema for structural correctness. - Scenario functionality includes: - Jinja2 templating for dynamic values within scenario definitions. - Conditional step execution using `when` clauses. - Iterative step execution via `loop` constructs over iterables or inventory filters. - Capture of step outputs into new scenario variables for subsequent steps. - Configurable error handling with `on_error` blocks and `exit_on_error` flags. - Management of a scenario-specific inventory for `checkout` and `checkin` actions. - Support for nesting and running other scenarios as steps. - Implement `ScenarioRunner` as the core class for loading, validating, and executing scenario definitions. - Add `ScenarioError` to handle exceptions specific to scenario execution. Configuration: - Introduce `broker/scenario_schema.json` to formally define and validate the structure of scenario YAML files. - Update `pyproject.toml` to include `paramiko` as a recognized SSH session and interactive shell entry point.

Features: - Add `output` action for scenario steps to write content to stdout, stderr, or files, with automatic format detection based on file extension. - Introduce `provider_info` action to query configured providers for available resources (e.g., Ansible Tower workflows, OpenStack images) and retrieve structured data for programmatic use. - Significantly enhance scenario loop capabilities: - Allow the `iterable` argument to be a Jinja2 expression, variable, or inventory filter (`@inv`, `@scenario_inv`). - Support tuple unpacking for `iter_var` (e.g., `key, value` for iterating over dict items). - Implement `capture.key` to dynamically set keys for loop output within the step memory. - Evaluate `when` conditions on a per-iteration basis within loops. - Allow `on_error` in scenario steps to be either a list of recovery steps or the string `"continue"` to proceed despite failures. - Export the new `save_file` helper for general use. - Add the `scenarios` command to the interactive Broker shell. - Provider `provider_help` methods (e.g., AnsibleTower, Beaker, Container, Foreman, OpenStack) now return structured data in addition to printing it, enabling programmatic use within scenarios. Fixes: - Correct the `helpers.merge_dicts` behavior to ensure values from the second dictionary argument consistently take precedence over the first. - Update all `merge_dicts` call sites (e.g., Broker initialization, inventory updates) to correctly reflect the desired precedence, ensuring user-provided arguments or new data override defaults or existing data. - Ensure scenario configuration values passed via CLI are deep-merged into existing settings rather than overwriting top-level dictionaries. - Resolve a definition order issue for `broker_shell` commands in `commands.py` by moving shell definition. Refactoring: - Improve scenario host resolution to support filtering against both the main Broker inventory (`@inv`) and the scenario-specific inventory (`@scenario_inv`), including proper host reconstruction. - Refine `_action_ssh`, `_action_scp`, and `_action_sftp` to return a single `Result` object for single-host operations and a dictionary mapping hostnames to `Result` objects for multiple hosts, simplifying templating. - Simplify `list_scenarios` implementation for clarity. - Add `_clear_scenario_inventory` to ensure scenarios start with a fresh inventory, preventing carry-over from previous runs. Documentation: - Update `scenario_schema.json` to reflect new actions, enhanced loop features, and flexible `on_error` handling. - Clarify the `merge_dicts` docstring regarding value precedence. - Add a `.gitignore` file to the `scenarios` directory to exclude non-versioned files.

…mentation Features: - Implement custom scenario-specific logging, allowing users to configure a dedicated log file for each scenario with flexible path resolution rules. - Improve Jinja2 templating to preserve Python types (e.g., int, bool, list, dict) for simple variable references, preventing unnecessary string conversions. - Introduce a comprehensive JSON schema (`scenario_schema.json`) for validating scenario files, formalizing the structure of steps, actions, arguments, loops, conditionals, and error handling. This enables a richer set of scenario capabilities. Fixes: - Improve container hostname resolution in `container_info`, adding robust fallbacks to container name or short ID if the `Config` or `Hostname` attributes are missing. Refactoring: - Adjust the log level for scenario initialization messages from 'info' to 'debug' for cleaner default output. Documentation: - Add a new, in-depth `scenarios_tutorial.md` covering all aspects of scenario creation and execution, including available actions, configuration options, templating, loops, error handling, and examples. Tests: - Introduce new functional tests for scenarios, including a comprehensive test for the Container provider, and specific tests for deploying/checking in containers and querying provider details. - Add `test_scenarios.py` to integrate and run these new functional scenario tests.

Copilot

Pull request overview

This pull request introduces a comprehensive "Broker Scenarios" feature that enables users to define and execute complex, chained workflows as YAML files. The feature provides powerful automation capabilities for infrastructure and cloud operations by orchestrating Broker actions in structured, dynamic, and testable workflows.

Key Changes

New Scenarios Module: Complete implementation of scenario execution engine with support for templating, loops, conditionals, variable capture, and error handling
CLI Integration: New broker scenarios command group with subcommands for listing, executing, validating, and inspecting scenarios
Schema Validation: JSON schema for validating scenario files with comprehensive coverage of all actions and configuration options

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
broker/scenarios.py	Core scenarios module with ScenarioRunner class and action handlers (1283 lines)
broker/scenario_schema.json	JSON schema defining valid scenario structure and validation rules
broker/commands.py	CLI command integration with scenarios subgroup and helper commands
broker/exceptions.py	New ScenarioError exception with contextual error reporting
broker/helpers/dict_utils.py	Fixed merge_dicts to ensure dict2 values take precedence
broker/helpers/file_utils.py	Added save_file helper with format detection and fixed UUID slicing bug
broker/settings.py	Enhanced config merging to support nested dictionary updates
broker/providers/*.py	Updated provider_help methods to return structured data for programmatic use
tests/test_scenarios.py	Comprehensive unit tests for scenarios utilities and ScenarioRunner
tests/functional/test_scenarios.py	Functional tests for end-to-end scenario execution
tests/data/scenarios/*.yaml	Test scenario files covering various features
tests/functional/scenarios/*.yaml	Real-world scenario examples for functional testing
scenarios/scenarios_tutorial.md	Extensive tutorial with examples and best practices (1068 lines)
pyproject.toml	Added jinja2 and jsonschema dependencies
tox.toml	Added test_scenarios.py to test suite

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/functional/test_scenarios.py

broker/helpers/dict_utils.py

broker/scenarios.py

broker/commands.py

tests/data/scenarios/capture_scenario.yaml

broker/scenarios.py

broker/commands.py

broker/scenarios.py

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 14 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

broker/scenarios.py

broker/settings.py

tests/test_scenarios.py

broker/scenarios.py

scenarios/scenarios_tutorial.md

broker/scenarios.py

Tests: - Introduce `test_scenarios.py` with extensive unit tests for the `broker.scenarios` module. - Cover scenario loading, schema validation, Jinja2 templating, expression evaluation, and conditional logic. - Verify support for loop iteration, output capture, and argument mapping within scenario steps. - Add specific sample scenario YAML files (`capture_scenario.yaml`, `conditions_scenario.yaml`, `loop_scenario.yaml`, `invalid_schema.yaml`, `valid_scenario.yaml`) to exercise various scenario features and validation rules. - Test `StepMemory` functionality and `ScenarioRunner` initialization, configuration, and built-in actions (`output`, `exit`). Configuration: - Add `jinja2` and `jsonschema` as core dependencies to `pyproject.toml` to enable scenario templating and schema validation. - Update `tox.toml` to include the new `test_scenarios.py` in the test suite.

JacobCallahan · 2025-12-19T19:37:36Z

I've removed the scenarios tutorial from the codebase and put it in the wiki: https://github.com/SatelliteQE/broker/wiki/Scenarios-Tutorial

JacobCallahan added 3 commits December 8, 2025 10:38

JacobCallahan self-assigned this Dec 15, 2025

JacobCallahan added enhancement New feature or request 0.8 Issues/PRs going into the 0.8.0 release labels Dec 15, 2025

JacobCallahan force-pushed the scenarios branch 2 times, most recently from d0ca075 to 4dc5047 Compare December 18, 2025 16:17

JacobCallahan marked this pull request as ready for review December 18, 2025 23:21

Copilot AI review requested due to automatic review settings December 18, 2025 23:21

Copilot started reviewing on behalf of JacobCallahan December 18, 2025 23:22 View session

Copilot AI reviewed Dec 18, 2025

View reviewed changes

JacobCallahan force-pushed the scenarios branch from 4dc5047 to 8886fe9 Compare December 19, 2025 03:05

JacobCallahan requested a review from Copilot December 19, 2025 03:08

Copilot started reviewing on behalf of JacobCallahan December 19, 2025 03:08 View session

Copilot AI reviewed Dec 19, 2025

View reviewed changes

JacobCallahan force-pushed the scenarios branch from 8886fe9 to 5e203cc Compare December 19, 2025 19:30

JacobCallahan force-pushed the scenarios branch from 5e203cc to 6101317 Compare December 19, 2025 19:37

JacobCallahan merged commit f24bd9f into SatelliteQE:0.8 Dec 22, 2025
7 checks passed

JacobCallahan deleted the scenarios branch December 22, 2025 21:46

Introducing Broker Scenarios #446

Introducing Broker Scenarios #446

Conversation

JacobCallahan commented Dec 15, 2025

Key Features Introduced and Enhanced

1. Core Scenario Workflow Management

2. Advanced Workflow Control and Data Handling

3. Configuration and Observability

Key Fixes and Refactoring

Documentation and Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JacobCallahan commented Dec 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant