Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 6, 2025

  • Analyze helpers.py file and identify function groupings
  • Create helpers directory structure
  • Create dict_utils.py with dictionary utilities
  • Create file_utils.py with file handling utilities
  • Create inventory.py with inventory management utilities
  • Create results.py with Result and MockStub classes
  • Create misc.py with CLI and miscellaneous utilities
  • Create init.py to re-export all functions for backward compatibility
  • Resolve merge conflicts with 0.8 branch
    • Update helpers modules to use Python standard logging
    • Update update_log_level() and fork_broker() to use new logging system
    • Add broker/logging.py from 0.8 branch
    • Remove deprecated broker/logger.py
    • Update all dependent modules
  • Address code review feedback
    • Fix duplicate collections import
    • Add __eq__ method to MockStub class
    • Simplify __getitem__ logic
  • Run tests to verify functionality
  • Run pre-commit checks
Original prompt

This section details on the original issue you should resolve

<issue_title>Refactor helpers</issue_title>
<issue_description>Right now all helpers are written in a single helpers.py file. This file has grown considerably since the inception of broker. We should separate these helpers into four to six separate files based on the similarity of their functionality.
We would then need to either update imports throughout broker to selectively import the helper file as needed or just import everything from the individual files into the main init.py file in the new helpers directory.</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Refactor helpers into separate files for better organization Refactor helpers into separate modules Dec 6, 2025
Copilot AI requested a review from JacobCallahan December 6, 2025 02:57
@JacobCallahan JacobCallahan changed the base branch from master to 0.8 December 6, 2025 03:03
@JacobCallahan JacobCallahan marked this pull request as ready for review December 6, 2025 03:03
Copilot AI review requested due to automatic review settings December 6, 2025 03:03
@JacobCallahan
Copy link
Member

@copilot I've changed the target branch this will be merged into. It now has merge conflicts that I need you to resolve

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR successfully refactors the large helpers.py file into a well-organized helpers/ package with modules grouped by functionality. The refactoring maintains full backward compatibility while improving code organization and fixing several bugs.

Key Changes

  • Split 785-line helpers.py into 5 focused modules: dict_utils.py, file_utils.py, inventory.py, results.py, and misc.py
  • Fixed a NameError bug in find_origin() by properly initializing the _frame variable
  • Removed buggy post-processing code in container.py that would have caused an AttributeError
  • Corrected spelling: "Uknown" → "Unknown"

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
broker/helpers/init.py Provides backward compatibility by re-exporting all helpers from submodules with complete __all__ list
broker/helpers/dict_utils.py Dictionary manipulation utilities (clean_dict, merge_dicts, flatten_dict, dict_from_paths)
broker/helpers/file_utils.py File handling utilities (load_file, FileLock, yaml operations, temporary files)
broker/helpers/inventory.py Inventory management (load_inventory, update_inventory, inventory_fields_to_dict) with proper circular import handling
broker/helpers/results.py Result and testing classes (Result, MockStub, eval_filter, FilterTest)
broker/helpers/misc.py Miscellaneous helpers (Emitter, find_origin with NameError fix, timeout translation, table formatting)
broker/providers/container.py Removed buggy code that would attempt .split() on a tuple
pyproject.toml Version bump to 0.7.3
uv.lock Version bump to 0.7.3
.pre-commit-config.yaml Updated ruff to v0.14.7
.gitignore Added .venv to ignore list

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@JacobCallahan
Copy link
Member

@copilot you've received some review feedback. Please consider the comments as well as their real implication given the context of the comment and decide whether you should accept that change or involve a little more involved code change because of it.

Copy link
Contributor Author

Copilot AI commented Dec 6, 2025

@copilot I've changed the target branch this will be merged into. It now has merge conflicts that I need you to resolve

Resolved the merge conflicts in commit 339713f. Updated the helpers module to use Python's standard logging instead of logzero, matching the logging refactor in the 0.8 branch. Also added broker/logging.py, removed the deprecated broker/logger.py, and updated all dependent modules.

Copy link
Contributor Author

Copilot AI commented Dec 6, 2025

@copilot you've received some review feedback. Please consider the comments as well as their real implication given the context of the comment and decide whether you should accept that change or invol...

Addressed the review feedback in commit 7d6c284:

  1. Duplicate collections import: Fixed by using from collections.abc import Hashable directly instead of import collections.

  2. Missing __eq__ method: Added __eq__ to complement __hash__. The implementation compares the underlying data dict for equality with other MockStub instances or plain dicts.

  3. Unnecessary assignment in __getitem__: Refactored the method to return directly from the try block, eliminating the intermediate assignment that was being overwritten.

@JacobCallahan
Copy link
Member

@copilot okay you've gotten into a pretty bad State when you tried to resolve the merge conflicts. Commit 339713f essentially just copied the changes that were already in the branch instead of rebasing your work on top of the 0.8 branch. Please attempt to resolve this

Copy link
Contributor Author

Copilot AI commented Dec 6, 2025

@copilot okay you've gotten into a pretty bad State when you tried to resolve the merge conflicts. Commit 339713f essentially just copied the changes that were already...

I'm unable to perform a proper rebase as I don't have access to force push operations. The git rebase command would require force pushing to update the branch history.

Could you either:

  1. Perform the rebase yourself and let me continue from there, or
  2. Create a fresh branch from 0.8 where I can recreate the helpers refactoring cleanly?

Alternatively, if you'd prefer me to work within my constraints, I can try to identify and remove the duplicate changes, though this won't clean up the commit history.

pre-commit-ci bot and others added 2 commits December 6, 2025 09:10
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.6 → v0.14.7](astral-sh/ruff-pre-commit@v0.14.6...v0.14.7)
This previously combined old and new logic to result not only in buggy
behavior, but also a loss of valuable information.
We'll simplify by just not modifying the origin information.
@JacobCallahan JacobCallahan force-pushed the copilot/refactor-helpers-into-modules branch from 7d6c284 to 818ade0 Compare December 6, 2025 14:26
- Split the monolithic helpers.py file into separate modules by functionality:
  - dict_utils.py: Dictionary manipulation utilities
  - file_utils.py: File handling utilities
  - inventory.py: Inventory management utilities
  - results.py: Result and MockStub classes
  - misc.py: CLI and miscellaneous utilities
- Created broker/helpers/__init__.py to re-export all functions for backward compatibility.
- Updated logging configuration to use standard python logging.
- Addressed code review feedback regarding imports and type checking.
@JacobCallahan JacobCallahan force-pushed the copilot/refactor-helpers-into-modules branch from 818ade0 to 36e4e64 Compare December 6, 2025 14:32
@JacobCallahan JacobCallahan merged commit 7e8ee71 into 0.8 Dec 6, 2025
7 checks passed
@JacobCallahan JacobCallahan deleted the copilot/refactor-helpers-into-modules branch December 6, 2025 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor helpers

2 participants