Skip to content

refactor(benchmark): repricing filter logic #1810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: forks/osaka
Choose a base branch
from
+20 −7
Open

refactor(benchmark): repricing filter logic #1810

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/benchmarking.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,19 @@ def pytest_configure(config: pytest.Config) -> None:
def pytest_collection_modifyitems(
config: pytest.Config, items: list[pytest.Item]
) -> None:
"""Remove non-repricing tests when --fixed-opcode-count is specified."""
"""
Filter tests based on repricing marker when both -m repricing and
benchmark flags are specified.
"""
gas_benchmark_value = config.getoption("gas_benchmark_value")
fixed_opcode_count = config.getoption("fixed_opcode_count")
if not fixed_opcode_count:
# If --fixed-opcode-count is not specified, don't filter anything

if not gas_benchmark_value and not fixed_opcode_count:
return

# Check if -m repricing marker filter was specified
markexpr = config.getoption("markexpr", "")
if "repricing" not in markexpr:
return

filtered = []
Expand Down
5 changes: 5 additions & 0 deletions packages/testing/src/execution_testing/specs/benchmark.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,11 @@ def model_post_init(self, __context: Any, /) -> None:

blocks: List[Block] = self.setup_blocks

if self.fixed_opcode_count is not None and self.code_generator is None:
pytest.skip(
"Cannot run fixed opcode count tests without a code generator"
)

Comment on lines +253 to +257
Copy link
Collaborator Author

@LouisTsai-Csie LouisTsai-Csie Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the fixed opcode count feature is supported only by benchmark tests that use the code generator. If a test does not support this feature, it should be skipped.

There are two possible ways to filter such tests:

We could skip them during collection via pytest_collection_modifyitems, but this is more complex because it requires determining whether each test is a benchmark test and whether it uses the code generator.

Therefore, I chose to skip the unsupported tests directly in benchmark test wrapper instead.

if self.code_generator is not None:
# Inject fixed_opcode_count into the code generator if provided
self.code_generator.fixed_opcode_count = self.fixed_opcode_count
Expand Down
7 changes: 3 additions & 4 deletions tests/benchmark/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,10 @@ def pytest_collection_modifyitems(config: Any, items: Any) -> None:
return

marker_expr = config.getoption("-m", default="")

run_benchmarks = (
marker_expr
and "benchmark" in marker_expr
and "not benchmark" not in marker_expr
)
"benchmark" in marker_expr and "not benchmark" not in marker_expr
) or ("repricing" in marker_expr and "not repricing" not in marker_expr)
run_stateful_tests = (
marker_expr
and "stateful" in marker_expr
Expand Down
Loading