Skip to content

Add rev11 real-input AZI_CHUNK sweep benchmark, scoped rev12, and rev11-vs-rev12 validator#9

Open
nicklasorte wants to merge 1 commit intomainfrom
codex/create-chunk-sweep-benchmark-for-rev11
Open

Add rev11 real-input AZI_CHUNK sweep benchmark, scoped rev12, and rev11-vs-rev12 validator#9
nicklasorte wants to merge 1 commit intomainfrom
codex/create-chunk-sweep-benchmark-for-rev11

Conversation

@nicklasorte
Copy link
Copy Markdown
Owner

Motivation

  • Benchmark real-input performance of subchunk_agg_check_maxazi_rev11 across AZI_CHUNK values to choose a safe default for further optimization.
  • Implement a carefully scoped rev12 that preserves rev11's one-time RNG/pre-generated random matrices and chunking while addressing the next likely bottlenecks.
  • Provide an automated statistical validator to ensure rev12 remains statistically equivalent to rev11 and to measure runtime impact.

Description

  • Add benchmark_subchunk_agg_check_maxazi_rev11_chunk_sweep_real.m which times subchunk_agg_check_maxazi_rev11 on real inputs over chunk sizes [32 64 128 256 512 1024], uses timeit when available, prints a clean summary table, and returns chunk_sizes, runtimes, best_chunk, best_runtime, and speedup_vs_128.
  • Add subchunk_agg_check_maxazi_rev12.m which preserves the one-time RNG/pre-generated random matrices and AZI chunking (default AZI_CHUNK=128, optional override), vectorizes the off-axis gain lookup via a flattened nearestpoint_app pass, and reduces temporary allocation by accumulating in linear mW domain and converting once per MC.
  • Add validate_subchunk_agg_check_maxazi_rev11_rev12_statistical.m which runs rev11 and rev12 on identical inputs, captures runtimes, compares metrics (mean, std, min, max, median, p90, p95, p99, and optional p99.9), emphasizes upper-tail checks, applies documented absolute/relative thresholds, prints a structured summary, and fails closed if thresholds are exceeded.
  • Keep debug checks gated with DEBUG_CHECKS=false and avoid changing aggregation semantics or helper-function call patterns to preserve statistical behavior.

Testing

  • Performed repository-level inspections and sanity checks (files reviewed with sed/nl and git status --short showed the three new files present); these checks succeeded.
  • Attempted to run a CLI MATLAB substitute via octave --version which failed in this container, so the MATLAB/Octave runtime benchmarks and the statistical validation could not be executed here.
  • No runtime benchmark/validation tests were executed due to missing MATLAB/Octave; the new files are committed and ready to be run in an environment with the MATLAB runtime.

Codex Task

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eec404382a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

end
end

sub_array_agg_check_mc_dBm(loop_idx,1)=10.*log10(max(max_azi_agg_mw,realmin('double')));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve -Inf when aggregate power is zero

This conversion now clamps zero aggregate power to realmin before log10, which returns a large finite negative value (~-3076 dBm) instead of the -Inf produced by rev11 (pow2db(0)). In scenarios with no active BS contributions (or all contributions omitted as NaN), this changes the output contract from non-finite to finite and can alter downstream filtering/statistics that use isfinite to detect no-signal cases.

Useful? React with 👍 / 👎.

Comment on lines +62 to +64
finite_mask=isfinite(x11) & isfinite(x12);
x11=x11(finite_mask);
x12=x12(finite_mask);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat non-finite output mismatches as validation failures

The validator drops any sample where either rev11 or rev12 is non-finite before computing drift metrics, so regressions that introduce NaN/Inf (or contract changes in non-finite handling) are silently excluded from overall_pass. That undermines the stated fail-closed behavior because the comparison can pass while problematic samples are ignored.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant