Skip to content

perf: fix top 5 high-performance-go violations (struct layout + allocations) - #833

Merged
jeduden merged 7 commits into
mainfrom
claude/wonderful-curie-yx2ugc
Sep 4, 2026
Merged

perf: fix top 5 high-performance-go violations (struct layout + allocations)#833
jeduden merged 7 commits into
mainfrom
claude/wonderful-curie-yx2ugc

Conversation

@jeduden

@jeduden jeduden commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

Scheduled performance audit against
docs/development/high-performance-go.md.
Three background agents scanned internal/ and pkg/ for allocation-budget
violations, "Patterns to avoid" anti-patterns, and struct-layout/data-structure
issues. The codebase turned out to already be heavily scrubbed for these
patterns (most candidates the agents flagged were already-correct, documented
optimizations); the five real, safely-fixable violations below were selected
and fixed with red/green TDD, each in its own commit:

  1. internal/rules/linkvalidity (MDS062, default-on)revMatch
    declared its two ints (col0, matchEnd) before its two byte slices
    (text, url). Go's GC ptrdata spans from offset 0 through the last
    pointer-containing field, so with scalars first, ptrdata still had to
    cover both slices anyway. Grouping the slices first and the scalars last
    shrinks ptrdata from 48 to 32 bytes — the largest win in this pass.
  2. internal/rules/uniquefrontmatter (MDS069, default-on)pathEntry
    sandwiched an int between two strings. pathEntry is stored once per
    violation in the run-wide scopeIndex.byPath index, so the 8-byte ptrdata
    saving scales with corpus size.
  3. internal/rules/samefileanchor (MDS070, default-on)anchorChecker
    declared a bool before its trailing []lint.Diagnostic. Moved after,
    saving 8 bytes of ptrdata per file that contains a #.
  4. internal/rules/noinlinehtml (MDS041)extractTag built the tag
    name via strings.ToLower(string(m[1])): one allocation for the
    string(m[1]) copy, plus a second whenever the tag had an uppercase byte
    (since strings.ToLower can't return the input unchanged). tagNameRe
    only ever matches ASCII, so the new asciiLowerTag lowercases the matched
    bytes directly into one strings.Builder allocation — a mixed-case tag
    drops from 3 allocs/call to 2.
  5. internal/rules/tableformat (MDS025, default-on, per-file)
    checkColumnCount unconditionally called
    make([]lint.Diagnostic, 0, len(t.rows)-1), then discarded it and
    returned nil on the common column-count-compliant path. Now starts
    diags as nil and grows it lazily via append, matching the project's
    own "return nil, not []T{}" convention — one fewer allocation per
    compliant table, on every workspace file.

Each fix carries a dedicated test (a structlayout.AssertPointerFieldsFirst
layout assertion matching the pattern in tablereadability, propernames,
astutil, or a testing.AllocsPerRun allocation budget) that was verified
red against the pre-fix code (via git stash, or a direct diff against the
pre-PR commit) before the fix made it green.

MDS025's full fix (merging the structure pass with tablefmt's own
alignment scan) is a larger, already-documented "single-table-walk refactor"
(see the comment in internal/rules/tableformat/alloc_test.go) that this PR
does not attempt — it's out of scope for a set of small, safely verifiable
fixes.

Code review: three xhigh-severity /code-review passes, as required.
Round 1: no findings. Round 2: three findings — two were valid (the three
new layout tests used hardcoded, pointer-size-dependent unsafe.Offsetof
constants instead of the repo's existing portable
structlayout.AssertPointerFieldsFirst helper; a //nolint:prealloc pragma
on checkColumnCount suppressed nothing, confirmed empirically) and were
fixed in a follow-up commit; the third (a claimed ptrdata-savings miscalculation
in a comment) was independently verified against Go's actual PtrDataSize
algorithm and found to be incorrect — the original 48→32 byte figure was
correct, so no change was made there. Round 3 (after the fix commit):
no findings.

Test plan

  • go build ./...
  • go vet ./...
  • go test ./... (full suite green)
  • go tool -modfile=tools/go.mod golangci-lint run (0 issues, repo-wide)
  • go test ./internal/integration/... -run TestPerRuleAllocBudget (per-rule alloc gate green)
  • mdsmith check . (584 files, 0 failures)
  • Each fix's test verified red (via git stash/pre-fix diff) before green
  • Three xhigh-severity /code-review passes, findings addressed

🤖 Generated with Claude Code

https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb

@jeduden jeduden added performance routine Created by a routine labels Sep 2, 2026 — with Claude
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.79%. Comparing base (f146892) to head (dfe995e).
⚠️ Report is 26 commits behind head on main.

Additional details and impacted files
Components Coverage Δ
Go 98.78% <100.00%> (+0.01%) ⬆️
TypeScript 99.54% <ø> (ø)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

MDS070's anchorChecker declared built (a bool) before diags (a
[]lint.Diagnostic), forcing the GC ptrdata to scan 8 bytes it did
not need to. Moving built after diags — all pointer-containing
fields first, scalars last — shrinks ptrdata to just what the
pointer fields need. Per docs/development/high-performance-go.md
"Struct layout".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
MDS069's pathEntry declared line (an int) between the value and
firstPath strings, forcing GC ptrdata to scan past it. Moving line
after firstPath — pointer fields first, scalars last — shrinks
ptrdata by one word. pathEntry is stored once per violation in the
run-wide scopeIndex.byPath, so the saving scales with corpus size.
Per docs/development/high-performance-go.md "Struct layout".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
MDS062's revMatch declared col0/matchEnd (two ints) before the
text/url byte slices, so GC ptrdata had to span both scalars plus
both slices. Grouping the pointer-containing slices first and the
scalars last shrinks ptrdata from 48 to 32 bytes — the largest of
this pass's three struct-layout fixes. Per
docs/development/high-performance-go.md "Struct layout".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
MDS041's extractTag built the tag-name string via
strings.ToLower(string(m[1])): the string(m[1]) copy, then a second
buffer from strings.ToLower whenever the tag had an uppercase byte.
tagNameRe only ever matches ASCII, so asciiLowerTag lowercases the
matched bytes directly into a single strings.Builder allocation,
cutting a mixed-case tag from 3 allocs/call to 2. Per
docs/development/high-performance-go.md "Allocations".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
MDS025's checkColumnCount always called make([]lint.Diagnostic, 0,
len(t.rows)-1) up front, even though most tables are column-count
compliant and the function then discards that slice and returns nil.
Starting diags as nil and growing it lazily via append matches the
project's own "return nil, not []T{}" convention and drops one
allocation per compliant table — the common case on every workspace
file with well-formed tables. Per
docs/development/high-performance-go.md "Allocations".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
- Switch the three new layout tests (linkvalidity, samefileanchor,
  uniquefrontmatter) from hardcoded unsafe.Offsetof byte constants to
  the project's existing internal/structlayout.AssertPointerFieldsFirst
  helper, matching the pattern already used by tablereadability,
  propernames, and astutil. This asserts the same invariant (pointer
  fields precede scalars) without hardcoding pointer/slice/string
  header sizes that only hold on 64-bit platforms.
- Drop the //nolint:prealloc pragma added to checkColumnCount: the
  prealloc linter does not flag this conditional-append pattern (the
  unmodified, identically-shaped checkPipeStyle has no such pragma
  either), so the directive suppressed nothing and only misled a
  reader into thinking it did. The explanatory comment stays as a
  plain comment.

Round 2 of 3 code-review passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
@jeduden
jeduden force-pushed the claude/wonderful-curie-yx2ugc branch from c306486 to e66c831 Compare September 4, 2026 17:36

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

Scheduled code-review pass complete — no code changes needed.

  • Rebased the branch onto the current main (3 commits behind, all docs/plan-only — no conflicts) and force-with-lease pushed.
  • Ran /code-review at high severity against the full diff: no findings. (/mdsmith-security-review was skipped — this diff is struct field reordering and allocation micro-optimizations in internal rule code; it doesn't touch untrusted-input parsing, directive resolution, or build-recipe linting, so it isn't a security-relevant surface.)
  • Validated post-rebase: go build ./..., go vet ./..., go test ./... (full suite green), go tool -modfile=tools/go.mod golangci-lint run (0 issues), mdsmith check . (585 files, 0 failures).

This PR already went through three xhigh-severity review rounds before this pass (see the PR description); this check-in found nothing new.


Generated by Claude Code

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

Security review — no findings

Reviewed against the mdsmith threat model (PR/diff mode). This is a behavior-preserving performance change with no trust-boundary impact.

  • Struct field reordering (revMatch, anchorChecker, pathEntry) is purely a GC-ptrdata layout optimization — field semantics and values are unchanged.
  • asciiLowerTag replaces strings.ToLower(string(m[1])); it only ever runs on bytes matched by tagNameRe ([a-zA-Z][a-zA-Z0-9-]*), which are ASCII, so the fold is equivalent with no Unicode edge cases.
  • checkColumnCount starts diags at nil and grows lazily instead of eager make(...) — same output, including nil on a compliant table.

No recipe/exec, path-resolution, include/catalog, or Git-integration code touched (§0/§1/§7 unaffected). No §0 defense regressed. Signed off from a security standpoint.


Generated by Claude Code

@jeduden
jeduden marked this pull request as ready for review September 4, 2026 22:35
Copilot AI lite review requested due to automatic review settings September 4, 2026 22:35
@jeduden jeduden added the queue Add to a PR to enqueue it label Sep 4, 2026 — with Claude
@jeduden jeduden added queue:active Applied automatically when a PR is in an active batch and removed queue Add to a PR to enqueue it labels Sep 4, 2026
@jeduden

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

🟢 Merge Queue — picked up

This PR is in the queue and will be batched with other queue-labelled PRs.

Next: No action needed — you'll get another comment when CI starts on the batch. View merge queue run.

jeduden pushed a commit that referenced this pull request Sep 4, 2026
@jeduden

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

🔵 Merge Queue — CI running

Merged into batch branch merge-queue/batch-827-1788561366 alongside #827, #828. View CI run.

Next: No action needed — you'll be notified when CI completes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are low-risk performance improvements with targeted tests added to prevent regressions.

Pull request overview

This PR applies a focused performance-audit pass (per docs/development/high-performance-go.md) to reduce GC ptrdata scanning and avoid unnecessary allocations in several default-on rules, adding targeted tests to keep those optimizations from regressing.

Changes:

  • Reordered struct fields in a few hot structs to place pointer-containing fields first (shrinking GC ptrdata) and added structlayout.AssertPointerFieldsFirst tests to pin the layout.
  • Removed an eager slice allocation on the common “no diagnostics” path in MDS025 column-count checking, and added an alloc test to ensure the compliant path stays allocation-free.
  • Replaced strings.ToLower(string(submatch)) in MDS041 tag extraction with an ASCII-specific lowercasing helper and added an allocation budget test.
File summaries
File Description
internal/rules/uniquefrontmatter/rule.go Reorders pathEntry fields to reduce GC ptrdata.
internal/rules/uniquefrontmatter/layout_test.go Adds layout test enforcing pointer-fields-first for pathEntry.
internal/rules/tableformat/structure.go Avoids allocating diagnostics slice on compliant tables in checkColumnCount.
internal/rules/tableformat/checkcolumncount_alloc_test.go Adds alloc test ensuring compliant checkColumnCount stays at 0 allocs/op and returns nil.
internal/rules/samefileanchor/rule.go Reorders anchorChecker fields to reduce GC ptrdata.
internal/rules/samefileanchor/layout_test.go Adds layout test enforcing pointer-fields-first for anchorChecker.
internal/rules/noinlinehtml/rule.go Introduces ASCII-only lowercasing helper to reduce allocations in extractTag.
internal/rules/noinlinehtml/race_on_test.go Adds raceEnabled constant for alloc-test gating under -race.
internal/rules/noinlinehtml/race_off_test.go Adds raceEnabled constant for non-race builds.
internal/rules/noinlinehtml/alloc_test.go Adds allocation budget test for extractTag on uppercase tags.
internal/rules/linkvalidity/rule.go Reorders revMatch fields to reduce GC ptrdata.
internal/rules/linkvalidity/layout_test.go Adds layout test enforcing pointer-fields-first for revMatch.
Review details
  • Files reviewed: 12/12 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/rules/tableformat/structure.go
Copilot review on PR #833: starting diags as a bare nil slice kept
the zero-alloc compliant-table path, but a table with several
column-count mismatches paid growslice's doubling reallocations on
top of that (10 allocs/op for 6 mismatches). Presizing to
len(t.rows)-1 lazily, on the first mismatch, keeps the compliant
path at 0 allocs while cutting the many-mismatch path to 7 — one
alloc for the slice plus one per diagnostic message. Mirrors
reversedInLine's lazy presize in internal/rules/linkvalidity.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RcWK6hTskkM6F6qVTpnWJb
@jeduden
jeduden requested a lite review from Copilot September 4, 2026 22:41
@jeduden jeduden added queue:attempt-1 queue Add to a PR to enqueue it and removed queue:active Applied automatically when a PR is in an active batch labels Sep 4, 2026
@jeduden

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

Merge Queue — requeued

The merge queue hit a transient error while processing this PR:

new commits were pushed to this PR while batch CI ran; the stale batch result was discarded and the new head will be re-tested

View merge queue run.

Next: No action needed — the queue will retry automatically on the next run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are narrowly scoped, performance-motivated, and each behavior change is protected by targeted layout/allocation tests with no issues found in review.

Review details
  • Files reviewed: 12/12 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@jeduden jeduden added queue:active Applied automatically when a PR is in an active batch and removed queue Add to a PR to enqueue it labels Sep 4, 2026
@jeduden

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

🔵 Merge Queue — CI running

Merged into batch branch merge-queue/batch-827-1788561836 alongside #827, #828. View CI run.

Next: No action needed — you'll be notified when CI completes.

@jeduden
jeduden merged commit 89031ce into main Sep 4, 2026
32 checks passed
@jeduden jeduden removed queue:active Applied automatically when a PR is in an active batch queue:attempt-1 labels Sep 4, 2026
@jeduden

jeduden commented Sep 4, 2026

Copy link
Copy Markdown
Owner Author

Merge Queue — merged

This PR landed on main via commit 89031ce. CI run that validated the merge.

Next: Done — nothing more to do here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance routine Created by a routine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants