Skip to content

perf: cut per-document overheads of pooled decoders#1099

Open
pelletier wants to merge 1 commit into
v2from
perf/decoder-overheads
Open

perf: cut per-document overheads of pooled decoders#1099
pelletier wants to merge 1 commit into
v2from
perf/decoder-overheads

Conversation

@pelletier

Copy link
Copy Markdown
Owner

Splitting #1088: this PR carries three small per-document costs of the pooled decoder in one reviewable unit.

What

  • Decoder.Decode read its input through io.ReadAll's growth sequence. Readers that know their size (bytes.Reader, strings.Reader, bytes.Buffer, anything with Len() int) now get an exactly-sized buffer up front, with a ReadAll fallback for readers that outgrow their announced size.
  • Resetting a pooled decoder walked the array-table count map to zero every slot. Slots are now stamped with a generation (acSlot), and a single counter bump invalidates all of them in O(1); stale slots read as zero.
  • Every struct table lookup paid an interface-hash sync.Map access for its struct plan. A small MRU memo on the decoder (8 entries) catches the handful of struct types a document actually uses.

Impact

Benchmarked on a dedicated linux/amd64 spot VM (t2d), go1.26.4, interleaved A/B vs the base of this PR, benchstat over 10 samples per side:

  • sec/op geomean: -0.18%
  • Unmarshal/SimpleDocument/struct: -6.07%
  • RealWorldContainerdConfig: -3.58%
  • RealWorldHugoFrontMatterBatch: -2.86%
  • Unmarshal/HugoFrontMatter: -2.78%
  • Unmarshal/SimpleDocument/map: -2.61%
  • UnmarshalDataset/code: -2.47%
  • RealWorldGolangciStrict: +3.60%
  • Unmarshal/ReferenceFile/struct: +9.56%
  • allocs/op geomean: -0.09%
  • allocs RealWorldGolangciStrict: -2.08%
Full benchstat (sec/op, allocs/op)

sec/op

UnmarshalDataset/config  10.38m ± 1%  10.43m ± 1%  ~ (p=0.218 n=10)
UnmarshalDataset/canada  23.64m ± 1%  23.96m ± 2%  ~ (p=0.123 n=10)
UnmarshalDataset/citm_catalog  15.02m ± 1%  15.14m ± 1%  ~ (p=0.075 n=10)
UnmarshalDataset/twitter  3.582m ± 0%  3.589m ± 1%  ~ (p=0.579 n=10)
UnmarshalDataset/code  34.55m ± 2%  33.70m ± 1%  -2.47% (p=0.000 n=10)
UnmarshalDataset/example  77.22µ ± 1%  77.34µ ± 0%  ~ (p=0.971 n=10)
Unmarshal/SimpleDocument/struct  332.2n ± 1%  312.1n ± 1%  -6.07% (p=0.000 n=10)
Unmarshal/SimpleDocument/map  425.6n ± 1%  414.5n ± 1%  -2.61% (p=0.000 n=10)
Unmarshal/ReferenceFile/struct  39.48µ ± 4%  43.25µ ± 5%  +9.56% (p=0.000 n=10)
Unmarshal/ReferenceFile/map  31.51µ ± 1%  31.33µ ± 2%  ~ (p=0.315 n=10)
Unmarshal/HugoFrontMatter  5.659µ ± 1%  5.501µ ± 2%  -2.78% (p=0.000 n=10)
Marshal/SimpleDocument/struct  398.8n ± 2%  400.4n ± 3%  ~ (p=0.643 n=10)
Marshal/SimpleDocument/map  595.2n ± 0%  597.6n ± 1%  ~ (p=0.101 n=10)
Marshal/ReferenceFile/struct  20.63µ ± 1%  20.82µ ± 2%  ~ (p=0.481 n=10)
Marshal/ReferenceFile/map  40.11µ ± 3%  40.26µ ± 3%  ~ (p=0.579 n=10)
Marshal/HugoFrontMatter  8.009µ ± 2%  7.911µ ± 3%  ~ (p=0.897 n=10)
RealWorldContainerdConfig  40.30µ ± 1%  38.86µ ± 2%  -3.58% (p=0.000 n=10)
RealWorldViperRead  8.297µ ± 1%  8.279µ ± 1%  ~ (p=0.469 n=10)
RealWorldViperWrite  12.37µ ± 4%  12.66µ ± 4%  ~ (p=0.971 n=10)
RealWorldHugoFrontMatterBatch  94.91µ ± 1%  92.19µ ± 1%  -2.86% (p=0.000 n=10)
RealWorldGitleaksRules  64.55µ ± 1%  64.31µ ± 1%  ~ (p=0.143 n=10)
RealWorldPyproject  14.94µ ± 1%  14.79µ ± 1%  ~ (p=0.105 n=10)
RealWorldGolangciStrict  11.84µ ± 0%  12.27µ ± 2%  +3.60% (p=0.000 n=10)
geomean  46.69µ  46.61µ  -0.18%

allocs/op

UnmarshalDataset/config  70.19k ± 0%  70.19k ± 0%  ~ (p=1.000 n=10) ¹
UnmarshalDataset/canada  223.2k ± 0%  223.2k ± 0%  ~ (p=1.000 n=10) ¹
UnmarshalDataset/citm_catalog  49.98k ± 0%  49.98k ± 0%  ~ (p=1.000 n=10) ¹
UnmarshalDataset/twitter  15.78k ± 0%  15.78k ± 0%  ~ (p=1.000 n=10) ¹
UnmarshalDataset/code  129.5k ± 0%  129.5k ± 0%  ~ (p=1.000 n=10) ¹
UnmarshalDataset/example  399.0 ± 0%  399.0 ± 0%  ~ (p=1.000 n=10) ¹
Unmarshal/SimpleDocument/struct  2.000 ± 0%  2.000 ± 0%  ~ (p=1.000 n=10) ¹
Unmarshal/SimpleDocument/map  5.000 ± 0%  5.000 ± 0%  ~ (p=1.000 n=10) ¹
Unmarshal/ReferenceFile/struct  83.00 ± 0%  83.00 ± 0%  ~ (p=1.000 n=10) ¹
Unmarshal/ReferenceFile/map  194.0 ± 0%  194.0 ± 0%  ~ (p=1.000 n=10) ¹
Unmarshal/HugoFrontMatter  54.00 ± 0%  54.00 ± 0%  ~ (p=1.000 n=10) ¹
Marshal/SimpleDocument/struct  4.000 ± 0%  4.000 ± 0%  ~ (p=1.000 n=10) ¹
Marshal/SimpleDocument/map  4.000 ± 0%  4.000 ± 0%  ~ (p=1.000 n=10) ¹
Marshal/ReferenceFile/struct  4.000 ± 0%  4.000 ± 0%  ~ (p=1.000 n=10) ¹
Marshal/ReferenceFile/map  91.00 ± 0%  91.00 ± 0%  ~ (p=1.000 n=10) ¹
Marshal/HugoFrontMatter  20.00 ± 0%  20.00 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldContainerdConfig  144.0 ± 0%  144.0 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldViperRead  65.00 ± 0%  65.00 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldViperWrite  33.00 ± 0%  33.00 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldHugoFrontMatterBatch  820.0 ± 0%  820.0 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldGitleaksRules  259.0 ± 0%  259.0 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldPyproject  142.0 ± 0%  142.0 ± 0%  ~ (p=1.000 n=10) ¹
RealWorldGolangciStrict  48.00 ± 0%  47.00 ± 0%  -2.08% (p=0.000 n=10)
geomean  211.1  210.9  -0.09%
¹ all samples are equal

Notes: RealWorldGolangciStrict decodes into a config with more struct types than the memo holds; the memo uses a transpose-on-hit / rotating-overwrite scheme so that this worst case degrades to a few pointer comparisons (+3.6% here) instead of an array shift per lookup, and it nets out negative once the rest of the series lands (−1.75% in the combined branch). The Unmarshal/ReferenceFile/struct row is alignment noise on this host — the function it exercises is untouched apart from the memo indirection, and the combined branch has it at −4.95%.

🤖 Generated with Claude Code

Three per-document costs on the decoder hot path:

- Decoder.Decode read its input through io.ReadAll's growth sequence.
  Readers that know their size (bytes.Reader, strings.Reader,
  bytes.Buffer, ...) now get an exactly-sized buffer up front.
- Resetting a pooled decoder walked the array-table count map to zero
  every slot. Slots are now stamped with a generation and a single
  counter bump invalidates all of them in O(1).
- Every struct table lookup paid an interface-hash sync.Map access for
  its struct plan. A small MRU memo on the decoder catches the handful
  of types a document actually uses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pelletier pelletier added the performance Issue related to a performance problem or pull request improving performance. label Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Issue related to a performance problem or pull request improving performance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant