Repository navigation
feat(benchmarks): compare packages by scenario - #37
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors the benchmark harness to be scenario-first and compares all five published packages for each scenario, validating that each target produces equivalent layout output (relative to yoga-layout) before accepting timing results. It also updates the website renderer and benchmark documentation to match the new results schema (v2) and presentation model.
Changes:
- Replace API-grouped benchmark suites with a unified scenario list and a unified target list (package/API/runtime rows per scenario).
- Switch benchmark correctness validation from a single checksum to per-node layout observations (
left/top/width/height) compared againstyoga-layoutwith a tolerance. - Update published results schema to v2 and update the website benchmark page/table rendering accordingly.
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| benchmarks/worker.ts | Runs per-target scenario transactions, captures validation observations, and times the scenario using Tinybench. |
| benchmarks/run.ts | Orchestrates scenario-first execution across all targets, validates equivalence vs yoga-layout, and writes schema v2 reports. |
| benchmarks/scenario.ts | Introduces layout observation primitives and updates scenario/target interfaces for unified scenario-first benchmarking. |
| benchmarks/suite.ts | Defines the new unified benchmarkTargets, benchmarkBaselineTargetId, and benchmarkScenarios. |
| benchmarks/coding-agent-chat/benchmark.ts | Reworks the chat benchmark into shared scenario definitions with both Taffy and Yoga transaction builders (initial + resize). |
| benchmarks/nested-ui/benchmark.ts | Adds deterministic nested-UI scenarios with explicit Taffy and Yoga implementations. |
| benchmarks/wide-wrapping-collection/benchmark.ts | Adds a wide wrapping collection scenario with both Taffy and Yoga implementations. |
| benchmarks/README.md | Updates benchmark workspace documentation to match the scenario-first model and validation approach. |
| benchmarks/yoga-wide-tree/benchmark.ts | Removes legacy Yoga-only “wide tree” scenario registration (superseded by scenario-first suite). |
| benchmarks/yoga-deep-tree/benchmark.ts | Removes legacy Yoga-only “deep tree” scenario registration (superseded by scenario-first suite). |
| benchmarks/yoga-coding-agent-chat/benchmark.ts | Removes legacy Yoga-only chat benchmark file (now unified into scenario-first chat scenarios). |
| benchmarks/wide-tree/benchmark.ts | Removes legacy Taffy-only “wide tree” scenario registration (replaced by scenario-first suite). |
| benchmarks/deep-tree/benchmark.ts | Removes legacy Taffy-only “deep tree” scenario registration (replaced by scenario-first suite). |
| apps/website/benchmarks/index.md | Updates benchmarks page copy to reflect scenario-first results and yoga-layout baseline. |
| apps/website/.vitepress/config.ts | Renames the Benchmarks nav item label. |
| apps/website/.vitepress/components/BenchmarkResults.vue | Updates website renderer for schema v2 and scenario-first tables (Package/API/Runtime columns). |
| .agents/docs/README.md | Updates PCR index entry wording for benchmark design changes. |
| .agents/docs/benchmark.md | Updates benchmark design PCR to reflect scenario-first comparisons, targets, and validation rules. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+399
to
+407
| return { | ||
| run(): LayoutObservation { | ||
| const viewport = viewports[viewportIndex]; | ||
| viewportIndex = (viewportIndex + 1) % viewports.length; | ||
| fixture.tree.updateStyle(fixture.root, { size: viewport }); | ||
| fixture.compute(viewport.width, viewport.height); | ||
| return readTaffyLayouts(fixture.tree, fixture.nodes, observation); | ||
| }, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
The benchmark should measure the actual cost of completing the same semantic layout transaction through each package's public JavaScript API. This makes package, API abstraction, and runtime tradeoffs visible without duplicating scenarios or assuming which implementation must win.
Validation
vp checkvp run benchmark:update-websitevp run build:website/benchmarks/with no console errorsThe committed publication data was recorded on Node v24.19.0, Linux x64, Intel Core i5-13500H.