Skip to content

Benchmark and optimize JSONSchema validation hot paths #164

Description

@ajevans99

Summary

Companion to #162 (OrderedJSON parser/serializer perf). This issue tracks performance work on the validation hot paths in JSONSchema itself — Schema.validate(_:) and the keyword evaluators it dispatches to.

Background

The validator currently prioritizes correctness over throughput. After #149 (deterministic emission/output), #159 (OrderedJSON parser), and #161 (direct jsonValue accessor), the API surface is in a stable shape — which means it's now the right time to look at where validation actually spends its cycles.

We don't have measured baselines yet, so this issue is intentionally light on prescriptions and heavy on the "things worth measuring" list. Optimization decisions get made from data, not from intuition.

Suspected hot paths (worth instrumenting)

1. Regex compilation in pattern and patternProperties

Each call to Pattern.validate and PatternProperties.validate currently constructs a Regex<AnyRegexOutput> from the schema's pattern string. For schemas validated against many instances (the common case — load schema once, validate thousands of payloads), recompiling the regex per validate(_:) is pure waste. Cache compiled regexes inside the keyword instance.

2. anyOf / oneOf short-circuiting

anyOf should return on first success. oneOf has to count matches but can short-circuit on second match. Worth verifying the current implementation actually does — and worth measuring for schemas with deep, expensive subschemas where one branch matching trivially saves a lot of work.

3. $ref / $dynamicRef resolution

Reference resolution walks documentCache and (sometimes) documentDynamicAnchors, both [URL: …] dictionaries keyed on URL. URL hashing in Swift is non-trivial; if reference-heavy schemas show up hot, consider keying on a precomputed string form.

4. Properties.validate instance-key iteration

Walks every key in the instance and looks up the corresponding subschema. With OrderedDictionary (post-#157 for the schema map), lookups are still O(1) — but allocation in the hot loop (subAnnotations container, ValidationResultBuilder) might dominate for instances with hundreds of keys.

5. AnnotationContainer.merge

Phase 2 of #149 made the merge order-preserving. The merge itself is O(n) where n is the source container's count. For instances with deep applicator trees (allOf / anyOf chains), merges happen at every level. Worth looking at whether we're over-allocating intermediate containers.

6. ValidationResult construction

Each leaf produces a ValidationResult even when validation succeeds and no errors/annotations are recorded. Lots of small struct allocations in the success path. Consider whether a "fast path" that returns a shared .valid constant for empty cases is worth the complexity.

7. JSONPointer construction

Every keyword invocation appends to instanceLocation and keywordLocation. JSONPointer.appending(_:) likely allocates. For deep traversals, this adds up. Worth checking if a small-buffer optimization or reference-counted segment list would help.

Proposed approach

Phase 1 — Wire up benchmarks

Reuse the package-benchmark setup from #162 (assuming that lands first). Add a Benchmarks/JSONSchema/ target with:

  • Single-instance throughput: load a representative schema (Poll, OpenAPI fragment, JSON Schema meta-schema) and time validating 10k instances against it
  • Schema-construction throughput: time Schema.init(rawSchema:) for the same schemas
  • Output-rendering throughput: time validate(_:output:) for each output level (Flag/Basic/Detailed/Verbose) — output rendering is its own cost

Phase 2 — Profile

Run benchmarks under Instruments (or package-benchmark's allocation counter) and identify which of the suspected hot paths above actually show up. Don't optimize what isn't slow.

Phase 3 — Targeted fixes

For each hot path the data confirms, ship a focused PR with before/after numbers in the description. Each fix is independently reviewable.

Out of scope

  • Beating ajv (the JS reference) or other established validators. Different language, different runtime model. We just want JSONSchema's own throughput to be a credible story.
  • Algorithmic spec changes. We're validating per RFC; no shortcuts that would break correctness.
  • Validating in parallel. Schema validation is per-instance and embarrassingly parallel from the caller's side — adding internal parallelism inside one validate call has poor ROI.

Acceptance criteria

  • Benchmarks/JSONSchema/ executable target with at least 3 representative schemas
  • Baselines committed for each
  • CI surfaces validation benchmarks in the same PR-summary mechanism as Benchmark and optimize OrderedJSON parser/serializer #162
  • At least 2 hot paths profiled with Instruments, with findings summarized in a follow-up PR description
  • At least 1 measured optimization landed (anything north of 10% on at least one benchmark)

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions