Skip to content

feat(es/parser): rewrite parser with OXC architecture#12016

Closed
kdy1 wants to merge 141 commits into
swc-project:mainfrom
kdy1:kdy1/oxc-parser-rewrite-v2
Closed

feat(es/parser): rewrite parser with OXC architecture#12016
kdy1 wants to merge 141 commits into
swc-project:mainfrom
kdy1:kdy1/oxc-parser-rewrite-v2

Conversation

@kdy1

@kdy1 kdy1 commented Jul 11, 2026

Copy link
Copy Markdown
Member

Description:

Replace SWC's legacy lexer/parser with an independent OXC-style, source-oriented engine and make the new API the only parser entry point:

Parser::new(source, source_type)
    .with_options(options)
    .with_start_pos(start_pos)
    .parse()

The parser constructs the owned SWC AST directly. It does not depend on OXC crates, an OXC AST, or an intermediate arena AST. The source cursor, checkpoint, packed-token, cold-error, re-lexing, and recursive-descent organization are derived from oxc_parser commit b6d2a29e47358a288dbfb2a635550243511ec497.

This PR:

  • implements independent JS, JSX, TypeScript, TSX, Flow, and Hermes productions;
  • uses an OXC-style byte cursor, cold EOF path, byte-handler dispatch, source-backed literals, and parser-directed regex/JSX/template/TypeScript angle re-lexing;
  • stores public and internal tokens as the same PackedToken(u128) and statically separates NoTokens from WithTokens;
  • rewinds compact lexer/parser checkpoints instead of cloning token streams and parser state;
  • constructs SWC AST nodes directly from productions without an AST conversion pass;
  • collects flat source-order comments and attaches them deterministically at compiler boundaries;
  • adds finite parse-depth limits on every platform while retaining native stack growth;
  • migrates compiler, fast-strip, React compiler, bundler, minifier, AST viewer, wasm, and Node consumers;
  • removes the legacy lexer/parser implementation, compatibility entry points, and the abandoned swc_ecma_lexer crate;
  • adds JS/JSX/TS/TSX/Flow, Test262, and Hermes normalized-AST differential coverage;
  • adds cursor/rewind fuzz coverage and expands the CodSpeed corpus with Flow, minified, Unicode, numeric-heavy, and escape-heavy inputs;
  • includes the upstream OXC MIT license, migration documentation, and breaking changeset.

Performance is evaluated through CodSpeed CI. Local benchmark timing is intentionally not reported; local runs only validate benchmark harnesses with --test.

Validation completed locally:

  • 4,397/4,397 normalized valid-fixture AST parity before legacy removal;
  • Test262 valid 1,928 passed and invalid 703 passed, excluding the repository's existing ignored cases;
  • JS 165, JSX 90, TypeScript 4,958, Flow 40, Hermes, comments, diagnostics, and parser API suites;
  • cargo test -p swc_ecma_parser and release-only prefix-consumption regression coverage;
  • cargo clippy --all --all-targets -- -D warnings and cargo fmt --all;
  • wasm32 and aarch64 parser checks plus fuzz-target compilation;
  • all four wasm binding scripts, including TypeScript default and nightly modes;
  • packages/core: 40 test files and 117 tests passed;
  • parser and fast-strip CodSpeed harnesses in --test mode.

BREAKING CHANGE:

The parser API is now Parser::new(&str, SourceType).parse() -> ParserReturn. Parser diagnostics, exact spans, comment attachment, and lexer/parser token APIs are not compatibility guarantees. The legacy parser and swc_ecma_lexer crate are removed. See crates/swc_ecma_parser/MIGRATION.md for migration details.

@changeset-bot

changeset-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: a99b530

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@socket-security

socket-security Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedcargo/​libfuzzer-sys@​0.4.134710093100100

View full report

@socket-security

socket-security Bot commented Jul 11, 2026

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
Obfuscated code: cargo libc is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?cargo/libfuzzer-sys@0.4.13cargo/libc@0.2.186

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/libc@0.2.186. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: cargo zerocopy is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?cargo/hashbrown@0.13.2cargo/hashbrown@0.14.5cargo/browserslist-rs@0.19.0cargo/lightningcss@1.0.0-alpha.68cargo/zerocopy@0.8.54

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/zerocopy@0.8.54. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

@kdy1 kdy1 changed the title feat(es/parser): add OXC-style parser API feat(es/parser): rewrite parser with OXC architecture Jul 13, 2026
@kdy1

kdy1 commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Closing as the gpt-5.6-sol experiment is over.

@kdy1 kdy1 closed this Jul 13, 2026
@github-actions github-actions Bot added this to the Planned milestone Jul 13, 2026
@kdy1 kdy1 deleted the kdy1/oxc-parser-rewrite-v2 branch July 13, 2026 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant