DRAFT: custom lexer in ecma parser.

[bvanjoi]

This is a draft PR still under active development. I'm primarily verifying CI is green at this stage - please hold off on reviewing until marked ready.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 40b4e06

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codspeed-hq]

CodSpeed Performance Report

Merging #10385 will degrade performances by 18.81%

_{Comparing bvanjoi:rm-token-contexts (40b4e06) with main (f802892)}

Summary

⚡ 1 improvements
❌ 23 regressions
✅ 128 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
❌	es/codegen/with-parser/colors	188.6 µs	192.9 µs	-2.26%
❌	es/lexer/angular	12.2 ms	14.5 ms	-15.75%
❌	es/lexer/backbone	1.7 ms	2 ms	-17.95%
❌	es/lexer/cal-com	33.8 ms	40.4 ms	-16.18%
❌	es/lexer/colors	44.2 µs	49.5 µs	-10.7%
❌	es/lexer/jquery	8.8 ms	10.8 ms	-18.34%
❌	es/lexer/jquery mobile	13.6 ms	16.5 ms	-17.77%
❌	es/lexer/mootools	7.1 ms	8.8 ms	-18.81%
❌	es/lexer/three	43.2 ms	52.1 ms	-17.18%
❌	es/lexer/typescript	236.2 ms	281.1 ms	-15.96%
❌	es/lexer/underscore	1.4 ms	1.7 ms	-18.3%
❌	es/lexer/yui	7.2 ms	8.7 ms	-17.08%
❌	es/parser/angular	29.6 ms	30.5 ms	-2.98%
❌	es/parser/backbone	4.6 ms	4.7 ms	-3.18%
⚡	es/parser/cal-com	90,419.6 µs	325.3 µs	×280
❌	es/parser/colors	122.1 µs	125.3 µs	-2.59%
❌	es/parser/jquery	23.7 ms	24.5 ms	-3.25%
❌	es/parser/jquery mobile	36.4 ms	37.6 ms	-3.23%
❌	es/parser/mootools	18.8 ms	19.5 ms	-3.38%
❌	es/parser/three	112.3 ms	115.7 ms	-2.89%
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.