TS/JS port of fli library - #172
Merged
Merged
Conversation
1:1 port of the Python fli library's core (models, core, search) to TypeScript, exposed as a Bun-managed npm package at fli-js/. The CLI and MCP server are intentionally out of scope. Architecture ------------ - Native fetch HTTP client (Bun built-in) with Chrome-like headers, TokenBucketRateLimiter (10 req/s), exponential-backoff retries, typed SearchClientError family, and HTTPS_PROXY/HTTP_PROXY env support — replaces curl_cffi's TLS impersonation with header + proxy-based configuration. - Promise-based concurrency: TokenBucketRateLimiter is an async-await variant; parallelMap is a bounded Promise.all with FIFO worker queue. - Wire format: UTF-8 byte-accurate iterWrbChunks (single + multi chunk), hand-rolled varint encoder/decoder reproduces a captured live booking token byte-for-byte. - Models use const-object enum pattern (Airport.JFK === "JFK") with generated airport.ts / airline.ts from data/*.csv via scripts/generate-enums.ts. Zod schemas drive validation for the filter input types. Tooling ------- - Bun as package manager + test runner. - Biome for formatting and base lint, oxlint as the secondary lint pass. - TypeScript strict mode + noUncheckedIndexedAccess. - Script entry points (bun run ...): generate:enums, typecheck, format, lint, lint:fix, test, test:e2e, ci. CI -- - New .github/workflows/fli-js.yml runs format-check, biome + oxlint, typecheck, and bun test on changes to fli-js/ or data/. Verifies generated enums stay in sync with data/*.csv. - .github/workflows/test.yml gains an fli-js job that calls the new reusable workflow alongside the existing Python jobs. Tests ----- 252 tests across 17 files (unit + integration + gated e2e): - Unit: wire, proto (byte-perfect captured-token reproduction), URL locale params, helpers, parsers, currency token extraction, airport search, decoders, concurrency primitives, HTTP client (stubbed). - Integration: FlightSearchFilters.format() / DateSearchFilters.format() snapshot tests verify byte-identical request payloads vs the Python upstream's test fixtures; SearchFlights/SearchDates run end-to-end against a stubbed fetch; public-API smoke test of the top-level barrel. - E2E: tests/e2e/live_search.test.ts is gated behind FLI_E2E=1 and skipped by default; CI does not invoke it.
Contributor
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- formatDateOnly (models/google-flights/flights.ts): switch UTC getters to local-time getters. parseDateTime in decoders.ts builds Date via the local-time `new Date(y, m-1, d, h, min)` constructor; reading back with getUTC* would shift the date by ±1 day for any caller not in UTC, causing the wrong departure date to be embedded in the selected_flight payload for return-leg lookups. - getClient (search/client.ts): when options are passed after the singleton already exists, replace the cached instance instead of silently discarding them. Document the behaviour and steer callers who need an isolated client toward `new Client(...)` + explicit injection. - _readVarint (search/proto.ts) / readVarint (core/currency.ts): replace `value |= chunk << shift` with `value += chunk * 2**shift`. JavaScript bitwise ops coerce to signed 32-bit, so any varint encoding past bit 30 (value ≥ 2^31) would silently flip into a negative number. Throw when the decoded value exceeds Number.MAX_SAFE_INTEGER. Adds regression tests: - proto.test.ts: _readVarint decodes 2^31, 2^32, 2^40 correctly. - filter_format_snapshots.test.ts: a Dec 15 20:00 local-time flight encodes "2026-12-15" in the selected_flight payload (would be "2026-12-16" with the old UTC-getter behaviour in TZ < UTC).
punitarani
changed the title
Code-review round 2 — addresses the Greptile findings plus issues
surfaced by parallel multi-agent review of the new port.
Real bugs:
- scripts/generate-enums.ts: naive `indexOf(",")` parser silently
corrupted 10 enum entries whose CSV names contain a comma or
escaped quote (e.g. `Y2,"Air Century, S.A."` and PAQ's
`"Warren ""Bud"" Woods…"`). Python uses csv.DictReader which is
RFC4180-compliant. Replace with a minimal stateful parser that
understands quoted fields and `""` quote-escaping, then regenerate
airline.ts and airport.ts so the wire-format sort keys match
Python again. Add enum-test pins for the previously-broken codes.
- search/dates.ts parseIsoDate: `Number.parseInt("abc",10)` returns
NaN and `NaN == null` is false, so the existing nullish-only guard
let malformed dates through and produced an Invalid Date whose
`.getTime()` silently propagated as NaN. Tighten to Number.isFinite
and add a regression pin via SearchDates._parseDate.
- search/client.ts: per-attempt `externalSignal.addEventListener` was
never removed, so a 3-retry request leaked 3 abort closures on the
shared signal. Store the listener and removeEventListener in the
attempt's finally block alongside clearTimeout.
- search/client.ts: `await response.text().catch(() => "")` was read
and immediately dropped via the no-op template `${text ? "" : ""}`.
Remove the dead read.
Quality cleanups:
- search/decoders.ts safeAirline: drop the dead digit-prefix branch.
AIRLINE_BY_CODE keys are bare codes (the `_` is stripped at lookup-
table build), so the `_${code}` lookup was always false and only
the fallthrough match ran.
- search/proto.ts extractBookingTokenFromTfu: distinguish URL-parse
failure from `tfu` param absence in the error message so callers
can tell a malformed input from a well-formed URL that just lacks
the param.
Local CI green: format, lint, typecheck, 258/258 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two parity gaps surfaced by additional review:
- search/decoders.ts parseDateTime: a partial-null date array like
`[2026, null, null]` slipped past the existing all-null guard, was
defaulted to `[2026, 0, 0]`, and `new Date(2026, -1, 0, ...)`
silently returned Nov 30 2025. Python's `datetime(2026, 0, 0)`
raises ValueError and the caller treats it as "skip row." Restore
that strictness with an explicit null/range check on y/m/d. Time
components keep the `?? 0` fallback to match Python's `or 0`.
- search/wire.ts iterWrbChunks: `Number.parseInt("12abc", 10)`
returns 12, so a malformed length header would silently shift the
cursor into garbage instead of aborting the loop. Python's
`int(raw[cursor:end])` raises and breaks. Add a `/^[0-9]+$/` test
before parsing so we match Python's all-or-nothing semantics.
Pin both with regression tests in decoders.test.ts. 259/259 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
) * ci: add npm publish + release flow for fli-js (mirrors PyPI) Adds a parallel manual-dispatch release pipeline for the fli-js (npm) package that matches the shape of the existing PyPI flow: release-npm.yml bumps fli-js/package.json + refreshes bun.lock, commits/tags fli-js-vX.Y.Z, creates a GitHub Release, then calls publish-npm.yml which runs the JS test suite, builds via tsc -p tsconfig.build.json, packs, and uploads to npm with --provenance under the 'npm' GitHub environment using NPM_TOKEN. - scripts/bump_version.py: generalise to handle package.json via a new --package-json flag (text-mode rewrite so formatting is preserved); add --tag-prefix so the JS release uses fli-js-v while PyPI keeps v. - tests/scripts/test_bump_version.py: cover the new JSON path, the --tag-prefix flag, and the pyproject/package-json mutual exclusion. - fli-js: switch package.json to dist-rooted exports, add a tsconfig.build with rewriteRelativeImportExtensions so .ts source imports compile to .js (and .d.ts) cleanly, add LICENSE, files whitelist, sideEffects: false, provenance publishConfig, and a prepublishOnly hook. - publish.yml: skip on fli-js-v* release events so a JS tag does not try to republish to PyPI (and vice-versa for publish-npm.yml). - dependabot: cover fli-js npm deps weekly with grouped updates. - docs: add docs/guides/release-npm.md and link from CLAUDE.md, release.md, mkdocs nav. * address greptile review: artifact versions, abort discrimination, parseIsoDate dedupe, JSON depth-aware version replace - publish-npm.yml: align upload-artifact + download-artifact to @v8 so the tarball produced by release-build is guaranteed to be readable by npm-publish. - search/client.ts: distinguish an external AbortSignal from the internal timeout. When the caller's signal triggered the abort we now propagate the original error (or the caller-supplied reason) as-is, without retrying and without relabelling it as SearchTimeoutError — so a consumer using `instanceof SearchTimeoutError` to decide whether to retry no longer retries on a deliberate cancellation. Adds two regression tests covering pre-aborted and mid-flight cancellation. - core/dates.ts (new): canonical parseIsoDate + formatIsoDate + strict ISO_DATE_RE in a shared module. The three near-identical copies in search/dates.ts, models/google-flights/base.ts, and models/google-flights/dates.ts (one of which threw `Error` instead of `TypeError` and skipped range validation) are replaced with imports. Updates one assertion in search_dates_stubbed.test.ts whose regex was tied to the looser old message. - bump_version.py: replace the regex-based package.json version rewriter with a depth-tracking JSON walker so a nested key literally named "version" (e.g. inside `dependencies` or `overrides`) is never mistaken for the top-level field. Adds three edge-case tests: before-top-level nested key, inline nested object, and a string value containing the substring "version". --------- Co-authored-by: Claude <noreply@anthropic.com>
…js-nXzUe # Conflicts: # .github/workflows/publish.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a complete TypeScript/JavaScript port of the fli library (
fli-js), enabling programmatic access to Google Flights data in Node.js and browser environments. The port is a 1:1 translation of the Python implementation, maintaining API compatibility and byte-identical request/response handling.Key Changes
Core Models (
src/models/)AirlineandAirportenums from CSV data (2,226+ airlines, 15,784+ airports)FlightSearchFilters,FlightResult,FlightLeg,BookingOption, etc.TripType,SeatType,MaxStops,EmissionsFilter,Currency,AllianceSearch Engine (
src/search/)SearchFlights: Flight search viaGetShoppingResults+GetBookingResultsRPC endpointsSearchDates: Date-range search viaGetCalendarGraphendpointCore Utilities (
src/core/)Comprehensive Test Suite (
tests/)FLI_E2E=1)Build & Configuration
scripts/generate_enums.pyNotable Implementation Details
safeGet,asInt,asBool, etc.) handle malformed responses gracefullydata/airlines.csvanddata/airports.csvfilesIntegration
.github/workflows/test.ymlto runfli-jsworkflow alongside existing Python testsdata/airlines.csv,data/airports.csv) used by both Python and TypeScript implementationssrc/index.tsfor clean consumer-facing interfacehttps://claude.ai/code/session_01R2vZihmSWV2wvRawjF4Gjo
Greptile Summary
This PR adds a complete TypeScript/JavaScript port of the
flilibrary (fli-js), providing programmatic access to Google Flights data for Node.js and browser environments. It is a 1:1 translation of the Python implementation covering HTTP client, wire-format parsing, protobuf encoding, response decoders, and an extensive test suite with snapshot, integration, and unit tests.src/search/):SearchFlightsandSearchDatesclasses with rate-limited HTTP client, multi-chunk JSONP wire parser, booking-token protobuf encoder/decoder, and round-trip/multi-city expansion via boundedparallelMap.src/models/): Auto-generatedAirline(2,226+ entries) andAirport(15,784+ entries) enums from shared CSV data, plusFlightSearchFilters,DateSearchFilters, and all supporting types.fli-js.yml): Bun-based project with TypeScript strict mode, Biome + oxlint, and a CI workflow that verifies generated enums are in sync with the data files before running tests.Confidence Score: 5/5
This is a well-structured new library with no regressions to existing Python code. All three issues flagged in previous review rounds are correctly resolved in this revision.
All three previously-flagged bugs are fixed: formatDateOnly now uses local-time getters, getClient replaces the singleton when options are passed, and varint decoding uses 2**shift instead of bitwise operators. The remaining findings are style/quality observations that do not affect correctness on any normal code path.
fli-js/src/search/client.ts — abort-signal listener accumulates per retry without removal; fli-js/src/search/decoders.ts — dead first branch in safeAirline.
Important Files Changed
Sequence Diagram
sequenceDiagram participant Caller participant SearchFlights participant Client participant TokenBucket participant GoogleAPI participant WireParser participant Decoders Caller->>SearchFlights: search(filters, options) SearchFlights->>Client: post(url, f.req body) Client->>TokenBucket: acquire() TokenBucket-->>Client: token granted Client->>GoogleAPI: POST GetShoppingResults GoogleAPI-->>Client: multi-chunk JSONP Client-->>SearchFlights: ClientResponse SearchFlights->>WireParser: parseFirstWrbPayload(text) WireParser-->>SearchFlights: inner JSON array SearchFlights->>Decoders: parseFlightRow(row) xN Decoders-->>SearchFlights: FlightResult[] SearchFlights-->>Caller: FlightResult[] or FlightResult[][]Prompt To Fix All With AI
Reviews (2): Last reviewed commit: "fli-js: fix UTC/local date, getClient op..." | Re-trigger Greptile