Skip to content

Support nested route fragment separate testing - #7

Merged
parsonsmatt merged 25 commits into
mainfrom
mattp/test-route-fragments
Jun 18, 2026
Merged

Support nested route fragment separate testing#7
parsonsmatt merged 25 commits into
mainfrom
mattp/test-route-fragments

Conversation

@parsonsmatt

@parsonsmatt parsonsmatt commented Jan 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Motivation

This PR provides separate testing of route fragments: you can test a subset of routes without depending on the full YesodDispatch for your web application. Our YesodDispatch requires compiling over 10k modules, but most Handler modules depend on only 2-3k modules in our codebase, so this makes testing dramatically nicer.

It builds on the nested-route-separation work that shipped in yesod-core-1.7.0.0 (originally yesod #1887). The branch was previously pinned to a yesodweb/yesod git commit; it now uses the released package from Hackage.

What you get

  • setUrlNested and the generalized RequestBuilderFor url site, which carries the route-fragment type, so a handler spec can depend only on the cheap YesodDispatchNested FooR instead of the whole app's dispatch.
  • get/post/postBody/performMethod/request now dispatch via yesod-core's UrlToDispatch url site.
  • getRunnerEnv, which builds a YesodRunnerEnv for the current test's site.

Breaking changes

  • Requires yesod-core >= 1.7.0.0.
  • RequestBuilderData gained a url type parameter (RequestBuilderData url site); getLatestRequest/requireLatestRequest return RequestBuilderData () site.
  • get/post/postBody/performMethod/request require UrlToDispatch url site in place of RedirectUrl site url.
  • A request must now set a URL explicitly; an unset URL used to default to /, now request fails with a clear error.
  • Removed TestApp, mkTestApp, and the yedCreateApplication field.

Runner-env handling

The WAI application for each request is built from a YesodRunnerEnv constructed for the current yedSite via mkYesodRunnerEnv (exposed as getRunnerEnv). It is built fresh per request so it always reflects the current foundation — including per-request changes made by harnesses that swap the site to mock time or adjust app state.

An earlier revision of this branch cached the env per test for performance. That turned out to be incorrect: the cache was only invalidated by testModifyFoundationAndMiddleware, but downstream harnesses (e.g. mercury-web-backend's localApp/mockStartTime) replace the dispatched foundation by updating yedSite directly, so dispatch ran against a stale site and ignored time stubs. Building fresh per request is simple and correct.

mkYesodRunnerEnv spawns auto-update worker threads; idle ones are reaped by the RTS. The one that genuinely lingers — the 24-hour max-expires worker, which sleeps a full day before it can notice it's idle — is fixed upstream in yesodweb/yesod#1923 (a patch release that recomputes the value instead of caching it behind a thread). After that lands, per-request construction is fully thread-free.

Review notes

This branch went through a full review pass; fixes included lenient decoding in debug formatters, withFrozenCallStack on assertion failures, removal of partial head/error calls, and assorted test/doc cleanups. Deferred follow-up: #9 (harden setUrl URL parsing with network-uri).

See ChangeLog.md for the full 0.3.0.0 entry.

🤖 Generated with Claude Code

@parsonsmatt parsonsmatt mentioned this pull request Jan 22, 2026
Merged
5 tasks
parsonsmatt and others added 6 commits January 23, 2026 16:34
@parsonsmatt
OK, maybe the RequestBuilder change is a little invasive
a2c9b0c
@parsonsmatt
Apply middlewares
a50d222
@parsonsmatt
incorporate 2.1.1.1 changes
a392e8b
@parsonsmatt
Require dispatch support for raw Text helpers
3ebec5d
@parsonsmatt
support newer branch
5001986
@parsonsmatt @claude
Use released yesod-core-1.7.0.0 instead of git fork
df9af22 
The route-fragment dispatch helpers on this branch required dispatch
support that was previously only available on a yesodweb/yesod git
branch. That support is now published as yesod-core-1.7.0.0 on Hackage,
so the git pin is no longer needed.

- stack.yaml: replace the git subdir pin with yesod-core-1.7.0.0
- hspec-yesod.cabal: bump lower bound to >= 1.7.0.0, since the new
  route helpers require the 1.7 dispatch API

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt
parsonsmatt marked this pull request as draft June 16, 2026 21:59
parsonsmatt and others added 4 commits June 16, 2026 16:11
@parsonsmatt @claude
Fill in 0.3.0.0 changelog notes
54ec98f 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt @claude
Clean up TODOs, drop design notes, document API changes
c6e795d 
- Replace the @SInCE TODO/blank tags on RequestBuilderFor and
  setUrlNested with 0.3.0.0
- Replace the scratch design-musings comment above mkApplicationFor
  with a real Haddock description
- Delete notes.md scratch file
- Expand the 0.3.0.0 changelog with the practical/breaking API changes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt
fix changelog
aaaa9b0
@parsonsmatt @claude
Apply code-review fixes: env caching, thread cleanup, robustness
a942d1b 
Addresses findings from the max-effort review of the route-fragment work.

Resource/performance:
- Cache the YesodRunnerEnv per test (new yedRunnerEnv field) instead of
  rebuilding it on every request. mkYesodRunnerEnv spawns auto-update worker
  threads, so per-request construction leaked threads proportional to the
  request count. The cache is invalidated whenever the site changes.
- Swap yreGetMaxExpires for a thread-free recomputation. Its default is a
  24h auto-update worker that, once read by dispatch, lingers for the life
  of the test process; the replacement lets the RTS reap the unused worker.
- Add getRunnerEnv to expose the cached environment.

Correctness / robustness:
- Decode request bytes leniently in the debug/preview formatters so a
  UnicodeException can't mask the real assertion message.
- failure now carries a real message and uses withFrozenCallStack, so the
  assertion is blamed on the caller; fix the "libary" typo.
- Replace a partial `head` in addToken_ and a partial `error` in setUrl with
  descriptive failures.
- voidRequestBuilderUrl maps the existing Maybe rather than forcing Just ().
- Drop the unused RankNTypes pragma; scope away main.hs's module-wide
  -Wno-incomplete-uni-patterns.

Tests / docs:
- Fix the getFooShowR fixture label (was a getFooEditR copy-paste).
- Split the double-setUrl nested-dispatch test so setUrl and setUrlNested are
  each exercised independently.
- Document why NestedRouteDispatchSpec.YesodDispatch is kept (verifies the
  full app dispatch still compiles alongside the fragment dispatch).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt parsonsmatt mentioned this pull request Jun 17, 2026
Open
parsonsmatt and others added 4 commits June 17, 2026 15:58
@parsonsmatt @claude
Stop overriding yreGetMaxExpires in the test runner env
5bf2f57 
The testGetMaxExpires replacement installed a max-expires getter whose
RFC1123 timestamp used "%Z" (renders "UTC") instead of a literal "GMT",
diverging from yesod-core's getCurrentMaxExpiresRFC1123. This caused a
regression in downstream test suites (66 failures in mercury-web-backend).

Back it out entirely and use the environment as built by mkYesodRunnerEnv;
the lingering 24-hour max-expires auto-update worker will be addressed
upstream in yesod-core instead. The per-test YesodRunnerEnv caching
(yedRunnerEnv / getRunnerEnv) is retained.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt @claude
Key the cached runner env on the site's StableName
e1c5ec1 
The per-test YesodRunnerEnv cache was only invalidated by
testModifyFoundationAndMiddleware. Downstream harnesses (e.g.
mercury-web-backend's localApp / mockStartTime) replace the dispatched
foundation via a direct state update, bypassing that invalidation, so
getRunnerEnv served an environment built from the original site and
dispatch ignored the swapped-in foundation (66 test failures involving
mocked time and per-request app state).

Key the cache on makeStableName of yedSite instead: any time the site is
a different heap object than the one the cached env was built from, the
env is rebuilt. yedSite is strict (always WHNF), so its StableName is
stable across requests while the site is unchanged. StableName equality
holds only for the same object, so a stale env is never served; the worst
case is a redundant rebuild.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt @claude
Drop the per-test runner-env cache; build it fresh per request
984b2bb 
The cached YesodRunnerEnv was only an optimization, and it broke
downstream test suites: the cache was invalidated only by
testModifyFoundationAndMiddleware, but harnesses like
mercury-web-backend swap the dispatched foundation by updating yedSite
directly (localApp / mockStartTime), so dispatch ran against a stale
site and ignored time stubs / per-request app state (66 failures).

Keying the cache on the site's StableName would fix correctness but
leak StableName into the exported YesodExampleData(..) API. Since the
underlying issue is that mkYesodRunnerEnv's auto-update workers aren't
cleanly closeable (a yesod-core concern), just build the env fresh per
request via getRunnerEnv (the proven pre-caching behavior) and remove
the yedRunnerEnv field entirely. Idle workers are reaped by the RTS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt @claude
Tighten the 0.3.0.0 changelog wording
866542a 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt
parsonsmatt marked this pull request as ready for review June 18, 2026 21:20
@parsonsmatt @claude
TEMP: constrain megaparsec < 9.8 to unbreak CI
8b7b872 
megaparsec-9.8.0 (pulled in transitively via yesod-form) fails to compile
in the CI build plan ("Variable not in scope" in Text.Megaparsec.Stream),
failing the Install-dependencies step across the whole GHC matrix. Pin
below it until a fixed megaparsec ships, then remove this cabal.project
constraint. Unrelated to this PR's code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@parsonsmatt
parsonsmatt merged commit 7e18273 into main Jun 18, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@parsonsmatt