Repository navigation
E2E matrix across Xcode 26/27 × Simulator.app/Device Hub, plus the #76 login-flow fixture fix - #77
Merged
Merged
Conversation
make e2e-matrix drives the iOS E2E suites across the four supported host environments: selected Xcode 26.x/27.x crossed with Device Hub closed at boot (the classic Simulator.app workflow, legacy indigo HID) or open at boot (CoreDevice dtuhidd). One leg runs the full 22-suite pass (default x27-hub, the primary workflow once Xcode 27 ships; --full <leg>|all|none), the rest run a new smoke tier (test-runner.sh --smoke: describe-ui, tap, type, scroll); legs whose Xcode is not installed are skipped. The package builds once on the xcode-select toolchain (build_products/ is toolchain-locked); each leg swaps only the runtime Xcode via SIM_USE_TEST_DEVELOPER_DIR, which TestUtilities injects as DEVELOPER_DIR into every process the suites spawn, and boots a device whose iOS runtime matches the leg's generation (runtimes are system-wide, so "newest available" would cross-contaminate). Every leg is double-gated against silently green-running the wrong combination: pgrep -x dtuhidd checks before and after the suites plus a SIM_USE_DEBUG probe that must report the matching transport-selection predicate; per-leg logs and an evidence file land in .build/e2e-matrix/<timestamp>/. test-runner.sh also gains arbitrary repeatable swift test --filter patterns (previously one suite from a stale whitelist) and resolves its default simulator by exact device name on the newest available runtime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwbDm4Nave4bm6AisX6BJV Signed-off-by: onevcat <onevcat@gmail.com>
…strip The system suggestion strip attaches ~41pt below a focused text field and swallows taps on anything it covers. The Batch Login Flow screen placed Continue/Sign In right inside that band, so the batch E2E's step-2 tap hit the strip, the screen never left the email stage, and the password was typed (and autocorrected) into the email field - failing step 4 in both login-flow cases. Pad the buttons clear of the band and disable autocorrection on the email field. Reproduced identically on legacy and DTU HID transports, so this was fixture rot, not a product regression. Fixes #76 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwbDm4Nave4bm6AisX6BJV Signed-off-by: onevcat <onevcat@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two commits:
make e2e-matrix(scripts/e2e-matrix.sh) — runs the iOS E2E suites across the four supported host environments:x26-simx27-simx26-hubx27-hubOne leg runs the full 22-suite pass (default
x27-hub, the primary workflow once Xcode 27 ships;--full <leg>|all|none), the rest run a new smoke tier (test-runner.sh --smoke: describe-ui, tap, type, scroll). Legs whose Xcode is not installed are skipped.Fixes BatchTests login-flow cases fail: QuickType bar overlaps the fixture's Continue button, tap never advances the screen #76 — the Batch Login Flow playground screen placed
Continue/Sign Ininside the band the system text-suggestion strip occupies below a focused field; the strip swallowed the batch's tap and both login-flow E2E cases failed. Pad the buttons clear of the band + disable autocorrection on the email field. Fixture-only.How the matrix stays honest
swift build/swift teststay on the xcode-select toolchain (build_products/is toolchain-locked); each leg passesSIM_USE_TEST_DEVELOPER_DIR, whichTestUtilities.CommandRunnerinjects asDEVELOPER_DIRinto every spawned process — the same shape as a released binary meeting whatever Xcode a user has selected. Injection verified by pointing it at a bogus path and watching the spawned sim-use fail on exactly that path.pgrep -x dtuhiddmust match the leg (present for hub legs, absent for sim legs) and aSIM_USE_DEBUGprobe (ios key 231, daemonless) must log the matching transport-selection predicate; after the suites, the dtuhidd state is re-checked for drift. Per-leg logs plus anevidence.txt(xcodebuild version, UDID, runtime, hub state, probe line) land in.build/e2e-matrix/<timestamp>/.Test plan (all executed on this branch's tree)
make test: 1205 passed (one unrelatedHIDSendDeadlineTeststiming flake under full parallel load; green in isolation).make e2e-matrixfull run:x26-sim✅x27-sim✅x26-hub✅ (smoke) ·x27-hubfull 21/22 — the only red was the pre-existing BatchTests login-flow cases fail: QuickType bar overlaps the fixture's Continue button, tap never advances the screen #76 fixture rot, reproduced identically with Device Hub closed, then root-caused and fixed here. After the fix,BatchTestspasses 14/14 on the same simulator. Evidence files confirm every leg ran its intended Xcode build, iOS runtime generation, hub state, and transport predicate.make e2e-android: 7/7 suites green.x26-hubgenuinely exercises the auto→DTU path on Xcode 26.🤖 Generated with Claude Code
https://claude.ai/code/session_01BwbDm4Nave4bm6AisX6BJV