Repository navigation
fix: make directional gesture presets orientation-aware - #73
Merged
Merged
Conversation
gesture scroll-*/swipe-from-*-edge name visual directions, but their stroke math ran on a hardcoded 390x844 portrait canvas and the endpoints reached HID untransformed - on a rotated device the gesture pointed 90/180 degrees away from its name (issue #66: scroll-up on a landscape iPad scrolled nothing while a native-horizontal swipe scrolled the list vertically). Directional presets now compute in the calibrated visual space and their endpoints cross uiToFramebuffer - the same issue #34 mapping tap selectors ride. The canvas defaults to the auto-detected UI size of the current orientation (--screen-width/height become a visual-space viewport override; 390x844 survives only when the screen cannot be probed). The batch gesture path rides the batch-wide calibration and degrades to an identity dispatch with a recorded advisory when the AX tree is unreachable. Calibration uses a new recovery-free fetch entry (one tree XPC + at most 3 probes) so gestures never pay for the collapsed-children quadtree walk. Explicit swipe/touch coordinates stay device-native portrait per the issue #34 contract; pinch/rotate presets stay raw (orientation-aware multi-touch needs per-sample mapping and is left out of scope). Verified live on an iPad Pro 13-inch (M5): landscape scroll-up now scrolls (ROW-001 -> ROW-005) where the pre-fix binary moved nothing, with the HID endpoints matching the unit-test expectations exactly; portrait behavior unchanged. 1179 tests green. Part of #66 (PR A; PR B adds --coordinate-space ui to swipe/touch) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rir4WwrPShf5DYzziJ8RyC Signed-off-by: onevcat <onevcat@gmail.com>
Review follow-up on the orientation fix: the --screen-width/height help claimed visual-space auto-detection unconditionally, but the pinch/rotate branch still runs on the raw device-native 390x844 canvas. Say so explicitly in both the ios subcommand and the top-level forwarder help, and scope the CHANGELOG entry to single-finger presets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rir4WwrPShf5DYzziJ8RyC Signed-off-by: onevcat <onevcat@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #66 — PR A of two. PR B (follow-up) adds
--coordinate-space uitoswipe/touch.Problem
gesture scroll-up/down/left/rightandswipe-from-*-edgename visual directions, but their stroke math ran on a hardcoded 390×844 portrait canvas (iPhone 15) and the endpoints reached HID untransformed. On a rotated device the emitted axis is 90° (landscape) or 180° (upside-down) away from the preset's name — reproduced live on a landscape iPad:scroll-upmoved nothing while a native-horizontalswipescrolled the page vertically.Fix
GestureOrientationMapping:visualSizeresolves the canvas (explicit--screen-width/heightper axis → calibrateduiSize(native:)→ legacy 390×844 last),hidStrokecarries UI-space endpoints acrossuiToFramebuffer— the same iOS tap coordinates are wrong when app orientation and Simulator window rotation differ #34 mappingtapselectors ride.AccessibilityFetcher.fetchOrientationCalibrationdoes one tree XPC + ≤3 hit-test probes and skips the collapsed-children recovery walk entirely, so gestures never pay for the quadtree probing (hundreds of XPC round-trips on WebView-heavy screens).Command+BatchConvertible) ridesBatchContext.orientationCalibration— one calibration per run, shared with tap selector steps. An unreachable AX tree degrades to identity dispatch with a step-prefixedorientation_calibration_fallbackadvisory instead of failing the batch.ExecutionResultnow conforms toCommandAdvisoryProviding, registered in the exclude-from-data contract tests).swipe/touchcoordinates stay device-native portrait per the iOS tap coordinates are wrong when app orientation and Simulator window rotation differ #34 acceptance contract (opt-in visual space arrives in PR B); pinch/rotate presets stay raw — orientation-aware multi-touch needs every interpolated sample mapped, not just endpoints.Testing
GestureOrientationMappingTestswritten against a stub reproducing the legacy behavior first — 23 failures across 4 orientations × 4 scrolls + edges (including portrait-on-iPad, pinning the canvas-default fix), then green after the real implementation. Endpoint expectations hand-derived from theuiToFramebufferformulas.BatchGestureOrientationTests: one calibration shared across directional steps, multi-touch never fetches, fetch failure records the advisory and still emits primitives.(645,688)→(387,688)match the unit-test expectations exactly.🤖 Generated with Claude Code
https://claude.ai/code/session_01Rir4WwrPShf5DYzziJ8RyC