vision-camera-resizer: cover/contain fit computed before rotation causes anamorphic squeeze on portrait frames

[hmelonjp]

Prerequisites

• I have read and followed every applicable step in the Nitro Modules Troubleshooting guide.
• I have read the VisionCamera Troubleshooting guide.
• I have searched existing issues and found nothing matching.
• I am on the latest version of react-native-vision-camera, or have a specific reason I cannot upgrade.

Reproduction

https://github.com/Melon-Technologies/vc-resizer-anamorphic-squeeze-repro

(Fallback repro repo — react-native-vision-camera-resizer has no existing test harness, and this is a compute-kernel math bug, not an imperative API bug. The repro is device-free: it runs the actual unmodified upstream Metal shader (iOS) directly on a Mac, and the actual unmodified upstream Vulkan compute shader (Android) via MoltenVK — both real GPU shaders, not a hand-ported replica of the math.

Steps to reproduce

1. Clone https://github.com/Melon-Technologies/vc-resizer-anamorphic-squeeze-repro
2. brew install glslang molten-vk vulkan-loader vulkan-headers
3. swift run ResizerAnamorphicSqueezeRepro ./output (requires only Xcode/Metal on a Mac)
4. Compare output/ios_buggy_output.png (unmodified upstream) against output/ios_fixed_output.png (patched).
5. Compare output/android_buggy_output.png (unmodified upstream) against output/android_fixed_output.png (patched).
6. The console output prints the measured aspect ratio of a synthetic test circle rendered through each variant.

In a live app: Use a front camera sensor (16:9 native), configure the output as portrait (rotated 90/270 degrees), and use ScaleMode.COVER.

What did you expect to happen?

A circle rendered through the resizer (via ScaleMode.COVER, rotated 90°) should stay a perfect circle. The fit should preserve aspect ratio regardless of frame rotation.

What actually happened?

The circle is squeezed into an ellipse with a measured width/height aspect ratio of 3.1622 on both iOS and Android — Both are ≈ (1920/1080)² = 3.16 (the sensor aspect ratio squared).

In practice, any app using the resizer on a portrait-oriented camera frame (the common case for a front-facing selfie camera) gets a visibly distorted output image. We hit this while validating a face-detection pipeline against 4K front-camera frames on iOS < 17; the down-stream ML inference degraded measurably due to the squeezed geometry.

Root cause

Both ResizerKernels.metal (iOS) and Resizer.comp (Android, Vulkan compute shader) compute their cover/contain fit ratio using the frame's pre-rotation source dimensions, then apply the frame's rotation to the sampling coordinate afterward. Because the fit ratio is computed against the wrong (pre-rotation) aspect ratio, the two axes end up scaled by different factors than intended.

// ResizerKernels.metal (iOS) -- Resizer.comp (Android) has the identical shape
float2 outputSize = float2(uniforms.outputWidth, uniforms.outputHeight);
float2 sourceSize = float2(yTexture.get_width(), yTexture.get_height()); // pre-rotation!
...
case 0u: // ScaleMode::COVER
  scale = max(outputSize.x / sourceSize.x, outputSize.y / sourceSize.y); // fit against the wrong axes
  break;
...
// rotation is applied to the sampling coordinate only *after* the fit above
switch (inverseRotation) {
  case 90: coordinate = float2(1.0f - coordinate.y, coordinate.x); break;
  ...

Fix

Compute the fit ratio against the source's upright (rotation-aware) dimensions — swap width/height for the fit calculation whenever rotationDegrees is 90 or 270 — instead of the raw sensor-native dimensions. We've been running the following patch in production (via patch-package, pinned to 5.1.1), applied to both platforms:

diff --git a/node_modules/react-native-vision-camera-resizer/android/src/main/shaders/Resizer.comp b/node_modules/react-native-vision-camera-resizer/android/src/main/shaders/Resizer.comp
index 733f94c..2173d87 100644
--- a/node_modules/react-native-vision-camera-resizer/android/src/main/shaders/Resizer.comp
+++ b/node_modules/react-native-vision-camera-resizer/android/src/main/shaders/Resizer.comp
@@ -53,20 +53,34 @@ vec2 outputToInputCoordinate(uvec2 gid) {
   vec2 sourceSize = vec2(textureSize(inputImage, 0));
   vec2 outputCoordinate = (vec2(gid) + vec2(0.5)) / outputSize;
 
+  int normalizedRotation = pushConstants.rotationDegrees % 360;
+  if (normalizedRotation < 0) {
+    normalizedRotation += 360;
+  }
+  int inverseRotation = (360 - normalizedRotation) % 360;
+
+  // The fitted coordinate below is rotated into the source's raw axes before
+  // sampling, so the cover/contain fit must be computed against the source's
+  // *upright* (rotation-aware) dimensions. Fitting against the raw dimensions
+  // while rotating 90/270 degrees axis-swaps the fit and anamorphically
+  // distorts the output by the source aspect ratio squared.
+  bool isSideways = normalizedRotation == 90 || normalizedRotation == 270;
+  vec2 uprightSourceSize = isSideways ? sourceSize.yx : sourceSize;
+
   float scale = 0.0;
   switch (kScaleMode) {
     case 0u: // 0u == ScaleMode::COVER
-      scale = max(outputSize.x / sourceSize.x, outputSize.y / sourceSize.y);
+      scale = max(outputSize.x / uprightSourceSize.x, outputSize.y / uprightSourceSize.y);
       break;
     case 1u: // 1u == ScaleMode::CONTAIN
-      scale = min(outputSize.x / sourceSize.x, outputSize.y / sourceSize.y);
+      scale = min(outputSize.x / uprightSourceSize.x, outputSize.y / uprightSourceSize.y);
       break;
     default:
       // Unsupported ScaleMode specialization. Force out-of-bounds so the caller gets black.
       return vec2(-1.0);
   }
 
-  vec2 renderedSourceSizeInOutput = sourceSize * scale;
+  vec2 renderedSourceSizeInOutput = uprightSourceSize * scale;
   vec2 renderedSourceOffsetInOutput = (outputSize - renderedSourceSizeInOutput) * 0.5;
   vec2 coordinate = ((outputCoordinate * outputSize) - renderedSourceOffsetInOutput) / renderedSourceSizeInOutput;
 
@@ -82,12 +96,6 @@ vec2 outputToInputCoordinate(uvec2 gid) {
     coordinate.x = 1.0 - coordinate.x;
   }
 
-  int normalizedRotation = pushConstants.rotationDegrees % 360;
-  if (normalizedRotation < 0) {
-    normalizedRotation += 360;
-  }
-  int inverseRotation = (360 - normalizedRotation) % 360;
-
   switch (inverseRotation) {
     case 90:
       coordinate = vec2(1.0 - coordinate.y, coordinate.x);
diff --git a/node_modules/react-native-vision-camera-resizer/ios/Metal/ResizerKernels.metal b/node_modules/react-native-vision-camera-resizer/ios/Metal/ResizerKernels.metal
index 79106ed..8caebfe 100644
--- a/node_modules/react-native-vision-camera-resizer/ios/Metal/ResizerKernels.metal
+++ b/node_modules/react-native-vision-camera-resizer/ios/Metal/ResizerKernels.metal
@@ -40,20 +40,34 @@ inline float3 sampleRgb(
   float2 sourceSize = float2(yTexture.get_width(), yTexture.get_height());
   float2 outputCoordinate = (float2(gid) + 0.5f) / outputSize;
 
+  int normalizedRotation = uniforms.rotationDegrees % 360;
+  if (normalizedRotation < 0) {
+    normalizedRotation += 360;
+  }
+  int inverseRotation = (360 - normalizedRotation) % 360;
+
+  // The fitted coordinate below is rotated into the source's raw axes before
+  // sampling, so the cover/contain fit must be computed against the source's
+  // *upright* (rotation-aware) dimensions. Fitting against the raw dimensions
+  // while rotating 90/270 degrees axis-swaps the fit and anamorphically
+  // distorts the output by the source aspect ratio squared.
+  bool isSideways = normalizedRotation == 90 || normalizedRotation == 270;
+  float2 uprightSourceSize = isSideways ? sourceSize.yx : sourceSize;
+
   float scale = 0.0f;
   switch (kScaleMode) {
     case 0u: // 0u == ScaleMode::COVER
-      scale = max(outputSize.x / sourceSize.x, outputSize.y / sourceSize.y);
+      scale = max(outputSize.x / uprightSourceSize.x, outputSize.y / uprightSourceSize.y);
       break;
     case 1u: // 1u == ScaleMode::CONTAIN
-      scale = min(outputSize.x / sourceSize.x, outputSize.y / sourceSize.y);
+      scale = min(outputSize.x / uprightSourceSize.x, outputSize.y / uprightSourceSize.y);
       break;
     default:
       // Unsupported ScaleMode ordinal. Return black so broken modes fail visibly at runtime.
       return float3(0.0f);
   }
 
-  float2 renderedSourceSizeInOutput = sourceSize * scale;
+  float2 renderedSourceSizeInOutput = uprightSourceSize * scale;
   float2 renderedSourceOffsetInOutput = (outputSize - renderedSourceSizeInOutput) * 0.5f;
   float2 coordinate = ((outputCoordinate * outputSize) - renderedSourceOffsetInOutput) / renderedSourceSizeInOutput;
 
@@ -70,12 +84,6 @@ inline float3 sampleRgb(
     coordinate.x = 1.0f - coordinate.x;
   }
 
-  int normalizedRotation = uniforms.rotationDegrees % 360;
-  if (normalizedRotation < 0) {
-    normalizedRotation += 360;
-  }
-  int inverseRotation = (360 - normalizedRotation) % 360;
-
   switch (inverseRotation) {
     case 90:  // counter-rotate +90°
       coordinate = float2(1.0f - coordinate.y, coordinate.x);

Happy to open this as a PR against the upstream repo instead, if that's preferred over a patch attached to an issue.

Affected platforms

• iOS (device)
• Android (device)

Device(s) affected

Originally observed on production iOS devices with a 16:9 front camera sensor while validating a face-detection pipeline on 4K front-camera frames (iOS < 17, where VisionCamera can't downscale before delivery).

VisionCamera version

5.1.1

React Native version

0.86.0

React Native architecture

New Architecture (Fabric / bridgeless)

Features being used

• Preview
• Frame Processors (worklets)
• Photo capture
• Video capture
• Skia Frame Processors
• Code/Barcode Scanner
• Location metadata
• Multi-cam
• Depth data
• HDR / custom dynamic range
• Custom format / FPS / resolution

(Note: react-native-vision-camera-resizer itself isn't a listed checkbox item — it's used from within a Frame Processor to GPU-resize+convert frames before running inference.)

Relevant logs / stack trace

This is a geometric/visual correctness bug, not a crash — there is no stack trace. See the reproduction repo's console output and the before/after PNGs in its docs/ folder for the measured distortion:

Input: 1920x1080 landscape sensor frame, rotationDegrees=90
Output: 1080x1920 portrait target, ScaleMode.COVER

--- ios_buggy_output ---
  shader: ResizerKernels.metal (fetched verbatim from mrousavy/react-native-vision-camera)
  circle bounding box: 819x259 px
  measured aspect ratio (width/height): 3.1622  (1.0 == circular, correct)
  output image: ./output/ios_buggy_output.png

--- ios_fixed_output ---
  shader: ResizerKernelsPatched.metal (fetched verbatim from mrousavy/react-native-vision-camera)
  circle bounding box: 259x259 px
  measured aspect ratio (width/height): 1.0000  (1.0 == circular, correct)
  output image: ./output/ios_fixed_output.png

Expected: both should print aspect ratio ~1.0 (circle stays circular).
Actual: the unmodified upstream shader distorts the circle into an ellipse;
the patched shader (same fix already running in production via patch-package)
keeps it circular.

Resizer anamorphic-squeeze tool -- Android Vulkan shader (issue #4080)

GPU: Apple M5 Pro (via MoltenVK/Vulkan)

Running shader:
  /Users/ht/dev/mre/vc-resizer-anamorphic-squeeze-repro/.build/arm64-apple-macosx/debug/ResizerAnamorphicSqueezeRepro_ResizerAnamorphicSqueezeRepro.bundle/Resizer.comp

  circle bounding box: 819x259 px
  measured aspect ratio (width/height): 3.1622  (1.0 == circular, correct)
  raw RGB888 output (1080x1920): /var/folders/1k/ljtq2j7d3vs8jr186zj_xw300000gn/T/android_buggy_output.rgb

RESULT: UNPATCHED -- shader fits before rotation; output anamorphically squeezed.

  (PNG written by Swift from the real Vulkan tool's raw output)
  output image: ./output/android_buggy_output.png

Resizer anamorphic-squeeze tool -- Android Vulkan shader (issue #4080)

GPU: Apple M5 Pro (via MoltenVK/Vulkan)

Running shader:
  /Users/ht/dev/mre/vc-resizer-anamorphic-squeeze-repro/.build/arm64-apple-macosx/debug/ResizerAnamorphicSqueezeRepro_ResizerAnamorphicSqueezeRepro.bundle/ResizerPatched.comp

  circle bounding box: 259x259 px
  measured aspect ratio (width/height): 1.0000  (1.0 == circular, correct)
  raw RGB888 output (1080x1920): /var/folders/1k/ljtq2j7d3vs8jr186zj_xw300000gn/T/android_fixed_output.rgb

RESULT: PATCHED -- shader fit is rotation-aware; output undistorted.

Additional context

This is the same class of bug on both platforms — ResizerKernels.metal (iOS Metal) and Resizer.comp (Android Vulkan compute shader) share identical logic for the cover/contain fit computation. The reproduction repo verifies both by running the actual, unmodified upstream shader source on each platform's own real GPU backend: the iOS half runs ResizerKernels.metal as a Metal compute kernel directly on the host Mac (no device needed), and the Android half runs Resizer.comp as a real Vulkan compute shader via MoltenVK (Vulkan-over-Metal, brew install glslang molten-vk vulkan-loader vulkan-headers) — no Android device or emulator needed, but the Mac does need that Vulkan toolchain installed. Both are run against the bundled unmodified and patched shader source for direct comparison.

Submission

• The reproduction I linked is either (preferred) a PR against this repo that adds a failing harness test following the harness-tests README, or (fallback) a public repo that reproduces the bug on a fresh clone. I understand the issue will be closed without one.
• I pasted logs as text (not screenshots).
• I wrote this report in my own words. I did not paste AI-generated descriptions of the bug.

[DaveyEke]

Hey @hmelonjp can you please edit the issue description to follow the issue creation template? thank you

[hmelonjp]

I have updated the description.

[DaveyEke]

Hey @hmelonjp thanks for updating the description with a great repro. I see that @mrousavy already has an open PR fixing this.

Here is the PR: #4087

[hmelonjp]

thank you so much.

[youngbrandon007]

I am running into the same issue also, thank you for working on this!

[hmelonjp]

Confirmed fixed in 5.2.1 by #4105.

Verified on the verify-upstream-5.2.1-fix branch of this issue's reproduction (vc-resizer-anamorphic-squeeze-repro): pulled the actual, unmodified ResizerKernels.metal and Resizer.comp shader source as they exist at the v5.2.1 tag and ran them through the same harness alongside the original buggy/patched comparison. Both platforms now render the test circle at a measured aspect ratio of 1.0000 (vs. 3.1622 pre-fix), matching the result of the patch we'd been carrying locally.

Thanks for the fix.