Fix projective rendering for image source quads

[i4innovationnet]

Launch Checklist

• Confirm your changes do not include backports from Mapbox projects (unless with compliant license) - if you are not sure about this, please ask!
• Briefly describe the changes in this PR.
• Link to related issues.
• Include before/after visuals or gifs if this PR includes visual changes.
• Write tests for all new functionality.
• Document any changes to public APIs. No public API changes.
• Post benchmark scores. Not applicable; no benchmark was run.
• Add an entry to CHANGELOG.md under the ## main section.
• Confirm you have read our AI policy here.

Summary

Fixes projective rendering for image source quads whose four coordinates do not form a parallelogram.

The raster shader now carries homogeneous texture coordinates for image, canvas, and video source quads and divides by the projective denominator in the fragment shader before sampling. Parallelogram quads and regular tiled raster sources keep affine behavior through an identity perspective transform.

ImageSource computes the complete inverse-homography denominator w = ax + by + c, rejects singular, non-finite, and zero-crossing transforms, and normalizes the homogeneous coefficients without assuming that c is nonzero. This preserves valid convex trapezoids while failing closed for degenerate, concave, and self-intersecting coordinates.

The texture correction remains separate from clip-space projection: projectTile owns gl_Position.w for Mercator, globe, and terrain paths, while this change performs the texture divide explicitly in the fragment shader.

Related Issue

Fixes #7886.

Visuals

The before screenshot and original reproduction are in #7886.

Render fixtures cover the corrected non-parallelogram case, a valid trapezoid whose homogeneous denominator has no constant term, and the projection modes discussed during review:

• test/integration/render/tests/image/projective/style.json
• test/integration/render/tests/image/projective-trapezoid/style.json
• test/integration/render/tests/projection/globe/image-non-parallelogram/style.json
• test/integration/render/tests/terrain/image-projective/style.json

The projective fixtures use projective-grid.png and include semi-transparent vector outlines so the expected corner geometry remains visible.

Reviewer Follow-Up

• Zoomed out the Mercator fixture so the complete image is visible.
• Added semi-transparent vector outlines to the projective fixtures.
• Added Mercator and terrain projective coverage, a globe non-parallelogram compatibility fixture, and zero-constant trapezoid coverage.
• Removed the test-only math export and raster-program plumbing test.
• Kept the transform calculation and state in ImageSource; regular tiled raster sources explicitly pass identity coefficients.
• Consolidated the perspective-transform type and identity coefficients in the raster program and reused them from draw and image-source code.
• Reverted unrelated draw-call formatting churn so the rendering diff stays focused.
• Confirmed initial coordinates flow through _finishLoading() to setCoordinates(); added an initial-load regression test instead of duplicating the calculation in load().
• Replaced hand-written determinant, cofactor, and corner-dot expressions with gl-matrix primitives backed by plain JavaScript arrays to preserve double precision.
• Added forward-denominator and cancellation-aware inverse-denominator validation, including a regression for a degenerate quad that previously slipped through the epsilon guard.
• Removed the unclear degenerate render fixture; numerical fallback behavior is covered through production ImageSource state tests.
• Restored the raster-resampling fixture to local://image/0.png, recentered it over a high-contrast region so nearest and linear exceed the render-test threshold, and regenerated its expected output.
• Renamed the globe fixture to image-non-parallelogram so its name reflects compatibility coverage rather than implying that it discriminates the projective correction.
• Replaced the custom tolerance with Number.EPSILON-scaled singularity checks.
• Converted the Heckbert source note to TSDoc with archive and PDF links.
• Preserved valid homogeneous denominators with c = 0 and rejected denominators that reach or cross zero inside a quad.
• Merged current main and regenerated the combined bundle-size baseline.

Validation

Local validation on Node 24.11.1, after merging current main:

• npm ci
• npm run lint -- --max-warnings 0
• npm run lint-css
• npm run generate-typings
• npm run typecheck
• npm run generate-docs
• npm run build-dist
• npm run test-unit-ci — 198 files, 2,917 tests passed
• npm run test-build-ci — 8 files, 699 tests passed
• npm run test-integration-ci — 4 files, 176 tests passed
• npm run test-render -- -t 'tests/(image/(projective|projective-trapezoid|default|raster-resampling)|projection/globe/image-non-parallelogram|terrain/image-projective|canvas/default|video/default|raster-resampling/default)' — 9 affected render tests passed
• git diff --check

Latest focused validation on Node 26.7.0 from .nvmrc:

• npm run test-render -- -t 'tests/(image/(raster-resampling|projective|projective-trapezoid)|projection/globe/image-non-parallelogram|terrain/image-projective)' — 5 passed
• Linear negative control for image/raster-resampling — failed as intended with 6 significant pixels against 1 allowed
• git diff --check

The prior Windows integration failure was a Puppeteer navigation timeout in an untouched query-test setup path. It did not reproduce locally; the full integration suite passed twice after the production build prerequisite.

[codecov]

Codecov Report

❌ Patch coverage is 98.27586% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.09%. Comparing base (8ee0ba3) to head (d415f0e).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/source/image_source.ts	98.07%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7887      +/-   ##
==========================================
+ Coverage   92.79%   93.09%   +0.29%     
==========================================
  Files         300      300              
  Lines       24904    24898       -6     
  Branches     6528     6540      +12     
==========================================
+ Hits        23109    23178      +69     
+ Misses       1795     1720      -75     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[HarelM]

Thanks for taking the time to open this PR, please make sure to fill the PR template.

[i4innovationnet]

Thanks, filled in the template now.

[CommanderStorm]

Nice work.

I have not quite undestood the math behind this PR.
The render tests look good, but I don't understand the underlying math.
Could you give me a bit of a pointer?

[CommanderStorm]

honestly I think that the grid image that you have is likely a better for the render tests.
Should we add it to that grid image here instead of the beach?

[HarelM]

I'm a bit confused from the before and after image here, is this a problem that was intorduced as part of this PR as I wouldn't expect this change - resampling should show pixelated image, and that's not the case here.
Am I missing anything?

[CommanderStorm]

I guess yes, for resampling the non-projection is maybe better.

We somewhat care about both projections and reasampling though, so lets revert this, that was a bad suggestion for this one.

[i4innovationnet]

Restored this fixture to local://image/0.png and regenerated its expected output, so it continues to cover nearest-neighbor resampling. The grid image is now limited to the new projective-rendering fixtures.

[HarelM]

I still expect this not to change, can you provide an explanation on why the image had changed?

[i4innovationnet]

Thanks for following up. I investigated this specifically because the new expected image also looked as though nearest-neighbor sampling might have stopped working.

It has not. The changed output is caused by correcting the mapping of this fixture’s non-parallelogram image quad.

In calculated tile coordinates, the fixture’s corners are:

TL (5320, 7456)
TR (7240, 4232)
BR (9600, 5568)
BL (7904, 8904)

They have a parallelogram closure error of (-224, -112), so this fixture receives a genuine non-identity projective transform:

[-1.30458979876778e-5, 2.07071621334893e-5, 1]

I compared upstream main at 8ee0ba36a with this branch at 65a93ecb4, using Node 26.7.0 from .nvmrc, headless Chrome, and SwiftShader. The upstream fixture passes its upstream expected image, and the unchanged fixture on this branch passes the updated expected image.

The sampling checks were:

• On this branch, switching only raster-resampling from nearest to linear changes 2,972 of 4,096 pixels exactly (72.5586%). If nearest were being ignored, this toggle would not change the output.
• Replacing normal sampling with an exact level-0 texelFetch at the projectively divided UV produces a byte-for-byte identical image. A deliberately incorrect +1-texel control changes 84.9% of pixels, confirming that the control can detect the wrong texel.
• An independent texel-cell model fits the branch render under projective nearest sampling 40–80× better than the incorrect affine pairing.
• GPU derivative instrumentation reports LOD -2.039 on every fragment (rho = 0.243), so all fragments are magnified. The texture minification filter and its LINEAR fallback are never used.

The image looks smoother because the corrected homography samples a different, flatter portion of the source photograph—not because the filtering became linear. At the viewport centre, the sampled source position moves from approximately (518.55, 231.40) to (498.17, 242.02), a displacement of about 23 source pixels.

Two controls isolate this further:

• Forcing the transform to identity reproduces the upstream render byte-for-byte.
• Closing the coordinates into an exact parallelogram makes upstream and this branch byte-for-byte identical.

The expected-image update is therefore required: on the same renderer, the upstream and corrected outputs differ by 267 significant pixels, while this 64×64 fixture allows only 1.

One separate issue surfaced: this old fixture is not a strong resampling regression test. On current main, changing it from nearest to linear still passes its checked-in expected image at the suite threshold, despite many small exact differences. That weakness predates this PR and should be addressed separately with a high-contrast, exact-parallelogram fixture.

So the conclusion is that this is intended projective behavior, not a nearest-neighbor regression, and no production shader or filtering fix is needed for this expected-image change.

However, one additional test-scope issue surfaced during this validation.

The new projection/globe/image-projective fixture does not currently discriminate the projective correction. With the correction completely disabled, it still passes: 4,763 of 262,144 pixels differ exactly, the maximum channel difference is 1, and 0 pixels are significant at the render-suite threshold, against a 65-pixel allowance.

This occurs because globe raster geometry is subdivided at least 32×32. The interior mesh vertices use bilinearly generated positions with linear texture coordinates, so applying the quad homography only at those vertices is effectively inert. This differs from the single-quad Mercator path, where the correction reproduces the inverse homography exactly.

I suggest retaining the fixture as useful globe compatibility coverage, but renaming it in this PR to projection/globe/image-non-parallelogram and changing its description to:

Tests rendering of a non-parallelogram image source on globe projection.

The expected image can be moved unchanged. Duplicating the fixture would not add independent coverage because neither copy would detect the projective correction.

Making subdivided globe rendering genuinely projective requires defining how the homography should interact with the curved mesh and should be handled as a separate design ticket. It does not block this PR or affect the conclusion that nearest-neighbor sampling remains active.

[HarelM]

Thanks for the research. I think it would be good to move the center of this test to a different place that will show the feature - e.g. show a pixelated image, which is what the resampling feature is.

Feel free to do any other renames you think will be useful, or change the description inside the style.json to improve the test description.

Thanks!!
If you prefer to do it in another PR so this can be merged, that's fine too, just let me know...

[i4innovationnet]

Implemented in d415f0efe.

I moved the fixture center to:

[-122.5152017, 37.562688978]

This keeps the same image, zoom, and non-parallelogram coordinates, but places the viewport over the high-contrast rock/surf region. The nearest render is now visibly pixelated, while the linear render is visibly smoothed.

The fixture now meaningfully enforces both behaviors:

• Nearest versus linear changes 3,902 of 4,096 pixels exactly, with 6 significant pixels against the test's 1-pixel allowance.
• With projective correction disabled, 1,757 pixels are significant, so the fixture remains strongly sensitive to projective mapping.
• Repeated renders of both resampling modes were byte-for-byte stable.

I regenerated the expected image and added a description explaining that this tests nearest-neighbor resampling on a projectively mapped image source.

I also renamed the new globe fixture from image-projective to image-non-parallelogram, since it provides globe compatibility coverage but does not independently discriminate the projective correction. Its expected image was moved unchanged.

The focused render suite passes. No production-code change was required.

[i4innovationnet]

Nice work.

I have not quite undestood the math behind this PR. The render tests look good, but I don't understand the underlying math. Could you give me a bit of a pointer?

The raster shader needs a mapping from rendered quad position (x, y) back to image texture coordinates (u, v). For a non-parallelogram quad this mapping is projective rather than affine.

In homogeneous form:

[u', v', w] = H^-1 [x, y, 1]
u = u' / w
v = v' / w

The CPU code computes the complete denominator row of that inverse homography:

w = ax + by + c

Those three coefficients are homogeneous: multiplying all of them by the same nonzero scale does not change u'/w or v'/w. The implementation therefore normalizes by the largest coefficient instead of assuming c = 1; this is required for valid projective transforms where c = 0.

Before uploading the transform, ImageSource evaluates w at all four corners and falls back to affine sampling if the denominator is non-finite, reaches zero, or changes sign. That prevents a divide by zero inside concave, self-intersecting, or degenerate quads.

The vertex shader passes (u * w, v * w, w), and the fragment shader samples at xy / z. For parallelograms and regular tiled raster sources, the identity coefficients [0, 0, 1] keep the existing affine behavior.

The derivation is documented in the ImageSource TSDoc with the Heckbert archive and PDF links.

[i4innovationnet]

Superseded by #7903 with a clean linear commit history and neutral branch naming.

[HarelM]

We are squash-merging anyway so git history isn't important in the branch.
I prefer to keep the conversation in one place and make sure the review comments are addressed.
Any chance you can reopen this one and close the other one please?

[i4innovationnet]

Reopened #7887 and closed #7903 as requested. The original PR now points to the same single clean commit (91e5ffb) that was verified in #7903, so no implementation or test changes were lost. All 12 existing review threads remain resolved.

[i4innovationnet]

The fresh CI run completed with one unrelated Windows render timeout in high-pitch/terrain-pitch95 after 268 tests passed. The same shard passed previously on this exact commit, and all other required checks are green. I do not have repository permission to rerun the failed Actions job; could a maintainer please rerun that job?

[HarelM]

I've added my comments, once addressed this could be merged. Thanks!

[i4innovationnet]

@HarelM Done.

[HarelM]

2 open comments left, otherwise can be merged. Thanks!

[HarelM]

Great work!! Thanks!

[HarelM]

@i4innovationnet can you take a look at the videos in the following PR:
mapmelabs/maplibregl-area-transform#13

It's a tool I'm helping maintain that was waiting for this fix to allow changing an image corners.
There are two videos there, one using this fix and the other is using a custom layer.

Do you think there might be an edge case here that needs to be fixed?

[HarelM]

CC: @igalgh

[i4innovationnet]

@i4innovationnet can you take a look at the videos in the following PR: mapmelabs/maplibregl-area-transform#13

It's a tool I'm helping maintain that was waiting for this fix to allow changing an image corners. There are two videos there, one using this fix and the other is using a custom layer.

Do you think there might be an edge case here that needs to be fixed?

I don’t believe the native video shows a bug in #7887. ImageSource uses a projective homography, so as one corner approaches the opposite diagonal, almost the entire source image becomes compressed into a very small part of the quad. Tiny corner movements then produce very large changes in the visible texture, which explains the apparent rotation or snap.

The effect is continuous in the video—even around the apparent flip—and the grid lines remain straight without the old affine diagonal seam. The custom layer behaves differently because it uses a subdivided bilinear mesh. That spreads the image more evenly across the quad, which is often preferable for free-form editing, but it is a different warp model rather than a correction to the projective mapping.

There is one real boundary caveat: at exact collinearity, or once the quad becomes concave, no valid invertible homography exists. The native implementation deliberately falls back to affine sampling there, which can cause an abrupt change and restore the diagonal seam.

I agree that focused regression coverage would be useful. I suggest testing:

• a convex quad immediately before collinearity, confirming finite and seam-free projective output;
• exact collinearity, confirming the deliberate affine fallback;
• a point just past the diagonal, confirming deterministic fallback behavior.

For the editing tool, the best protection is to keep corners away from that boundary using a scale-relative distance or area constraint rather than a fixed two-screen-pixel margin.

So my conclusion is: no projective-math fix appears necessary for valid convex quads, but the near-degenerate and fallback behavior should be explicitly tested. Matching the custom layer would require a separate bilinear warp mode with mesh subdivision. So this is a separate, pre-existing near-degenerate-quad edge case that needs its own fix.

[HarelM]

Not sure I fully understand everything you wrote, but I think the near triangle shape feels like a bug worth fixing here, wouldn't you agree?

[i4innovationnet]

@HarelM Yes, I agree that the near triangle case feels like a bug and is worth fixing.

The distinction I’m making is that it is not a regression in #7887. Before it, the near triangular and other non-parallelogram quads were already rendered incorrectly as two affine triangles with a diagonal seam. #7887 fixes the projective mapping for valid quads, but the behavior as a quad approaches degeneracy remains an existing edge case.

So I suggest addressing it in a separate follow-up.

[HarelM]

Ahh, no, definitely not a regression due to this PR, just another case worth fixing which I hoped you could take a look at since it seems you understand this code area very well.

[HarelM]

I've opened the following PR:

• Add a mesh to image source to solve edge cases. #8172

Any feedback is welcomed.