Add directional light projector

[TheMagnat]

Added projector texture to the directional light (godotengine/godot-proposals#8237)

Done in this commit:

• Added projector texture (stored in the decal atlas texture)
• Added projector scale / offset in directional light
• Added directional light matrix, scale and offset in forward/mobile shaders
• Added computation logic of the projection in the forward and forward mobile shaders
• Computing projector matrix in the update_light_buffers and pushing it to the gpu
• Adapted light models to have projector scale and offset stored (even in gles3 for the future inclusion of projectors)
• Added a sub group for the directional light projector properties
• Added directional projector pipeline specialization constant for mobile and forward rendering

Exemple :

There is still one main problem, since it use the decal_atlas, the texture don't repeat well when using a projector filtering other than nearest, see this image :

I'm looking for help to find a solution to this problem, maybe we should use separate textures for the directional light projectors ?

Here is my test project to test this PR : TestProject.zip

• Maintainer edit: This closes Add support for directional light projectors or custom per-light shaders (for cloud shadows) godot-proposals#8237.

[RPicster]

Will this work with volumetric shadow? I just encountered the need for such a feature and I'm curious 😁

[TheMagnat]

Will this work with volumetric shadow? I just encountered the need for such a feature and I'm curious 😁

Currently projectors does not affect volumetric fogs, but since the work is close, I also started working on it (Still WIP and based on this branch) : #106395

[Radivarig]

Can the atlas bilinear bleed be circumvented with adding one pixel padding around the uv island coordinates?

[TheMagnat]

Can the atlas bilinear bleed be circumvented with adding one pixel padding around the uv island coordinates?

That was my first approach to resolve this issue but I don't think I found a way to do it that does not deform too much the texture, maybe I did it wrong, but I'm still working on it, if you have any idea of how to implement it, or any other ideas, I would be happy to try it

[Calinou]

Tested locally, it works as expected in Forward+ and Mobile. This is a great start 🙂

Some feedback:

• When using Compatibility and a projector texture is set, a node configuration warning should be emitted to state that it's not supported. This is already done in OmniLight3D and SpotLight3D.

• The scale property in the inspector should have | PROPERTY_HINT_LINK so that it has a "link" icon you can click to change the X and Y axes proportionally to each other.

• The scale shouldn't have a unit defined in the inspector, just like the offset. Neither are actually defined in pixels. The offset is relative to texture size, with 1.0 being a full offset that ends up looking identical to 0.0.

• Regarding seams at texture edges, I could improve this somewhat for projector textures sized by at least 33×33 using this code in light_storage.cpp:

						Rect2 rect = texture_storage->decal_atlas_get_texture_rect(projector);
						TextureStorage::Texture *texture = texture_storage->get_texture(projector);
						float inv_width = 1.0f / texture->width;
						float inv_height = 1.0f / texture->height;

						// Inset a bit to avoid black lines on texture edges (at the cost of seams).
						// NOTE: The offset is halved here, this may be wrong.
						light_data.projector_rect[0] = rect.position.x + inv_width * 0.5;
						light_data.projector_rect[1] = rect.position.y + rect.size.height - inv_height * 0.5; //flip because shadow is flipped
						light_data.projector_rect[2] = rect.size.width - inv_width;
						light_data.projector_rect[3] = -(rect.size.height - inv_height);

However, while it fixes the black lines between repeating patterns, it still leaves visible seams. These seams remain visible even when using nearest filtering, so maybe there is a way to (mostly) get rid of them.

The seams are only visible on one axis, even if the light is perfectly perpendicular to the ground. This is how the decal atlas looks with a 512×512 NoiseTexture:

Note that for the decal atlas texture to be represented with a correct aspect ratio, you need to resize the 3D editor viewport to have a 1:1 aspect ratio (use View Information in the Perspective menu to make this easier).

I've also tried to base the offset based on the decal atlas' texture (rather than our projector texture), which arguably makes more sense:

						TextureStorage::Texture *texture = texture_storage->get_texture(texture_storage->decal_atlas_get_texture());
						float inv_width = 0.0;
						float inv_height = 0.0;
						if (texture) {
							inv_width = 1.0f / texture->width;
							inv_height = 1.0f / texture->height;
						}

But it seems texture is always null when used at this point.

Having a definitive fix for this issue would also make it possible to fix #89440, which is due to the same cause.

[Radivarig]

@Calinou there's also the edge dilation technique where pixels are copied outward from the seam so it interpolates with the same color instead of the black (grey in gif below), which would solve the ticket you linked.

https://helpx.adobe.com/substance-3d-painter/technical-support/workflow-issues/export-issues/texture-dilation-or-padding.html

However atlas items are rectangular so it would be simpler to compute copying the edges outwards.

And for the seams, maybe they could be solved by tiling instead of dilating? If the decal gets marked as tiled/repeated it could copy over n pixels from the right side over to before the left side and vice versa for both axis and corners diagonally.

I think it would make bilinear interpolation amount to the same color on each of the opposite sides making the seam invisible including higher mipmaps.

[TheMagnat]

Thanks @Radivarig for pointing me to this direction, this is the solution I implemented and it seems to work well

@Calinou for the moment I left the comments and the arbitrary values for the projection matrix because I don't have any reason to put these values, it just give a good size for the effect (even if you can then use the scale property), near and far does not have any impact and size will give the initial size of the projected texture on the scene. I felt like 50 or 100 was doing great but it will vary from a project to another.

In the code I edited, I'm a bit worried by copy_to_fb. I haven't took the time to verify were it is used and if it can have some impact on other part of the engine, I put my modification in NOT_USE_MULTIVIEW branch as I felt it was the more coherent and also I write the offset in the color push_constance since it look unused in the branchs that use source texture. If there is any other draw call that may use this shader non correlated with the decal atlas, way may want to add a specific push constant value or manually set color[1 and 2] to 0.0 to prevent residual values in the memory ?
Or maybe we want to add another preprocessor constant like "USE_EXPANDED_UV" or something like this.

I also checked and this fix #89440

I left some comments as question to the reviewers that we will want to remove as soon as this PR is no longer a draft

Edit: Now that I think about it, maybe we could left the projection matrix size as a parameter in the directional light ?

[TheMagnat]

I just updated the branch, the code should be clean now

[Bonkahe]

Tested and works fine on my end, I lost my work via git shenanigan's, so I am not able to test against that work but all seems well as your fix is much better than my ham-handed attempt.

That being said this does bring up a anisotropy issue along the horizon I believe:

And it does have minor artifacts along the edges when at extreme angles and certain projector filtering settings (just mipmap seems to work pretty good, it's just anisotropic that this happens):

After talking in a rendering team meeting some time back though I believe the majority of use cases for this system should be able to work regardless of these issues (my own included), as they are mostly limited to making shadows for raymarched clouds and as such would likely be generated as a single image stretched over the whole play area, updated at regular intervals.
I would like to see this make it into master sooner rather than later, as such I will bring it up at the next meeting and see if we can get a little light on it.

I do think the scale could use a little clarification, right now it seems like (1.0,1.0) in the scale has a effective size in world as 100x100 meters when the sun is perfectly straight down, which is fine if that's intended, but I wonder if just having a size in meters would almost be better from a user friendliness standpoint?

Edit: Another possible improvement would be to add the ability to disable repeating on the projector, this would actually be ideal for my use case, but I can work with it repeating, so it's more a like to have than anything I think is required.

[TheMagnat]

I do think the scale could use a little clarification, right now it seems like (1.0,1.0) in the scale has a effective size in world as 100x100 meters when the sun is perfectly straight down, which is fine if that's intended, but I wonder if just having a size in meters would almost be better from a user friendliness standpoint?

Yes this is a good idea, I changed the scale to a size parameter that's now based on resizing the orthogonal projection matrix. Now that I thing about it, is the offset parameter really pertinent ? Since we can achieve the same result by just moving the directional light and since I removed the scale from the shader directional light data structure, it could free 4 float of space in the GPU.

And it does have minor artifacts along the edges when at extreme angles and certain projector filtering settings (just mipmap seems to work pretty good, it's just anisotropic that this happens):

After some testing, this seems to be an issue with my scaling factor that I just removed, I can't reproduce it now that I use the projection size, however I haven't tested it extensively, so if you could give me another feedback when you got the time I would appreciate it :)

Edit: Another possible improvement would be to add the ability to disable repeating on the projector, this would actually be ideal for my use case, but I can work with it repeating, so it's more a like to have than anything I think is required.

Yes I can see some use case where you would want to do this instead of a spotlight, I can work on an option to deactivate it, but how should it react, should it display full light outside of the texture or full shadow (and in this case It become some sort of spotlight) or maybe clamp to the last pixel of the texture (but this feel weird) ?

[Bonkahe]

Yes this is a good idea, I changed the scale to a size parameter that's now based on resizing the orthogonal projection matrix. Now that I thing about it, is the offset parameter really pertinent ? Since we can achieve the same result by just moving the directional light and since I removed the scale from the shader directional light data structure, it could free 4 float of space in the GPU.

I think freeing up the memory is probably more important than having two separate positions that effectively do the same thing, that being said I'm sure the user-friendliness of just modulating that will appeal to a fair number of people.
Since we already have it in I think it might be best to just leave it, if your not using the last float of that vec4 you could use it for controlling if repeating is enabled or not though.

Yes I can see some use case where you would want to do this instead of a spotlight, I can work on an option to deactivate it, but how should it react, should it display full light outside of the texture or full shadow (and in this case It become some sort of spotlight) or maybe clamp to the last pixel of the texture (but this feel weird) ?

In my case, and I think most use cases would be similar, I will be using it for cloud shadows which will be driven via compute. As such having it go to full light in the distance would likely be ideal, but I can control that by putting the edge pixel of the image to black or white as necessity requires, for this purpose the ideal solution is use the last pixel, that way if It's overcast I can fade off distance directional light on everything outside of the play area by just a value 0.0-1.0 by controlling that last pixel.

After some testing, this seems to be an issue with my scaling factor that I just removed, I can't reproduce it now that I use the projection size, however I haven't tested it extensively, so if you could give me another feedback when you got the time I would appreciate it :)

Absolutely! Thank you so much for the effort your putting into this, I'll test it when I get back today, right now I'm away on family events, but I am excited xD

Edit: One thing I would need to do what I want to do is have a dirty function to trigger to let the directional light know that the texture has been updated in order to pass that update to the decal sheet. (Speaking of which I feel like this would be useful for decals as well, as having a clean way to notify it of changes would allow for animated decals so long as the framerate isn't too high)
Not sure if this is already in the system, I am not aware if it is or not, I believe it is not though.

[Calinou]

(Speaking of which I feel like this would be useful for decals as well, as having a clean way to notify it of changes would allow for animated decals so long as the framerate isn't too high)

DrawableTexture should be able to handle this automatically thanks to #115653. As for other texture types such as ViewportTexture, this is being tracked in #73400.

[mertkasar]

I've been eyeing this PR for a while, it's a feature I'd like to use, but it seems a bit stale lately. How open are you to implementing this by bypassing the decal atlas?

DirectionalLight3D lights entire levels so repeatability is essential for the projector texture. Seems like atlas is making this a bit difficult. Skipping the atlas seems inefficient on paper but realistically how many directional lights does a scene have? Even the engine caps them at 8. One or two extra texture depending on the scene might be a fair trade-off.

The other upside of a standalone texture is that it opens the door to dynamic projectors. I did a quick test on top of this PR in a separate branch , bypassing the atlas and allowing ViewportTexture as projector, it shifts two textures with different movement patterns and uses them as a projector;

directional-projector-demo.mp4

I'm not sure if a dedicated texture binding is the best long-term approach though. A dual path could work: static textures go through the decal atlas, dynamic ones (ViewportTexture, DrawableTexture) get a standalone slot in a texture2DArray binding that bypasses it. This avoids unnecessary bindings for static projectors while not forcing dynamic textures through an architecture that wasn't designed for them. Same pattern could extend to spot/omni or even decals (#73400, #74352, #74353)

Either way, I think dynamic projector support is worth pursuing. There's clear demand for it (godotengine/godot-proposals#8237). I know there's also work being done to support dynamic textures within the atlas itself (#115653), so maybe that ends up being the preferred path.

Just wanted to share what I've been experimenting with.

---

project: directional-light-projector.zip

[Calfur]

I just tested the implementation from @mertkasar merged to 4.6.1 to create a cloud projection based on two noise textures in a SubViewport. It's exactly what I'm looking for, thank you!

SubViewportCloudProjection.mp4

[TheMagnat]

Thanks for your contribution to this PR @mertkasar. Using an a standalone texture was my first guess to bypass the repeat atlas problem and in the current state of this branch, it work well and no artifact can be found (at least in my testing). As said by @Calinou :

DrawableTexture should be able to handle this automatically thanks to #115653. As for other texture types such as ViewportTexture, this is being tracked in #73400.

In 4.7 using DrawableTexture, it should work automatically with this PR to have animated textures in the decal (Even if in the long term, I think having a way to inject our own shadow code in the main shader should be the optimal way to go for this)

[TheMagnat]

I've just updated the PR to be up to date with the current master.
I will update it again when #115653 get merged to adapt the modifications to the decals texture in the new function too.