Gamescope caches wl_shm buffers and does not upload new contents to vulkan texture

[leroycep]

Is there an existing issue for this?

• I have searched the existing issues

Are you using any gamescope patches or a forked version of gamescope?

• The issue occurs on upstream gamescope without any modifications

Current Behavior

wl_shm backed wl_buffers are not updated after the first time they are committed, leading some applications to appear frozen.

Steps To Reproduce

1. Download zig nightly build (I'm on version 0.14.0-dev.3234+e5174c744, I think the minimum nightly you will need is version 0.14.0-dev.3179+933ba935c)
2. Clone https://codeberg.org/klaji/shimizu
3. Build shimizu examples
4. Run examples under gamescope, exposing a wayland session

shimizu> zig build -Dinstall-examples
shimizu> gamescope  --expose-wayland -- ./zig-out/bin/shimizu_example_02_gradient

Hardware information

- Distro: Arch Linux
- CPU: AMD Ryzen 9 3900X 12-Core Processor
- GPU: AMD Radeon RX 580 Series (RADV POLARIS10)
- Driver Version: Mesa 24.3.4-arch1.1

Software information

- Desktop environment: niri
- Session type: wayland
- Gamescope version: 3.16.1 (gcc 14.2.1)
- Also testing with built from source Gamescope version: 3.16.1-43-g0a09639
- Gamescope launch command(s): gamescope  --expose-wayland -- %command%

Which gamescope backends have the issue you are reporting?

• Wayland (default for nested gamescope)
• DRM (default for embedded gamescope, i.e. gamescope-session)
• SDL
• OpenVR

Logging, screenshots, or anything else

I'll copy the text from my tracking issue for shimizu: https://codeberg.org/klaji/shimizu/issues/1

Running the gradient example under gamescope results in what appears to be a still image:

gamescope --expose-wayland -- ./shimizu_example_02_gradient

gradient-under-gamescope-session.mp4

Which differs from the result I see when running under niri (and other wayland sessions).

./shimizu_example_02_gradient

gradient-under-niri-session.mp4

As far as I can tell, this issue appears because gamescope caches the imported buffer, and doesn't update it:

    // gamescope/src/steamcompmgr.cpp:1297
	if ( gamescope::OwningRc<CVulkanTexture> pTexture = s_BufferMemos.LookupVulkanTexture( buf ) )
	{
		// Going from OwningRc -> Rc now.
		commit->vulkanTex = pTexture;
		return commit;
	}

A potential fix on the shimizu side is destroying the wl_buffer every time and recreating when we grab a framebuffer to render to.

It's not clear to me whether this is a shimizu bug, or a gamescope bug. The protocol spec doesn't call out this specific case, and the closest thing answer is this paragraph in the wl_surface:attach request:

Committing a pending wl_buffer allows the compositor to read the pixels in the wl_buffer. The compositor may access the pixels at any time after the wl_surface.commit request. When the compositor will not access the pixels anymore, it will send the wl_buffer.release event. Only after receiving wl_buffer.release, the client may reuse the wl_buffer.

So perhaps this is something that needs clarification from wayland-protocols.