Increased input-to-screen latency on vulkan

[adrian17]

Description
Some users (of Ruffle) reported noticeable lag between input and game response, we were very surprised to realize that it doesn't seem to be related to event handling and that it only happens on Vulkan wgpu backend.

Repro steps
In general: write any code that's based on user input, observe the latency between user input and visible response.

Specific repro:
https://github.com/adrian17/wgpu-latency-repro
(This is based on tutorial code https://github.com/sotrh/learn-wgpu/tree/master/code/beginner/tutorial4-buffer , I just updated wgpu and added code to make the triangle follow the mouse)

Just run the code with cargo run --release, move the mouse and observe the triangle following the mouse. Test with different wgpu::Backends values.

Expected vs observed behavior
On DX12 and GL, the triangle seems to follow the cursor very closely, seems to be a frame behind the OS cursor, the latency isn't really noticeable unless I pause the recording. (GL seems to occasionally lag behind slightly more, but that might be my observation error)
On Vulkan, the triangle very noticeably lags behind the cursor.

Extra materials
Vulkan demo:

repro_vulkan.mp4

DX12 demo:

repro_dx12.mp4

Platform
Windows 10, tested on wgpu 0.23, 0.24.

[cwfitzgerald]

Try playing with https://wgpu.rs/doc/wgpu/type.SurfaceConfiguration.html#structfield.desired_maximum_frame_latency and see if that helps things out?

[adrian17]

Reducing it to 1 indeed seems to reduce visible latency a bit, but it's still noticeably worse than on DX12.

I know it's not much of a scientific measurement, but to my naked eye on DX12 (no matter what desired_maximum_frame_latency value I use) the drawn cursor appears nearly glued to the OS cursor, while with Vulkan it always seems to be trailing behind.

[Wumpf]

Someone documented this here as well and was able to demonstrate it for the different backends

• High input latency with wgpu backend emilk/egui#5037 (comment)

[aloucks]

You're using Fifo with Vulkan and Mailbox with DX12. I'm not sure why the surface caps are reported in different orders but when you keep the present mode fixed, the discrepancy with the backends goes away. Both DX12 and Vulkan have high latency with Fifo and both have less latency with Mailbox or Immediate. Interestingly, the GL backend has less latency that Vulkan and DX12 in Fifo mode.

[adrian17]

Oops, you're right. That said, the original issue in Ruffle was present even though we already only used Fifo there.

And indeed, when I force present_mode: wgpu::PresentMode::Fifo in the reproduction code, I still see the difference (it's less visible on 30/60fps recordings, but more obvious on an 144hz monitor). Across values of desired_maximum_frame_latency between 1 and 10, it always feels like the dx12 latency is several frames lower / halved.

Here's another repro, with Fifo, desired_maximum_frame_latency: 1 and a cursor movement based on SystemTime. Top is Vulkan, bottom is DX12. Vulkan variant is visibly behind. (for reference, the source for this is at https://github.com/adrian17/wgpu-latency-repro/tree/system-time-comparison )

chrome_2025-01-18_12-58-24.mp4

(With Mailbox, they appear equal. However, Vulkan+Mailbox appears to use way more CPU, seemingly a full core all the time. Is this a driver quirk, to be expected...? Mentioning this on a side, since the main topic is about Fifo latency.)