Render wgpu frames as webview overlay

[Armaldio]

Hey all 👋

I'm trying to render a wgpu frame over the webview with the goal to support the steam overlay.
Here is my advancement so far:

Kooha-2024-12-09-08-13-17.mp4

The green part is wgpu rendering to the window.
You can see the webview buttons rendering on top of it
What I'm trying to do is to have the webview under the wgpu frames with transparency (to show the overlay over the webview)
Using maybe some sort of frame ordering, I don't know

My work is based on:

• https://github.com/clearlysid/tauri-wgpu-cam/blob/main/src-tauri/src/main.rs
• Render WebView on Top of Native GPU Rendered Content [feat]  #8246
• https://github.com/FabianLars/tauri-v2-wgpu
• [bug] Flickering with raw display handle + winit + wgpu + tauri v2 + transparency #9220
• WebView2 based app and Steam Overlay MicrosoftEdge/WebView2Feedback#4019

Other solutions I've explored but don't know the viability:

• Do some headless wgpu rendering to compute a texture, send the buffer to the frontend and draw it into a canvas, on top of the rest.
  • I've see 300ms rendering on a low-end PC so i'm concerned about performance
  • I don't know how to handle mouse events since only an image of the overlay is transfered
• Use a second window:
  stacking two windows on top of each other
  make the overlay window topmost, transparent, frameless
  when overlay is present disable passthrough, otherwise enable passthrough (to allow mouse clicks and other to hit the correct window)
  • I'm also concerned it doesn't work in practice
  • What about fullscreen ?

What do you think about these solutions ?

[cdbattags]

Any code to review?

[Armaldio]

You can find two of my lib.rs experiments
I don't remember exactly how it works nor how to exactly setup, but you'll find out if you know what you are doing

https://pastebin.com/CnvXj5wF
https://pastebin.com/kWTYN1GX

I'll try to find some time later to properly setup multiple reproduction test cases

[jzade]

https://github.com/PSG-Team/tauri-steam-overlay-surface?tab=readme-ov-file I believe we have made substantial progress in this working for windows

[Elanis]

I didn't post any message, but I'm following the post as I initially raised the issue for my game (which is still not solved).
So if you need any help or review, feel free to post indeed.

---

As for your questions, I don't have the answer as I don't know exactly how it works inside, but I'd be apply to try tuning your code and seeing if I find alternative ways to fix the issue :)

[Elanis]

Since I wasn't able to make it work properly after a few tries, and have a few others important bugs. I've given up on the idea of having a Tauri app on Steam for the moment

[jzade]

Hi there, I believe we have made substantial progress. Please give this plugin a shot and let us know: https://github.com/PSG-Team/tauri-steam-overlay-surface?tab=readme-ov-file

[Dexus]

Only for Following. Hope someone get a success way to do it.

EDIT:
I would also like to add my two cents, even if it might mean a bit of overhead.

Currently, and purely theoretically, I would do the following:

Give the Tauri app a switch that can only be called up again from within its own programme – with environment or parameter, whatever. Then create a full screen with a logo or whatever, for example with winit and wgpu, create the overlay that is only started on request and provided with the corresponding URL (I'm assuming in-game payment here).

It would also be possible to create a click-through here, but not all platforms would be supported – as far as I know from a former colleague, Wayland and the mobile apps would be left out. Windows, MacOS and X11 under linux should work.

Or ask today a AI to implement something that works.

EDIT:
It would need a sidecar or own app via command parameter. As the Code would need to be on the Main Thread on MacOS. Difficult.

[skinkairewalker]

Could you submit the project to a Git repository?

[jzade]

https://github.com/PSG-Team/tauri-steam-overlay-surface?tab=readme-ov-file please review our solution which is confirmed to work through windows 11 via steam deploy.

[qwook]

steamoverlay_mac_compressed.mov

I've cracked it for OSX using the code in the pastebin's as a starting point! Let me see if I can clean it up and turn it into a plugin.

[qwook]

I've uploaded my work here so far: https://github.com/qwook/tauri-plugin-steam-overlay

I will turn it into an official plugin later, and will test on windows eventually.

Edit:
I just published it as a plugin! Use it with the following command:

tauri add steam-overlay

More details in the repo.

[blaknite]

I've been struggling with this same thing. Your plugin works perfectly on macOS but results in immediate crash on windows. I'm making some progress. I've gone from crash to drawing the wgpu frame over the game. Steam's not quite hooking cleanly.

When I crack it I'll submit a PR.

[jzade]

I've uploaded my work here so far: https://github.com/qwook/tauri-plugin-steam-overlay

I will turn it into an official plugin later, and will test on windows eventually.

Edit: I just published it as a plugin! Use it with the following command:

tauri add steam-overlay

More details in the repo.

hey Qwook,

Thank you for the inspiration. My team put this together: https://github.com/PSG-Team/tauri-steam-overlay-surface?tab=readme-ov-file and you are credited. Please let us know of issues, PRs welcome.

[jzade]

I've been struggling with this same thing. Your plugin works perfectly on macOS but results in immediate crash on windows. I'm making some progress. I've gone from crash to drawing the wgpu frame over the game. Steam's not quite hooking cleanly.

When I crack it I'll submit a PR.

try this plugin we just pushed out on MIT license last night: https://github.com/PSG-Team/tauri-steam-overlay-surface?tab=readme-ov-file

let us know what windows version, resoluition and DPI you are on when testing, thanks and cheers. PRs also welcome

[jzade]

@blaknite — following up on my earlier reply with something more useful than a link, since "drawing the wgpu frame but Steam's not quite hooking cleanly" is a specific enough symptom to guess at. We hit that exact state on the way through, and in our case it was almost always one of two things.

Init order is the most likely culprit. SteamAPI_Init has to run before you create the wgpu device. Steam's injected DLL hooks device and swapchain creation, so if the device already exists when you initialise Steam, the DLL loads and finds nothing to attach to. Everything looks correct — the overlay reports as enabled, your frames present fine — and nothing ever composites. Valve documents this ordering requirement for the OpenGL/D3D case and it applies just as strictly here. If your Steam init sits in Tauri's setup hook and the surface is created earlier, that alone will do it.

Second most likely: sizing the swapchain before the window has its real size. If you configure the surface while the decoy window is still at its default dimensions, Steam composites the overlay into that region — you get the overlay rendering into a small corner rather than filling the window, which reads as "hooking, but wrong." Create the window, get its true client size, then configure the surface.

A few others that cost us time, in case they match:

• Present continuously at vsync, even when nothing changes. Steam wants a steady Present cadence to composite against. Presenting only on demand gives a frozen or torn overlay.
• Keep the decoy surface empty. Don't render anything real into it. If you draw actual content there, you're back in the surface-contention fight with the webview that #9220 describes. Ours clears transparent and presents nothing, purely as a hook target.
• The child window needs to be click-through, not just transparent, or input stops reaching the webview underneath.
• On the immediate crash you saw on Windows: for us the Windows path needed its own child HWND rather than deriving a surface from the webview's window handle, which is the part that differs most from the macOS/Metal route @qwook took.

Implementation with all of it wired up, if it's useful as a reference rather than a dependency: https://github.com/PSG-Team/tauri-steam-overlay-surface — the README lists the invariants, each of which came from one of these failures. Verified on a Steam-launched build on Windows 11; Windows 10 is still unverified on our side.

If you do crack it independently I'd honestly rather compare notes than have you adopt ours — two implementations agreeing on the invariants would make this much more trustworthy for everyone in this thread.

[jzade]

Hey all 👋

I'm trying to render a wgpu frame over the webview with the goal to support the steam overlay. Here is my advancement so far:

Kooha-2024-12-09-08-13-17.mp4
The green part is wgpu rendering to the window. You can see the webview buttons rendering on top of it What I'm trying to do is to have the webview under the wgpu frames with transparency (to show the overlay over the webview) Using maybe some sort of frame ordering, I don't know

My work is based on:

• https://github.com/clearlysid/tauri-wgpu-cam/blob/main/src-tauri/src/main.rs
• Render WebView on Top of Native GPU Rendered Content [feat]  #8246
• https://github.com/FabianLars/tauri-v2-wgpu
• [bug] Flickering with raw display handle + winit + wgpu + tauri v2 + transparency #9220
• WebView2 based app and Steam Overlay MicrosoftEdge/WebView2Feedback#4019

Other solutions I've explored but don't know the viability:

• Do some headless wgpu rendering to compute a texture, send the buffer to the frontend and draw it into a canvas, on top of the rest.

  • I've see 300ms rendering on a low-end PC so i'm concerned about performance
  • I don't know how to handle mouse events since only an image of the overlay is transfered

• Use a second window:
  stacking two windows on top of each other
  make the overlay window topmost, transparent, frameless
  when overlay is present disable passthrough, otherwise enable passthrough (to allow mouse clicks and other to hit the correct window)

  • I'm also concerned it doesn't work in practice
  • What about fullscreen ?

What do you think about these solutions ?

Hi there, please take a look at a windows 11 solution we came up with (should also work on win 10, but we need more eyes on it) https://github.com/PSG-Team/tauri-steam-overlay-surface?tab=readme-ov-file

[SighLaw]

Awesome! But there's one more issue: the Steam screenshot feature isn't working — it only works when the Steam overlay is active. I tested this on Windows 10.

[jzade]

@SighLaw thanks for the report — you were right, and it's fixed in 0.1.3 (just published).

Turns out it had nothing to do with Windows 10, the same thing reproduced on 11. Steam's F12 works by copying the backbuffer of the swapchain it hooked, and ours never has the game in it: with the overlay closed the plugin isn't presenting at all, and with it open the buffer holds that frozen backdrop frame. So F12 either did nothing or saved a stale frame.

What shipped in 0.1.3:

• HookScreenshots(true) so Steam asks the game for screenshots instead of reading the backbuffer
• a new capture_screenshot_png helper in the plugin that grabs a live frame of the real window, which you hand to AddScreenshotToLibrary
• forwarding F12 from the frontend the same way as Shift+Tab, since the keypress lands in the webview process where Steam's input hook can't see it — that's exactly why it only worked for you with the overlay open

One trap to save you time if you wire this up yourself: don't call TriggerScreenshot from the forwarded command. It looks like the right API, but with hooking enabled the ScreenshotRequested callback never arrives — it silently does nothing. Capture directly instead. The Spacewar example in the repo has the full working wiring.

Small heads up on behavior: Steam doesn't play the shutter sound for hooked screenshots, and the "screenshot saved" toast takes a few seconds (the shot shows up in the overlay's screenshot panel immediately though). Worth adding your own flash or sound in-game for feedback.

Details in the README: https://github.com/PSG-Team/tauri-steam-overlay-surface#steam-screenshots-f12

If you get a chance to try 0.1.3 on your Win10 machine, I'd love to confirm it works there too.

[Abab-bk]

I've been looking into this issue again recently...
If I write a simple CEF wrapper, using OSR rendering, it can be hooked.
CEF's OSR is already usable, but it seems terrible on Wayland + Nvidia(performance), and I'm not sure if it works on Wayland + AMD.