macOS port

[Ivshti]

Summary

This draws inspiration from https://github.com/Ivshti/playback-native/blob/master/index.js, which is a VLC+WebView combo on macOS

Turns out that we can do the same with libMPV, since it can be given a pointer to NSView as the wid (like HWND on Windows): https://github.com/mpv-player/mpv-examples/blob/master/libmpv/cocoa/cocoabasic.m#L104

The only missing piece is how we initiate the window and webview.

WebView libraries

Most of them use WKWebView (“Starting in iOS 8.0 and OS X 10.10, use WKWebView to add web content to your app.Do not use UIWebView or WebView.”)

• wry - made by tauri; directly written over cocoa/objc; for whatever reason it doesn't load https://app.strem.io/shell-v4.4
• WebView - originally by zserge (now webview/webview), has bindings for rust (https://github.com/Boscop/web-view or https://github.com/webview/webview_rust); uses a C lib which is suboptimal
• rust-darwin-webkit: https://github.com/yamadapc/rust-darwin-webkit - looks close to native, this is good; maybe a bit outdated
• cacao: https://docs.rs/cacao/latest/cacao/webview/index.html - it is kinda high level (follows apple-is high-level patterns) and it doesn’t support transparency (setting drawsBackground false)
• cocoa and objc directly - wry and rust-drawin-webkit use this directly; see https://github.com/search?l=Rust&q=+cacao%3A%3Awebview&type=Code

The best option is probably to use cocoa/objc directly, as the code will port almost 1:1

Here's an initial prototype: https://github.com/Ivshti/wry/blob/dev/examples/transparent.rs

[Ivshti]

Initial prototype here: https://github.com/Ivshti/wry/blob/dev/examples/transparent.rs

next steps:

• figure out the resize crash - the prototype crashes when resizing on M1 mac
  • try the cocoa example of libmpv to see if it crashes - this one crashes, so it's a bug in MPV
  • cocoa-rendergl libmpv example is also worth reading and testing - this one doesn't crash and has the same performance (see below)

Two interesting things discovered in those tests

• cocoa-rendergl does not init OpenGL with OpenGL 3 (NSOpenGLProfileVersion3_2Core), which threw "[libmpv_render/videotoolbox] error: need >= OpenGL 3.0 for core rectangle texture support" and degraded performance a bit; after fixing this, it's performance was equal to the cocoa example and the shell prototype
• the shell prototype generated a bit more load for WindowService because MPV logging was enabled (msg-level all=v, terminal yes)

[Ivshti]

next steps:

• compare cocoa-rendergl vs the cocoa vo
• figure out scaling in cocoa-rendergl
• figure out resizing in cocoa-rendergl (works but is slow)
• research IINA and other cocoa+rendergl solutions
• try a different version of libmpv - not doing this cause it's irrelevant; even if we get this to work, cocoa gpu context is obsolete, so it's best to use the mpv render API anyway
• profile GPU usage as well between old stremio, iina, cocoa renderfb, transparency (new shell proto)

[Ivshti]

Another update

• resizing is slow because everything runs in the main thread and goes through a objc queue waiting for it's turn
  • read IINA's video view to research how threading works there
• dimensions are wrong in the example, and not wrong in other impls, because we use the "virtual" pixels we get from contentView bounds, and IINA/mpv (video/out/mac) get them from glViewport (real pixels).

[Ivshti]

It appears mpv has deprecated window embedding on macOS so this prototype should be re-made.

[Ivshti]

some performance tests done on an m3 max mbp, measuring CPU time and GPU time over 2 minutes of BBB (https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4)

all tests done with mpv 0.39.0, except stremio

• cocoabasic 35.9 / 6.13
• cocoa-rendergl 32.42 / 4.41
• mpv: 41.52 / 6.16
• stremio: 49 / 5.4 (INTEL)
• opengl example from libmpv-rs 36.44 / 9.72
• opengl example with sdl2 from libmpv2-rs 36.60 / 7.85

the good news is that the best performer is the cocoa-rendergl libmpv example, which is what we'd be basing our shell on

[Ivshti]

closing in favor of #9