Inaccurate colors

[lucadalli]

Compared below are two 8-bit MP4s.
The first one is recorded with OBS and displays colors correctly, the second one is muxed with MP4-muxer as shown in https://github.com/lucadalli/mp4-muxer-colorspace/blob/main/src/main.ts

As you can see, the greens in the videos muxed with MP4-muxer are a lot more dull.
While investigating the issue I came across these issues/bug reports reported by yourself. I'm thinking this could be somewhat related?
w3c/webcodecs#598
https://issues.chromium.org/issues/40061457

FFprobe output when checking bit bepth (both 8)

obs-color.mp4

mp4-muxer-color.mp4

[Vanilagy]

Are you using the BT.709 color space for recording in OBS? It may be set to BT.601 or something by default. I actually remember OBS having incorrect colors by default, having way darker greens than what I recorded on my monitor.

[lucadalli]

Yes, my OBS is configured to use Rec. 709 (I think by default)

[lucadalli]

Updated the lucadalli/mp4-muxer-colorspace repo to demonstrate something that's quite bizarre but very interesting.

If the videoTrack frames are drawn to a canvas, and new frames are created from said canvas, the output with mp4-muxer is the same, but with MediaRecorder the recording displays the correct colors!

canvas-mediarecorder.mp4

canvas-mp4muxer.mp4

[Vanilagy]

@lucadalli I appreciate the in-depth digging you're doing! I was down this color space rabbit hole a while ago, as noted in the issues you sent.

I may need to add the colr box to my muxed output containing information about the color space. Perhaps this will fix the behavior. I remember when I did this for WebM (adding the Colour element), it didn't fix anything as the VP9 codec stream contained color space information baked into it, and the Chromium team was setting the color space incorrectly in the VideoEncoder implementation. The issue I opened on the Chromium bug tracker was quite active afterwards, so I would hope this issue is fixed in all other codecs too. It could also be that the AVC codec stream emitted by Chromium does not contain color space information, and therefore we'll need the colr box in the container to complete it.

I'll need to play around with it.

[Vanilagy]

Made some local modifications to my muxer to add the colr box - and it doesn't seem to fix it! Check this out:
https://github.com/user-attachments/assets/747676e7-9ebf-4bc3-8a67-fb9b5f80ae16

This uses the following values in the colr box:

These values I obtained by using the tables specified in the "Colour" section here combined with the colorSpace object within the decoder config:

I also tried setting them all to 1, which is what the OBS export did. Results in this file:
https://github.com/user-attachments/assets/8b587d35-396d-4b49-8db6-bb74818558de

Both of the greens don't appear to match the actual green I see on the site.

So, I'm not sure if I'm doing anything wrong here, or if this is a Chromium issue once again. Perhaps it would be good to report this specific case to them, too? Last time I reported it, it was WebP and VP9-related.

[lucadalli]

I think it's best if we start sharing images of side-by-side comparisons because in my case both of your video files accurately show the colors on the website! 😵‍💫
Left is first vid, middle is second vid, right is website

Edit: When viewed on my phone (Pixel 8), the greens are dull and do not match the colors on the site.
Edit 2: It briefly flashes the right colors, then settles on the dark greens during playback. I see it with my eyes, but when I try to record it with the phone's screen recorder, the changes are not shown in the recording...

[lucadalli]

Here's a side-by-side comparison of canvas-mediarecorder.mp4 and canvas-mp4muxer.mp4 I shared previously

[lucadalli]

@Vanilagy The color space of the VideoFrame created from canvas is
{ primaries: 'bt709', transfer: 'iec61966-2-1', matrix: 'rgb', fullRange: true }

Could you try put those values in the colr box?

[Vanilagy]

I just released a new version 5.1.4 which contains this colr box, since I figured it's probably right to include it regardless of the conclusion of this issue. Go play around with it on your end, see if it improves anything.

[Vanilagy]

@lucadalli How's it going?

[lucadalli]

@Vanilagy Tried with 5.1.4, no change.

I created a bug report on the Chromium Issue Tracker.
In hindsight, I should've have emphasized the difference between browser and monitor MediaStreams instead of the difference observed with the DRAW_TO_CANVAS flag.
I was hoping for some update/comment of significance before updating you here. So far it's yet to be triaged.

In the meantime I'm converting to YUV420 manually as per your comment.