Skip to content

vo_gpu_next: set color directly using Wayland protocol#17345

Open
kasper93 wants to merge 9 commits intompv-player:masterfrom
kasper93:wayland-color
Open

vo_gpu_next: set color directly using Wayland protocol#17345
kasper93 wants to merge 9 commits intompv-player:masterfrom
kasper93:wayland-color

Conversation

@kasper93
Copy link
Member

(yep)

@kasper93 kasper93 force-pushed the wayland-color branch 4 times, most recently from 75522dd to e9ec715 Compare January 28, 2026 20:37
@kasper93 kasper93 requested a review from Dudemanguy January 28, 2026 21:11
@kasper93
Copy link
Member Author

@mahkoh: This should work, and allow us better control over how things work. Needs https://code.videolan.org/videolan/libplacebo/-/merge_requests/796 and more fixes, because the fact the color update is not synchronized with rendered is bit jarring to look at.

Comment on lines +4194 to +4195
if (wl->color_manager && !wl->color_surface)
wl->color_surface = wp_color_manager_v1_get_surface(wl->color_manager, wl->callback_surface);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So does this set the color_surface before mesa gets a chance to? I'm a little confused on how this doesn't blow up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is no descriptor by default, and with libplacebo change it won't be created, I need to add some code to go back from pass-through when target-colorspace-hint is disabled, will do later, need to destory our color surface

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have this hardcoded to only dmabuf-wayland in vo_wayland_init at the moment. It can't just be created there?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is to create it on demand. Same as existing hint works.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want, we can remove target-colorspace-hint option, but people will not like it probably.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to start creating color surfaces at all to manage color in this VO, then imo we should just completely bypass mesa and just do it all ourselves (granted, I dunno how much work this is on the libplacebo side). Having a weird mix-match where sometimes it's created, sometimes it's not, maybe gets destroyed, etc. sounds confusing and error prone.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So does this set the color_surface before mesa gets a chance to? I'm a little confused on how this doesn't blow up.

It's not a race. Only one color surface can be created per wl_surface. Attempting to create more than one hangs the wayland descriptor, as we know. Handling that gracefully is probably doable.

The trick here relies on the the assumption that drivers won't create an idle color surface "just for the heck of it"™ sitting around (for passthrough) in the future. For now, that's the case for openg/vk+passthrough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trick here relies on the the assumption

It's not an assumption or a hope. It's specified that there won't be a color surface created for PASSTHROUGH.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could only find an open issue on github on this. But it's more than enough for me. It's great we don't have to make this assumption!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the discussion in this PR KhronosGroup/Vulkan-Docs#2410

Comment on lines +4203 to +4207
if (!primaries || !transfer) {
wl->target_params.color = pl_color_space_srgb;
// but gamma2.2...
wl->target_params.color.transfer = PL_COLOR_TRC_GAMMA22;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we save the colorspace from the hint somewhere in the wayland vo state and let set_color_management actually do the logic here?

Copy link
Member Author

@kasper93 kasper93 Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to know what parameters needs to be used before rendering. If params are supported all the rest is handled by existing code. The hint is already saved.

There are way more corner cases, for example what happens when some of the parameters are not supported, but frankly I don't think it's that important, as with Vulkan we didn't have this info anyway.

@mahkoh
Copy link
Contributor

mahkoh commented Jan 29, 2026

There are a few separate issues:

1) For each image description, there are many other image descriptions that will be treated identically by the compositor. Given two image descriptions,

  • A: primary-min[1], primary-ref[1], primary-max[1], target-min[1], target-max[1]
  • B: primary-min[2], primary-ref[2], primary-max[2], target-min[2], target-max[2]

If for each

$$X \in \lbrace\textrm{primary-min}, \textrm{primary-ref}, \textrm{primary-max}, \textrm{target-min}, \textrm{target-max}\rbrace$$

the equation

$$(X[1] - \textrm{primary-min}[1])\frac{\textrm{primary-ref}[2] - \textrm{primary-min}[2]}{\textrm{primary-ref}[1] - \textrm{primary-min}[1]} = (X[2] - \textrm{primary-min}[2])$$

holds, then the compositor will render surfaces with image descriptions A and B identically. (Assuming all other parameters (primaries etc.) are identical.)


2) Therefore it seems reasonable that the client should also treat any two such image descriptions identically.


3) However,

$$\textrm{contrast} = \frac{\textrm{target-max}}{\textrm{target-min}}$$

is not uniquely defined in the equivalence classes created by 1 above. An example is

  • A: 0.2, 80, 80, 0.2, 80
  • B: 0.1, 100, 100, 0.1, 100

Where contrast = 400 for A and 1000 for B.


4) Therefore, if mpv needs to calculate the contrast quantity, it needs to consistently choose one of the elements from the equivalence class before calculating the contrast.

The code currently in master does this by transforming to a color space C with the following parameters:

  • C: PL_COLOR_HDR_BLACK, PL_COLOR_SDR_WHITE, irrelevant, irrelevant, irrelevant

The issue that was observed with this is that, if primary-min=target-min and primary-ref=target-max, then we get

$$\textrm{contrast} = \frac{\textrm{target-max}}{\textrm{target-min}} = \frac{\textrm{203.0}}{10^{-6}} = 203000000$$

This is different from the previous mpv behavior which in practice always uses contrast=1000 in this case.


5) I don't believe choosing the element from the equivalence class that is sent by the compositor is a good solution for this. These elements are random both in theory (according to 1) and in practice:

  • KDE sends target-max=200, target-min=0.01
  • Other compositors send target-max=80, target-min=0.2

Using the calculation from this PR, this leads to contrast=20_000 on KDE and contrast=400 on other compositors.

Neither of these seem in any way accurate. I've looked at the top 3 recommended monitors on amazon and they advertise contrasts of 1000, 2000, and 4000.

On KDE, this might lead to dark colors appearing too dark, but I have not tested this. On other compositors, mpv has a fallback where it increases contrast to at least 1000, so this would probably be ok, at least it would be the same as mpv without color management support.


6) In another issue it was said that contrast=1000 is good for LCDs and contrast=inf is good for OLEDs. But this information is currently not available via any wayland protocol.


7) Therefore I would instead suggest the following:

  • If target-max=primary-ref, hard code a contrast of 1000. This likely indicates that content is being watched on an SDR monitor. This restores the previous behavior for mpv users and most users are going to be on LCD monitors. The current default value of 1000 was presumably chosen because it produces good results on such systems. This is the 99% case.
  • Keep the current behavior if target-max!=primary-ref.

Users can always override the behavior on the command line.

diff --git a/video/out/wayland_common.c b/video/out/wayland_common.c
index 7acc0b0ef..0367e1c8a 100644
--- a/video/out/wayland_common.c
+++ b/video/out/wayland_common.c
@@ -2164,6 +2164,7 @@ static void info_done(void *data, struct wp_image_description_info_v1 *image_des
         // we need to round it.
         if (fabsf(wd->csp.hdr.max_luma - PL_COLOR_SDR_WHITE) < 1e-2f) {
             wd->csp.hdr.max_luma = PL_COLOR_SDR_WHITE;
+            wd->csp.hdr.min_luma = PL_COLOR_SDR_WHITE / PL_COLOR_SDR_CONTRAST;
             if (wd->csp.hdr.max_cll != 0)
                 wd->csp.hdr.max_cll = MPMIN(wd->csp.hdr.max_cll, wd->csp.hdr.max_luma);
             if (wd->csp.hdr.max_fall != 0)

@kasper93
Copy link
Member Author

kasper93 commented Jan 30, 2026

Thank you for the response, let me answer to the points. (EDIT: Also sorry if I'm being hard to discuss with, I genuinely don't mean bad, it's just how I approach things sometimes)


1) I think we already had this math in the other thread. The issue is still the same, it remaps display referred like transfer to another one disregarding the minimum luminance. It as any display referred space will map 0 to 0 relative black.

Consider that one space has min-luminance of 100, the 2nd one has min-lumiance of 1. All other parameters are the same, we don't want to map the first 100 to 1 luminance. Because it will be significantly darker.

I think all this stems from implementation details from current compositor implementations, and I think we already established this part, but let me recap some things, just so we are on the same page.

cd/m² aka. nit is is the unit of luminance. It's an SI unit. Display referred transfers like naturally doesn't have luminance range, they are 0-1, just as the display can reproduce. Now we enter HDR/PQ where the encoding is absolute, in cd/m² and theoretical goal is to reproduce those values. The scale is unambiguous as defined in ST 2084. The trick part is where display referred transfer should land in PQ. That's where reference white value is useful.

Now in Wayland all transfers are neither display referred, nor absolute scaled. (joking here :)) They both have luminances and references. And in each case cd/m² means a different thing. If that was an intention of protocol, why even use cd/m²? Why not just speak in relative normalized float?


2) In the end it likely will happen, I'm being stubborn here, because using cd/m² and saying

Compositors should make sure that all content is anchored, meaning that an input signal level of 'reference_lum' on one image description and another input signal level of 'reference_lum' on another image description should produce the same output level, even though the 'reference_lum' on both image representations can be different.

is bit confusing. Well, maybe the intention was to compositor scale brightness, but still cd/m² means what it means. It's not for client to worry if they set luminances.

Either way, the main question which one of those cd/m² is proper scale? Don't think about libplacebo or mpv. Which one of those values is compatible with HDR metadata, which is encoded into movies as PQ and sent to display as metadata?


3, 4) You can think about this way. Let's consider simple example

PREFERRED (P): 0.2, 80, 80, 0.2, 80

A: 0.5, 80, 80, 0.5, 80
What is black point of A when blended into P? The same reference.

0.5, correct? Or if you apply conversion, it would map to 0.2. This like I said is not correct, because you are ignoring luminance value of A colorspace.

Now, of course, you have to ask what is P? Think about it, as if it is our virtual display. How exactly parameters are mapped to the real display is different story, because compositor may apply ICC profile, any other target adjustments, it's irrelevant from the point of view of client. But we can go there, if needed.

P is an abstract space that we target, that's all. In perfect world it could describe display, but it's perfectly valid to any blend space really. It's what compositor requests as they would like to get.

Complication comes from the fact that display referred spaces, doesn't have min luminance or they do, but it's display. So Generally you set 0, and that's all. With pipelines with BPC, it's different. But I digress... it's again compositor issue, not clients.

Back to our example, if P reproduce 0.2-80 range and A reproduce 0.5-80 range, which is explicitly stated by the descriptor. A should remain at 0.5. Note that the reference is the same, so you can't say that reeeelative.

Now, more interesting example, HDR metadata space, B:

We established, that Wayland really wants to make reference white blended at the same level. While the values are still cd/m², they suddenly become different scale to A and P, which ok, we can work with.

Where my current assumption that Wayland's compatible scale with HDR metadata and PQ signal is:

$$ \begin{align} \textrm{min-b} &= 0 \\ \textrm{max-b} &= 10000 \\ \textrm{ref-b} &= 203 \\ \textrm{target-b-min} &= \textrm{hdr-meta-min} \\ \textrm{target-b-max} &= \textrm{hdr-meta-max} \end{align} $$

I've split it into steps, so we can better follow, the intermediate normalized value.

$$ \begin{align} \textrm{lum-norm} &= \textrm{lum-a} / (\textrm{ref-a} - \textrm{min-a}) \\ \textrm{lum-b} &= \textrm{lum-norm} \cdot (\textrm{ref-b} - \textrm{min-b}) \\ \textrm{lum-b} &= \min(\max(\textrm{lum-b}, \textrm{target-b-min}), \textrm{target-b-max}) \end{align} $$

Which works out to:

$$ \textrm{lum-b} = \min\left(\max\left(\textrm{lum-a} \cdot \frac{\textrm{ref-b} - \textrm{min-b}}{\textrm{ref-a} - \textrm{min-a}}, \textrm{target-b-min}\right), \textrm{target-b-max}\right) $$

Now, according to 1) we know that compositors will map relative 0 to relative 0 of another space, so having targeting the above space and assuming black at $\textrm{lum-b}$ it will commute to 0.2 in the requested preferred space.

The math is working here, if however we assume that compositors are broken and give us descriptors that are not usable, than we can ignore those descriptors. Though, it shouldn't affect the above math to calculate things. We can level higher sanitize metadata, and say, that it makes little sense.

Also user can override that at will.


5) Compositors implementations doesn't have final word.

That's the point I'm trying to make, those descriptors should be meaningful. Not random or irrelevant. It's like that because min/max luminance doesn't mean much for "SDR" transfers, so it kinda works with any values. But this shouldn't prevent clients from using those values for processing, else protocol shouldn't provide them.

Now only KDE does send more relevant info, both in max and min luminance. When altering display brightness.

I know it's not currently the case. But it could be a descriptor that really describes the output parameters.

Initially I though preferred descriptor would be an amalgamation of user setting and hdr calibration options.

Either way, that's not the case. I could use EDID info or any other, hardcode some values, but the point here is that we want to avoid any compositor adaptations, we do most of the things in-house. According to 1) if equivalence is not provided compositor will do adjustment. So, that's why changing to 1000:1 contrast as you suggest makes it not equivalent and would trigger compositor to do adaptations.

Also, mpv is Wayland client, not compositor client. We shouldn't adapt to compositor implementations, only to make it work. This solidifies this behavior and will not fix anything ever in the future.


7)

This is the 99% case.

Well I doubt that. Especially with HDR displays. For SDR none of this is relevant anyway, we could just disable color management and would result in perfectly valid output. And in HDR it has been for years complaints that users have to manually adjust those parameters.

All this color management is for people who needs it. And justifying shortcuts, by the fact "99%" users don't need it, is no good.

However if that's the case, we can disable color management by default and make it opt-in as it were before.

@mahkoh
Copy link
Contributor

mahkoh commented Jan 31, 2026

Thank you for the response, let me answer to the points. (EDIT: Also sorry if I'm being hard to discuss with, I genuinely don't mean bad, it's just how I approach things sometimes)

I do not mind it.


1)

I think all this stems from implementation details from current compositor
implementations

At least on the compositors I've tested, 0.0 in each buffer is treated as a
fixed point. That is, if a client attaches an SDR image and the compositor
presents the image on an HDR output, then absolute black in the source image
will be mapped to absolute black on the output.

It would be possible to do it differently, e.g. the compositor could uplift SDR
black to be somewhat lighter, so that absolute black on the output could only
be reached by clients attaching HDR images, but no compositor does this at the
moment, I think.

Now in Wayland all transfers are neither display referred, nor absolute
scaled. (joking here :)) They both have luminances and references. And in
each case cd/m² means a different thing. If that was an intention of
protocol, why even use cd/m²? Why not just speak in relative normalized
float?

Maybe the intention was different, but I don't see a way to treat it as
anything else but completely relative with how the protocol has been
implemented in practice.

The scaling is in any case required because, in the protocol, SDR transfer
functions defined a reference luminance of 80 whereas PQ defines a reference
luminance of 203. Without scaling, applications using PQ would appear 2.5 times
as bright as applications using SDR transfer functions.


2)

Either way, the main question which one of those cd/m² is proper scale? Don't
think about libplacebo or mpv. Which one of those values is compatible with
HDR metadata, which is encoded into movies as PQ and sent to display as
metadata?

I think ITU-R BT.2408-7 ("Guidance for operational practices in HDR television
production") is the guideline here.

https://www.itu.int/dms_pub/itu-r/opb/rep/R-REP-BT.2408-7-2023-PDF-E.pdf

It says that

The reference level, HDR Reference White, is defined in this Report as the
nominal signal level obtained from an HDR camera and a 100% reflectance white
card resulting in a nominal luminance of 203 cd/m2 on a PQ display or on an
HLG display that has a nominal peak luminance capability of 1 000 cd/m2. That
is the signal level that would result from a 100% Lambertian reflector placed
at the centre of interest within a scene under controlled lighting, commonly
referred to as diffuse white1. There may be brighter whites captured by the
camera that are not at the centre of interest, and may therefore be brighter
than the HDR Reference White.

Graphics White is defined within the scope of this Report as the equivalent in
the graphics domain of a 100% reflectance white card: the signal level of a
flat, white element without any specular highlights within a graphic element.
It therefore has the same signal level as HDR Reference White, and graphics
should be inserted based on this level.

Nominal luminance, cd/m2 (for a PQ reference display, or a 1 000 cd/m2 HLG display)
Reference Level: HDR Reference White (100%) (1) also diffuse white and Graphics White 203

CTA-861.3-A - "HDR Static Metadata Extensions" says that

These data structures allow signaling of SMPTE ST 2084 HDR EOTF [2] and SMPTE
ST 2086 Mastering Display Metadata [3]

SMPTE ST 2084 in turn says that the minimum luminance is 0.

Therefore I would assume that the scale that should be used when using
CTA-861.3-A would be 0 to 203.


3, 4)

3, 4) You can think about this way. Let's consider simple example

PREFERRED (P): 0.2, 80, 80, 0.2, 80

A: 0.5, 80, 80, 0.5, 80 What is black point of A when blended into P? The
same reference.

0.5, correct? Or if you apply conversion, it would map to 0.2.

All compositor that I know of treat 0.0 in the buffer as a fixed point.
Therefore 0.5 maps to 0.2 and vice versa.

Back to our example, if P reproduce 0.2-80 range and A reproduce 0.5-80
range, which is explicitly stated by the descriptor. A should remain at 0.5.

Maybe it should, but it doesn't.

We established, that Wayland really wants to make reference white blended at
the same level.

As per the recommendation from ITU-R BT.2408-7.

Where my current assumption that Wayland's compatible scale with HDR metadata
and PQ signal is:

min-b = 0 max-b = 10000 ref-b = 203 target-b-min = hdr-meta-min target-b-max
= hdr-meta-max

I believe so. Although the protocol says that min=0.005 for PQ. I'm not sure
where that comes from since SMPTE ST 2084 says that min=0.

I've split it into steps, so we can better follow, the intermediate
normalized value.

lum-norm = lum-a / ( ref-a − min-a ) lum-b = lum-norm ⋅ ( ref-b − min-b )

I don't believe that is correct. It should be

lum-norm = (lum-a - min-a) / ( ref-a − min-a )
lum-b = lum-norm ⋅ ( ref-b − min-b ) + min-b

Since otherwise ref-a does not map to ref-b.

lum-b = min ( max ( lum-b , target-b-min ) , target-b-max )

Maybe. You could use more complex tone mapping, I suppose.

Which works out to:

lum-b = min ( max ( lum-a ⋅ ref-b − min-b ref-a − min-a , target-b-min ) ,
target-b-max )

Now, according to 1) we know that compositors will map relative 0 to relative
0 of another space, so having targeting the above space and assuming black at
lum-b it will commute to 0.2 in the requested preferred space.

The math is working here, if however we assume that compositors are broken
and give us descriptors that are not usable, than we can ignore those
descriptors.

I'm not sure what you mean by this.


5)

  1. Compositors implementations doesn't have final word.

That's the point I'm trying to make, those descriptors should be meaningful.
Not random or irrelevant. It's like that because min/max luminance doesn't
mean much for "SDR" transfers, so it kinda works with any values. But this
shouldn't prevent clients from using those values for processing, else
protocol shouldn't provide them.

Now only KDE does send more relevant info, both in max and min luminance.
When altering display brightness.

I'll have you know that I, too, send a changed reference level when the user
reduces or increases the brightness via compositor settings.

I know it's not currently the case. But it could be a descriptor that really
describes the output parameters.

This information is usually not available. The EDID of my secondary monitor
contains no brightness information. The EDID of my primary monitor contains
only the maximum brightness but not the minimum brightness.

The values that KDE sends are not related at all to my monitors.

Initially I though preferred descriptor would be an amalgamation of user
setting and hdr calibration options.

Is it not?

Either way, that's not the case. I could use EDID info or any other, hardcode
some values, but the point here is that we want to avoid any compositor
adaptations, we do most of the things in-house. According to 1) if
equivalence is not provided compositor will do adjustment. So, that's why
changing to 1000:1 contrast as you suggest makes it not equivalent and would
trigger compositor to do adaptations.

Adaptations here means tone mapping if the target-min-lum of the surface is less
than the target-min-lum of the output (after scaling).

However, mpv already increases the contrast to at least 1000. Therefore,
reducing the constrast to 1000 in SDR mode, as I've suggested, can never cause
more tone mapping (since it causes target-min-lum of the surface to become
larger, if anything).

Also, mpv is Wayland client, not compositor client. We shouldn't adapt to
compositor implementations, only to make it work. This solidifies this
behavior and will not fix anything ever in the future.

mpv should not be used as a means to exert pressure. It should be used to
deliver maximum value to its users.


7)

This is the 99% case.

Well I doubt that.

You don't think that SDR displays with primary-ref=target-max are the 99% case?

However if that's the case, we can disable color management by default and
make it opt-in as it were before.

Maybe disabling color management if primary-ref=target-max is the way to go.

@kasper93
Copy link
Member Author

1)

The scaling is in any case required because, in the protocol, SDR transfer
functions defined a reference luminance of 80 whereas PQ defines a reference
luminance of 203. Without scaling, applications using PQ would appear 2.5 times
as bright as applications using SDR transfer functions.

Whether it's relative or not, is workable. I think we currently have a bit of mismatch in particular in black point and how the luminances are handled in compositor, what I think is bit incomplete.

As you said:

At least on the compositors I've tested, 0.0 in each buffer is treated as a
fixed point. That is, if a client attaches an SDR image and the compositor
presents the image on an HDR output, then absolute black in the source image
will be mapped to absolute black on the output.

Let say we have some luminances like so:

min: 0.2
ref: 80
max: 160

Exact values doesn't matter, whether it's relative or not. We can normalize this with regards to reference white val / ref:

min: 0.0025
ref: 1.0
max: 2.0

We have around ~2x HDR headroom, but at the same time this should mean that <0.2 is pure black. Think about this like that we have HDR headroom, but also SDR range.

If we ignore minimum, we should be consistent and don't set it as luminances.


All compositor that I know of treat 0.0 in the buffer as a fixed point.
Therefore 0.5 maps to 0.2 and vice versa.

Yes, that we established by now. Which in practice means that minimum value can be anything and it doesn't have significant influence on the image.

I believe so. Although the protocol says that min=0.005 for PQ. I'm not sure
where that comes from since SMPTE ST 2084 says that min=0.

Yes. I'm confused at this too. 0.005 is common value in HDR movies metadata as close to black, but not sure why it's not just 0.

Since otherwise ref-a does not map to ref-b.

Ok, actually I fell into trap of your previous code. It should be just lum-norm = val / ref and that's all. We have reference white at 1.0. The rest is scaled accordingly.

We don't want 0.2 to become 0.005, we only want to rescale it.

Previously we also discussed that transfers are not "HDR" or "SDR", there is however a difference between traditional gamma transfer or PQ transfer. Of course you can reproduce any luminance range with either, but the significant difference is that one is display referred, while the other is not.

Converting between those luminances as if they are all black scaled (i.e. display referred) is not fully correct. Particularly when converting to PQ transfer, we just shouldn't throw away the "nits" information of the traditional gamma transfer input.

Maybe. You could use more complex tone mapping, I suppose.

This is for "target" parameters, which are fixed, more fancy tonemapping will map source to those values.

I'm not sure what you mean by this.

If compositor always treat black as display referred. If we set min luminance to higher value, we would map it correctly, either way, this is not that pragmatic, because compositors give dummy values for traditional transfers.

mpv should not be used as a means to exert pressure. It should be used to
deliver maximum value to its users.

I don't mean to exert pressure, but who else can do reality check for compositor implementation except client application?


Let's talk solutions. If we follow your advice to adjust code to compositors implementation rather than the specification.

PQ transfer in Wayland world is offset by min luminance value, effectively making the transfer instead of 0 to 10000 nits represent 0+b to 10000+b nits, where b is min luminance value.

So, let say I want to put "black" at the display level, say 0.2 nits. We could do

A) set_lumianances(0.2, 10000.2, 203) and output 0 PQ code point for black.
This requires new PL_COLOR_TRC_WAYLAND_PQ that is offset removed, same as traditional gamma transfers. I don't see how would that be useful addition for us.

B) set_lumianances(0, 10000, 203), set_mastering_luminance(0.2, ...) and output 0.2 PQ code point for black.
This will do what we expect at least for KWin implementation with BPC enabled. Granted with compositor specific strategy how to handle black point, but at least this should make mostly expected output with current impl.

This is also nominal case how any HDR video with metadata should be handled. So, while I have doubts the current compositor impl will map it correctly to different PQ space, it's out of mpv's hands.

The main problem here, is that with Wayland we are not targeting the (real) display, but image descriptor that compositor request from us, which is kinda virtual display if you want to think about it this way.

So, when compositor asks us to render 80 nits with 0.2 nits black level, we can respect that, scale it up to our reference and go from there. But apparently we have conflict of interest here, because those values are not "valid" for us to use, so we are advice to ignore them.

There is also scaling issue, because 0.2 nits is actually perfectly valid black point target, but apparently in Wayland nits are not nits, so it's unknown what to do. We could just use this value as-is without scaling or ignore it completely. This unfortunately, limits ability of compositor to control our output.

C) Output gamma2.2, also for HDR. This would require to use 16hf backbuffer.
This has the already discussed issue that the reported min luminance by compositors is not usable, and advice is to ignore it. This seems problematic long term, because it may change depending on implementation changes.

D) Output linear, this is similar to C), just without gamma part.

E) Fix compositors to adhere to SMPTE ST 2084 as Wayland specification mandates for PQ transfer.


B) seems most reasonable from mpv side, except E) of course. The only issue is that mapping PQ to PQ output will still do an incorrect mapping, but this is on compositor side and we really cannot workaround that here.


For black clipping there are multiple patterns, I use Spears & Munsil, but I cannot share it here. This one is good too https://github.com/user-attachments/assets/8d105cd3-c9fe-40b7-b3f6-589079f7a090 the two bars on the right side should be visible.

You don't think that SDR displays with primary-ref=target-max are the 99% case?

I don't care about the %. Like I said we are optimizing for 1% (with that metric), the 99% case is already working fine and doesn't need any color management.

@mahkoh
Copy link
Contributor

mahkoh commented Feb 11, 2026

I don't care about the %. Like I said we are optimizing for 1% (with that metric), the 99% case is already working fine and doesn't need any color management.

I thought it wasn't but looking at it again it seems that SDR -> SDR playback is indeed unaffected by this issue.

@kasper93
Copy link
Member Author

I will wait for #17395 and update this one after, we do some prerequisite changes there.

@kasper93 kasper93 marked this pull request as draft February 11, 2026 21:10
@kasper93 kasper93 changed the title vo_gpu_next: add VOCTRL_COLOR_SPACE_HINT vo_gpu_next: set color directly using Wayland protocol Feb 28, 2026
@kasper93 kasper93 requested a review from Dudemanguy February 28, 2026 09:51
@kasper93
Copy link
Member Author

kasper93 commented Feb 28, 2026

Rebased on top of small color refactoring.

Luminances situation is still pending fixes.

Using Wayland protocol is a regression to current Vulkan based solution, color surface is not synchronized with the frames, so for example changes to preferred descriptor (ex. by changing brightness) produce visible mismatch from our rendering output and what's used by Wayland. I believe this is not directly related to this PR, but the wayland_common.c.

@kasper93 kasper93 marked this pull request as ready for review February 28, 2026 09:52
@llyyr
Copy link
Contributor

llyyr commented Feb 28, 2026

We probably need equivalent of 468d34c for vo_gpu_next now

@kasper93 kasper93 force-pushed the wayland-color branch 2 times, most recently from 2778678 to 304f0ae Compare March 2, 2026 08:34
@kasper93
Copy link
Member Author

kasper93 commented Mar 2, 2026

I consider this ready for review.

@llyyr, @Dudemanguy: I will need help with vo_wait_on_vo(), because I'm not sure I understand what should wait for what and where.

@mahkoh: Now we have full control of luminances and other parameters, we can bikesheed this further.

I thought it wasn't but looking at it again it seems that SDR -> SDR playback is indeed unaffected by this issue.

That's the point of display referred transfers, luminances doesn't really mean much for them, unless you convert to PQ (absolute one). I know Wayland treats everything relative, but now I set the descriptor for PQ, same as you would get Blu-ray movie metadata, which I belive Wayland should handle correctly. (if not, it's outside the scope of mpv)

@Dudemanguy
Copy link
Member

I will need help with vo_wait_on_vo(), because I'm not sure I understand what should wait for what and where.

You will probably need to mimic vo_dmabuf_wayland assuming that waiting for the image description from the compositor also needs to happen in this case. When vo_gpu_next starts a frame, you should set vo_wait_on_vo if the wayland code sets image_description_pending. If that happens, then in flip_page you should not submit the frame yet. After the wayland code sets the image description and flips the bool, then you can unset that and allow the frames to process again.

@kasper93 kasper93 requested a review from Lompik March 3, 2026 01:43
@llyyr
Copy link
Contributor

llyyr commented Mar 3, 2026

Besides the one minor nit, this looks good to me. Thanks

@kasper93 kasper93 force-pushed the wayland-color branch 2 times, most recently from 732ef8b to 88054d4 Compare March 3, 2026 02:05
#if HAVE_WAYLAND_PROTOCOLS_1_41
if (!params) {
if (wl->color_surface) {
wp_color_management_surface_v1_destroy(wl->color_surface);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have this destroy behavior? I guess that would mean hacks like VK_hdr_layer won't work but I can't see that mattering?

Copy link
Member Author

@kasper93 kasper93 Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is when we set --target-colorspace-hint=no we no longer hint anything. Which in practice means, Vulkan sets VK_COLOR_SPACE_SRGB_NONLINEAR_KHR, which means that Mesa take over color_surface duty. So we release our color surface, to let others use it. Conversely when we do --target-colorspace-hint=yes we switch to VK_COLOR_SPACE_PASS_THROUGH_EXT and use color surface ourselves.

I guess that would mean hacks like VK_hdr_layer won't work but I can't see that mattering?

Mesa supports Wayland nativelly now, so not only VK_hdr_layer is affected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I forgot about the --target-colorspace-hint=no case. I guess it is fine as long as that is the only time this ever receives NULL params.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in all other cases, hint is defined to "something". Also we might at some point get rid of this option completely, but people still complain that "color is wrong", so restoring to dumb player mode is good to debug or just workaround other bugs.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have this destroy behavior? I guess that would mean hacks like VK_hdr_layer won't work but I can't see that mattering?

Well you will remove hdr support for us nvidia users, since amd have so many issues for me at the moment, I will be forced back to windows if this happens.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have this destroy behavior? I guess that would mean hacks like VK_hdr_layer won't work but I can't see that mattering?

Well you will remove hdr support for us nvidia users, since amd have so many issues for me at the moment, I will be forced back to windows if this happens.

this doesn't remove HDR support for nvidia. this PR makes it so that you dont need that layer for HDR support on nvidia

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have this destroy behavior? I guess that would mean hacks like VK_hdr_layer won't work but I can't see that mattering?

Well you will remove hdr support for us nvidia users, since amd have so many issues for me at the moment, I will be forced back to windows if this happens.

this doesn't remove HDR support for nvidia. this PR makes it so that you dont need that layer for HDR support on nvidia

Ah okay thanks, kind of traumatised by a certain other open source project's disregard for nvidia.

@Lompik
Copy link

Lompik commented Mar 3, 2026

when switching from srgb to pass_through, mpv now picks another surface format(rgba16), is it possible to keep a2bgr10 ?

...
[vo/gpu-next/libplacebo]     85: VK_FORMAT_A1R5G5B5_UNORM_PACK16          VK_COLOR_SPACE_HDR10_ST2084_EXT
[vo/gpu-next/libplacebo]     86: VK_FORMAT_A2R10G10B10_UNORM_PACK32       VK_COLOR_SPACE_HDR10_ST2084_EXT
[vo/gpu-next/libplacebo]     87: VK_FORMAT_A2B10G10R10_UNORM_PACK32       VK_COLOR_SPACE_HDR10_ST2084_EXT
[vo/gpu-next/libplacebo]     88: VK_FORMAT_R16G16B16A16_UNORM             VK_COLOR_SPACE_HDR10_ST2084_EXT
[vo/gpu-next/libplacebo]     89: VK_FORMAT_R16G16B16A16_SFLOAT            VK_COLOR_SPACE_HDR10_ST2084_EXT
[vo/gpu-next/libplacebo] Picked surface configuration 12: VK_FORMAT_A2B10G10R10_UNORM_PACK32 + VK_COLOR_SPACE_SRGB_NONLINEAR_KHR
...
[vo/gpu-next/libplacebo] Picked surface configuration 28: VK_FORMAT_R16G16B16A16_UNORM + VK_COLOR_SPACE_PASS_THROUGH_EXT

@Lompik
Copy link

Lompik commented Mar 3, 2026

I'm not sure if this is intended but there is now a discrepancy in the transfer function between gpu-next/opengl and gpu-next/vulkan (for sdr content).

  • --target-colorspace-hint=yes --hwdec=vulkan --gpu-api=vulkan
[vo/gpu-next/wayland] Generating image creator params:
[vo/gpu-next/wayland] primaries: bt.709, transfer: gamma2.2
[vo/gpu-next/wayland] Setting color representation:
[vo/gpu-next/wayland]   Coefficients: rgb, Range: full
  • --target-colorspace-hint=yes --hwdec=vaapi --gpu-api=opengl
[vo/gpu-next/wayland] Generating image creator params:
[vo/gpu-next/wayland] primaries: bt.709, transfer: bt.1886
[vo/gpu-next/wayland] Setting color representation:
[vo/gpu-next/wayland]   Coefficients: rgb, Range: full

for sdr content, the stat (i) display/transfer is always gamma2.2 for vulkan regardless of the target-colorspace-hint and bt1886 for opengl.

@kasper93
Copy link
Member Author

kasper93 commented Mar 3, 2026

when switching from srgb to pass_through, mpv now picks another surface format(rgba16), is it possible to keep a2bgr10 ?

I decided for passthrough to prefer 16-bit backbuffers, to allow most flexibility. We could prefer a2bgr10, this should handle fine most things, except linear and things like extender range gamma.

I'm not sure if this is intended but there is now a discrepancy in the transfer function between gpu-next/opengl and gpu-next/vulkan (for sdr content).

Yeah, opengl doesn't use preferred colorspace, so by default it fallbacks to not adjusting gamma or srgb. Basically opengl works the same as master in regards to color output. This can be extended, but I didn't mix too much things here.

@kasper93
Copy link
Member Author

kasper93 commented Mar 3, 2026

Yeah, opengl doesn't use preferred colorspace, so by default it fallbacks to not adjusting gamma or srgb. Basically opengl works the same as master in regards to color output. This can be extended, but I didn't mix too much things here.

Fixed this now. But still opengl will use 8-bit backbuffer, so not recommended for anything other than sdr really.

@kasper93 kasper93 force-pushed the wayland-color branch 3 times, most recently from 8bd1e0b to cdb4c6e Compare March 3, 2026 08:05
@kasper93
Copy link
Member Author

kasper93 commented Mar 3, 2026

Will merge later today, if there won't be more comments. I want to stress test this and we can fix issues as we go. Luminances situation is still unknown, we may just hardcode in the end if needed.

@Lompik
Copy link

Lompik commented Mar 3, 2026

opengl will use 8-bit backbuffer

in my case, fbo-format is rgba16hf and the mesa egl surface format is XR30. where is this 8-bit backbuffer ?

Edit: I guess it depends on what the bit depth the compositor uses for the screen. AFAIK opengl is not limited to 8bit.

kasper93 and others added 9 commits March 3, 2026 11:16
This is required, because other modules like Vulkan may need to use it,
and only single color surface is allowed in Wayland.
Wayland color managment is not really flexible to be usable with Vulkan
color spaces, so set it directly.
This fixes --target-trc=linear output and makes our output luminance
ranges more explicit.
Co-authored-by: Kacper Michajłow <kasper93@gmail.com>
Those has Dolby Vision metadata stripped and have all parameters
correctly guessed.
Wayland likes to kill the window or application on protocol
violation.
Comment on lines +4275 to +4278
if (wl->color_queue) {
wl_event_queue_destroy(wl->color_queue);
wl->color_queue = NULL;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (wl->color_queue) {
wl_event_queue_destroy(wl->color_queue);
wl->color_queue = NULL;
}

I think it's fine to keep the queue alive.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, will do.

Comment on lines +4282 to +4283
if (!wl->color_queue)
wl->color_queue = wl_display_create_queue_with_name(wl->display, "image description creator queue");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!wl->color_queue)
wl->color_queue = wl_display_create_queue_with_name(wl->display, "image description creator queue");

I'd move this to the place where the queue is used for the first time.

#if HAVE_WAYLAND_PROTOCOLS_1_41
if (!params) {
if (wl->color_surface) {
wp_color_management_surface_v1_destroy(wl->color_surface);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe this is sound as is.

Consider the following code:

static void image_description_failed(void *data, struct wp_image_description_v1 *image_description,
                                     uint32_t cause, const char *msg)
{
    struct vo_wayland_state *wl = data;
    MP_VERBOSE(wl, "Image description failed: %d, %s\n", cause, msg);
    wp_color_management_surface_v1_unset_image_description(wl->color_surface);
    wp_image_description_v1_destroy(image_description);
}

// ...

    struct wp_image_description_v1 *image_description = wp_image_description_creator_params_v1_create(image_creator_params);
    wl_proxy_set_queue((struct wl_proxy *)image_description, wl->color_queue);
    wp_image_description_v1_add_listener(image_description, &image_description_listener, wl);

If the color_surface is destroyed while the image description is pending, the callback will access a NULL pointer.

Furthermore, it is probably a good idea to store the image_description inside wl and to destroy it at the appropriate times, since otherwise the image descriptions might complete out of order.

Copy link
Member Author

@kasper93 kasper93 Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's very good observation. However, my expectation is that @llyyr in commit 8012dc8 made it so that image_description_listener is read or failed by the time set_color_management function returns. And we will never call set_color_management() concurrently.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe that is necessarily the case. See the discussion in #17495.

Comment on lines +3564 to +3565
wp_image_description_creator_params_v1_set_luminances(image_creator_params,
hdr.min_luma * WAYLAND_MIN_LUM_FACTOR, hdr.max_luma, PL_COLOR_SDR_WHITE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused about this (and similar with the linear case above). Shouldn't max=ref in this case?

Copy link
Member Author

@kasper93 kasper93 Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's configured correctly, it's will be max=ref. However, it can be configured differently, setting higher max works fine.

m_opt_choice_str(pl_csp_levels_names, params->repr.levels));
wp_color_representation_surface_v1_set_coefficients_and_range(wl->color_representation_surface, coefficients, range);
} else {
MP_WARN(wl, "Color representation %s / %s not supported! Output can be incorrect.\n",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mahkoh: Why Jay doesn't advertise any coefficients as supported? I would expect any implementation that supports this protocol at least advertise the baseline, which it in fact supports.

I'm not sure if that's intended, because it triggers this warning. I can exclude identity, because this in practice is always supported, but that seems weird.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why Jay doesn't advertise any coefficients as supported?

Jay does not support any YCbCr formats.

I would expect any implementation that supports this protocol at least advertise the baseline, which it in fact supports.

I've implemented the protocol only for the alpha modes.

Copy link
Member Author

@kasper93 kasper93 Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. I think I will keep this warning, it may be weird thing to log when the output is fine. But this is the information we get. RGB/identity is not advertised as supported.

Protocol states.

When this object is created, it shall immediately send this event once for each matrix coefficient and color range combination the compositor supports.

So, I assume that implementation should send RGB/identity as supported, if it is in fact supported.

EDIT: Note to self, in case I forget, change it to Output may be incorrect., sounds better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it doesn't support them. If you call set_coefficients_and_range at all, you will get a protocol error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it doesn't support them.

I kindly disagree that Jay doesn't support identity. It displays fine.

If you call set_coefficients_and_range at all, you will get a protocol error.

That's why I don't call it to avoid protocol error and display warning.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I don't think we need to scare people with a warning here. Verbose is probably fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should warn, it would save us dozens of "wrong colors" issues that we get for dmabuf wayland. Note that it will warn only when color repr proto is actually supported, so it's quite valid thing to say that output will be wrong, if certain params are not supported.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly my guess is that people would open an issue about the warning they see instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As they should, not for mpv probably, but their compositor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I don't think we need to scare people with a warning here. Verbose is probably fine.

The warning would only be visible with dmabuf-wayland in most cases, since full range rgb should be supported on all most compositors. In which case, I do think it being a warning is fair.

Comment on lines +3641 to +3642
wl->color_representation_surface =
wp_color_representation_manager_v1_get_surface(wl->color_representation_manager, wl->callback_surface);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe this is correct. There could be theoretical compositor that doesn't support the coefficient/range combination we want to set but does support the alpha/chromaloc. In such a case, we'll try to set them without a color repr surface

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have it fixed locally, there was unrelated issue that caused problems. I will push later when I have time to work on this again. I got side-tracked again with luminances, because I'm still thinking if we should prefer PQ output and just output in gamma2.2 as compositors like to have it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants