Shared DND Cursors are no longer shown when read from ImageDescriptor

[ptziegler]

Consider the following code snippet where sourceName is ISharedImages.IMG_OBJS_DND_LEFT_SOURCE and maskName is ISharedImages.IMG_OBJS_DND_LEFT_MASK:

private static Cursor createCursor(String sourceName, String maskName) {
	ImageDescriptor source = ISharedImages.get().getImageDescriptor(sourceName);
	ImageDescriptor mask = ISharedImages.get().getImageDescriptor(maskName);
	ImageData sourceData = source.getImageData(100);
	ImageData maskData = mask.getImageData(100);
	// Hotspot should be the center of the image. e.g. (16, 16) on 100% zoom
	int hotspotX = sourceData.width / 2;
	int hotspotY = sourceData.height / 2;
	return new Cursor(null, sourceData, maskData, hotspotX, hotspotY);
}

With 58e4313, these icons no longer show up, because the PNG icons have been swapped out with SVG icons.

Recording.webm

If I forcefully switch to the PNG icons, the cursor shows up again.

Recording2.webm

Notes

• The issue is related to the mask. If the mask is a PNG, the cursor is shown, even if the source is an SVG. So the issue is not with the SVGs in general.
• If I create the maskData via mask.createImage().getImageData(100); the cursor is shown, even when using an SVG image.
• If I use ISharedImages.get().getImage(maskName), the cursor is also shown.

So the problem seems to be some weird interaction between SVGs and the ImageDescriptor.

[HeikoKlare]

I can confirm the issue. But for me, the behavior I face is a bit different.

Out of the box (with latest state of the Eclipse and GEF repositories), the cursors appear as black boxes for me (the yellow do inside is from screen capturing):

When I replace the mask images with their PNG versions, the cursors vanish for me (like in the original post video).
When I (also) replace the sources images with their PNG versions, the cursors appear fine to me.

That perception is conflicting with:

The issue is related to the mask. If the mask is a PNG, the cursor is shown, even if the source is an SVG. So the issue is not with the SVGs in general.

So I need to figure out where our configurations are different such that we experience differing behavior.

I will try to have a look whether I find something obivous today. But I will leave for vacation tomorrow, so maybe I won't be able to proceed with this until July.

Actually, I am not even sure how much effort we should put into this mask handling. Especially with SVGs, you will probably not use masks but rather have a properly SVG itself. The mentioned images have been deprecated in favor of replacements that do not require a separate mask. So it may have not have been a good idea to replace the mask-based images (both source and mask) with SVG versions at all.
If I see correctly, in GEF you are still using the "old", mask-based versions because of compatibility with 2024-06. That definitely renders the replacement of the deprecated images with SVG versions useful, as otherwise you would only benefit from SVG-based cursors in GEF once you drop support for 2024-06 so that you can adopt the "newer" cursor image not requiring a mask that have been introduced with 2024-09.
Just out of curiosity: do you have plans to upgrade the minimum required Eclipse version for GEF at some (not so far) point in time? I am just asking because if that was the case you could then move to the cursors that do not require a mask and we could simply revert the replacement of the old cursors with the PNG versions.

[HeikoKlare]

That was easier than expected.

Analysis

The issue is rooted in the SVG image rasterization. When the rasterization result is converted into SWT image data, that image data is created with a 32bit data palette. This leads to a difference in the image data used for the cursors, as they originally contained 24 bit data (not having an alpha channel).

PNG:

SVG:

The cursor implementation takes the data arrays of source and mask such that now the alpha in there is used as well, which leads to the erroneous behavior.

That also explains these:

If I create the maskData via mask.createImage().getImageData(100); the cursor is shown, even when using an SVG image.
If I use ISharedImages.get().getImage(maskName), the cursor is also shown.

If you first create an image and then retrieve the image data from that, a handle is created and the image data is then extracted out of that image handle, which yields some data is "consistent" with what we used to have before/independent from SVG support.

Solution Options

There are a bunch of solution options that we could discuss. One more general thing we need to do (and that would implicitly resolve this issue= is to revise the Cursor class to support properly support scaling. Currently, cursors are initialized with a single ImageData and if then requested for a different zoom, that image data is auto-scaled. That leads to bad results and is also completely unnecessary, as the code instantiating a cursors usually has an image or image descriptor available which would be able to provide the image data at different zooms (with SVG support even for all zooms at high resolution).

As a more short term solution, I propose to try revising the SVG rasterizer implementation to yield image data with a 24 bit palette as in my understanding that would be sufficient. That would reduce memory consumption (as the data array of a loaded SVG would be 3/4 od the size), it may even correct the data representation (as am not sure if the 32 bit data even contains any alpha) and it would also solve this issue.
This is how it looks with the data palette at SVG rasterization be changed to 24 bit:

[ptziegler]

Wew, that was fast! Thanks for also looking into this issue.

So I need to figure out where our configurations are different such that we experience differing behavior.

Maybe just as an update, this is likely an OS-specific behavior. Based on your cursor, you're using Windows while I tested this on Linux. I just booted up my Windows laptop and now I also see the black box, so the erroneous data is probably interpreted differently, on different operating systems.

Just out of curiosity: do you have plans to upgrade the minimum required Eclipse version for GEF at some (not so far) point in time? I am just asking because if that was the case you could then move to the cursors that do not require a mask and we could simply revert the replacement of the old cursors with the PNG versions.

That's actually what I suggested to Alois. There is no rule that forces us to remain compatible with the 2024-06 indefinitely. So the moment things become more trouble than they're worth, the minimum requirement will be raised. Especially if it's a minor jump to 2024-09.

[HeikoKlare]

Maybe just as an update, this is likely an OS-specific behavior.

Oh yeah, that was what I first though as well, but your video looked so much like Windows that I dropped it. Now I also see the Linux cursor in your video :-)

I have now created:

• Remove duplicate representation of alpha values in image data from SVG eclipse.platform.swt#2254

I think that even independent from this issue that would be an improvement. I am not completely sure yet whether there is a reason for doing it like it was done (using this 32 bit data array) or whether the change is fine. In particular, I need to check the behavior on Linux because there I also found different behaviors in image data handling in past.