Wayland Cursor Position

[Combobobobo]

I'm on EndeavourOS KDE Wayland using Chrome Screen AI.
I installed using pypi, python3.13, and a virtual environment, had to pip install pygobject as well.
It launches correctly and displays no error but I can't get a pop-up to show up when shifting over Japanese text, it sees the text but I think it fails to get my cursor position so it doesn't translate it nor shows a pop-up.
I'm using two displays with only one being recorded. (it still doesn't work with a scan area including both displays)

(this is my first post and I'm new to this so if I forgot any info just ask me)

[rtr46]

thanks for reporting @Combobobobo

• did the popup ever show up for you in any configuration or never at all?
• i assume the issue has nothing to do with chrome screen ai, but affects other ocr providers as well, right?
• did you ever see logs like 10:13:53 - [INFO ] - [meikipop] - Looking up: 見ると、ほとんど...? from the screenshot of your console i can see it recognized text, but these kind of logs tell you wether or not meikipop recognized that/what text is under the mouse cursor. you may need to press and hold the hotkey while pointing at text for those logs to appear.
• please see if it works with auto_scan_mode = true and auto_scan_mode_lookups_without_hotkey = true. that removes some potential issues, helping to debug the issue
• does it work when connecting only one display? does it work when connecting only one display and selecting a region?

[Combobobobo]

Hello, thanks for the reply :)

• The popup has never appeared in any configuration.
• That's correct, I've tried all of them.
• I've never seen those logs, even when pressing the hotkey and hovering over japanese text, that's why I assume it's Wayland not knowing my pointer's coordinates.
• Same deal, it sees new paragraphs but doesn't look them up on hover.
• Disabling one display doesn't work either, I've just tried with manual, auto, and region.

[rtr46]

@KamWithK any idea?

[uunniiblog]

I've given it a try under EndeavourOS KDE wayland too. Seems to work for me but it only detects Xwayland windows (expected), and need to disable 2nd monitor for the mouse/popup issue to work correctly. With 2nd monitor enabled it does seem it's bugging out to detect windows/resolution correctly (since it shows both monitors with either 1440p or 2160p resolution even though one is 1440p and the other 2160p). Testing with this settings:

ocr_provider = meikiocr (local)
auto_scan_mode = true
auto_scan_mode_lookups_without_hotkey = true

If I enable both monitors this is what I mean

Other than that I don't see any error in the terminal with 2 monitors.

[rtr46]

thanks for sharing. i wasn't aware it only works on xwayland windows. @Combobobobo does that new info help? does it work for you with xwayland windows?

[KamWithK]

thanks for sharing. i wasn't aware it only works on xwayland windows. @Combobobobo does that new info help? does it work for you with xwayland windows?

Its a painful limitation I've tried to explain a few times when helping people on Discord, there isn't an easy way I can fully support Wayland without heaps of work and it isn't 100% guaranteed to be possible with QT

[KamWithK]

I've given it a try under EndeavourOS KDE wayland too. Seems to work for me but it only detects Xwayland windows (expected), and need to disable 2nd monitor for the mouse/popup issue to work correctly. With 2nd monitor enabled it does seem it's bugging out to detect windows/resolution correctly (since it shows both monitors with either 1440p or 2160p resolution even though one is 1440p and the other 2160p). Testing with this settings:

ocr_provider = meikiocr (local) auto_scan_mode = true auto_scan_mode_lookups_without_hotkey = true

If I enable both monitors this is what I mean

Other than that I don't see any error in the terminal with 2 monitors.

I think this might just be some odd XWayland behavior...
if you try setting both to the same resolution does it work?
I only have one monitor so I don't think I can help debug anything which is caused by having many monitors

[KamWithK]

Hello, thanks for the reply :)

* The popup has never appeared in any configuration.

* That's correct, I've tried all of them.

* I've never seen those logs, even when pressing the hotkey and hovering over japanese text, that's why I assume it's Wayland not knowing my pointer's coordinates.

* Same deal, it sees new paragraphs but doesn't look them up on hover.

* Disabling one display doesn't work either, I've just tried with manual, auto, and region.

The suggestion I usually make to people on Discord is to always start by testing with the Steam client, since it's a known xwayland application (at least for now) and it is easy to find some Japanese text in a VN store page or whatever

Try resetting your display scaling too

[Combobobobo]

thanks for sharing. i wasn't aware it only works on xwayland windows. @Combobobobo does that new info help? does it work for you with xwayland windows?

Hello, thanks for the reply :)

* The popup has never appeared in any configuration.

* That's correct, I've tried all of them.

* I've never seen those logs, even when pressing the hotkey and hovering over japanese text, that's why I assume it's Wayland not knowing my pointer's coordinates.

* Same deal, it sees new paragraphs but doesn't look them up on hover.

* Disabling one display doesn't work either, I've just tried with manual, auto, and region.

The suggestion I usually make to people on Discord is to always start by testing with the Steam client, since it's a known xwayland application (at least for now) and it is easy to find some Japanese text in a VN store page or whatever

Try resetting your display scaling too

Tried Steam with only one display enabled and can confirm it works! (I was trying on floorp earlier) Doesn't work with both of them enabled, though. I have default scaling.
They're both 1080p, one 60hz and the other (primary) 144hz.

[uunniiblog]

I think this might just be some odd XWayland behavior... if you try setting both to the same resolution does it work? I only have one monitor so I don't think I can help debug anything which is caused by having many monitors

It doesn't seem to be a resolution problem. It seems that it only works in display 0 (in my case 2nd one thats why i didn't notice) regardless of resolutions or scaling even with both monitors set up at 1440p 100% scaling it only works in display 0 and not display 1. There are however some scaling issues even in display 0 if one of them has different scaling (but not resolution).

Doing some quick testing in case it can help you debug it:

If both monitors have scaling 100% Seems to only work in screen 0 , regardless of if you set screen 0 or 1 in the tray. Even with screen 1 selected it is still working in screen 0 and not 1.

Changing display scale in screen 0 to for example 110%, restarting meikipop and game. Selecting screen 0 or 1 mouseover doesn't work again, it seems to find it sometimes moving the mouse around so probably a coordinates issue with scaling.

Disabling 1 monitor either 0 or 1. It works correctly even changing display scaling, the popup shows correctly at scaling 110% or 100% or 125%, etc with only 1 monitor enabled.

Selecting custom region (~720p pixels above game window) in screen 1 gives error:

an error occurred in meikiocr (local): OpenCV(4.13.0) /io/opencv/modules/imgproc/src/resize.cpp:4211: error: (-215:Assertion failed) inv_scale_x > 0 in function 'resize'
Traceback (most recent call last):
File "/home/uni/Documents/gits/meikipop/src/meikipop/ocr/providers/meikiocr/provider.py", line 63, in scan
ocr_results = self.ocr_client.run_ocr(
image_np_rgb,
...<2 lines>...
punct_conf_factor=0.2
)
File "/home/uni/Documents/gits/meikipop/.venv/lib/python3.14/site-packages/meikiocr/ocr.py", line 115, in run_ocr
text_boxes = self.run_detection(image, det_threshold)
File "/home/uni/Documents/gits/meikipop/.venv/lib/python3.14/site-packages/meikiocr/ocr.py", line 162, in run_detection
det_input, scale = self._preprocess_for_detection(image)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/home/uni/Documents/gits/meikipop/.venv/lib/python3.14/site-packages/meikiocr/ocr.py", line 212, in _preprocess_for_detection
resized = cv2.resize(image, (w_resized, h_resized), interpolation=cv2.INTER_LINEAR)
cv2.error: OpenCV(4.13.0) /io/opencv/modules/imgproc/src/resize.cpp:4211: error: (-215:Assertion failed) inv_scale_x > 0 in function 'resize'
13:13:24 - [INFO ] - [meikipop] - meikiocr (local) found 0 paragraphs in 0.001s.

Selecting custom region (~720p pixels above game window) in screen 0 works as expected (if both monitors have same % scaling).

If display 1 has different scaling than display 0 and try to custom region display 0 mouseover coordinates are not properly calculated and the mouseover doesn't work correctly

[rtr46]

i dont know how it works on wayland, but i think the way it works everywhere else:

• display 0 is "all screens"
• display 1, 2, 3, ... are the respective screens.

if you had 2 monitors you would expect there to be:

• display 0 (all screens)
• display 1
• display 2

here an example from windows of a 2 display setup with 1st display having 1920x1200 and 2nd having 4k:

it possibly works different under wayland, dont know.

[uunniiblog]

Ok never mind, it seems to be related to the XDG portal on first launch. I plugged in a third monitor and the XDG pop out appeared again, this time I selected the entire workspace and it does work in any screen (if scaling 100% in all of them, if not it bugs out).

and it works in any screen with regardless of screen 0 or 1, also works with custom region.

Now I don't think there is an "easy" way to make the XDG portal option to pop again? I don't see an option from tray or settings, and I guess the setting gets saved somewhere outside of the application because even deleting the config.ini or unplugging and plugging a monitor again doesn't make it pop again.

So yeah it seems with wayland and XDG portal it just ignores the screens setting in the application and whatever you select it there works (as long as scaling is same in all monitors)

[KamWithK]

Ok never mind, it seems to be related to the XDG portal on first launch. I plugged in a third monitor and the XDG pop out appeared again, this time I selected the entire workspace and it does work in any screen (if scaling 100% in all of them, if not it bugs out).

and it works in any screen with regardless of screen 0 or 1, also works with custom region.

Now I don't think there is an "easy" way to make the XDG portal option to pop again? I don't see an option from tray or settings, and I guess the setting gets saved somewhere outside of the application because even deleting the config.ini or unplugging and plugging a monitor again doesn't make it pop again.

So yeah it seems with wayland and XDG portal it just ignores the screens setting in the application and whatever you select it there works (as long as scaling is same in all monitors)

We can add an option to delete the token file, you can just manually for now delete it from ~/.cache/.ocr_screencapture_token (which should actually be documented, I assume I haven't yet though)
Note if you are using the WIP flatpak (which I've been distributing privately for testing) that this file will be found inside the flatpak so it is a bit more tricky to do manually (still possible though)

[uunniiblog]

We can add an option to delete the token file, you can just manually for now delete it from ~/.cache/.ocr_screencapture_token

From what I can test in my setup with same scaling in both monitors:

Selecting the Asus 1440p monitor works in that specific display, selecting the main 4k gigabyte display doesn't work at all. Selecting to share full workspace works in both monitors.

Changing the 4k gigabyte to 1440p to match the other one also doesn't make it work.

If one of the monitors have one scaling different from the other then nothing works regardless of I pick the Asus monitor that works or entire workspace