Auto-language switch inside math mode

[arielf212]

Hello, thank you for this amazing tool - it certainly beats setting up LyX :P

One feature I do miss is LyX's ability to auto-switch to English when inside math mode.
Would you entertain merging in a PR that adds this as an optional feature?

[IgKh]

Hi Ariel, for sure - it would be highly convenient if done right.

I'm not sure how it is possible though to do though. My understanding is that LyX filters raw key scan codes through key maps that it ships or something like that, right? I don't really want to maintain custom key maps, and deal with QWERTY vs Devorak or other such things.

So it is basically depends on your plan on how to implement. Thanks!

[arielf212]

I think that in katvan we can do something much dumber since typst is a much simpler language.
Let the user configure a "primary/text" language and a "secondary/math" language (e.g. Hebrew as primary and English as secondary).
Then when inside math mode initiate a switch to secondary, and otherwise stay in primary.
Checking if you're in math mode is trivial in typst - traverse from cursor location until you hit dollar symbol.
As for how to actually switch the language, that I'll need to think about (maybe just set the language keymap for the user? - this is certainly possible in linux- but unsure about windows)/

[IgKh]

Knowing when the cursor is inside math mode isn't a problem at all - Katvan already has a (relatively) full parser that is tightly integrated with the editor related classes - for syntax highlighting, spell checking, smart auto-indent, etc. There is plenty of features already that know when they apply to a math or code area, e.g if you notice when you have inline math the BiDi direction in it changes to LTR, and when you use the "Insert Symbol" dialog the expression that is generated if the insert point is in math is appropriate (no sym. prefix).

Actually switching the keyboard layout is indeed the hard part. Asking the operating system to do so only for the requesting application is possible on Windows, but not on Linux or macOS as far as I know. If I remember correctly (I haven't used LyX in a long time... mostly due to not being able to install it :P), LyX does that consistently by basically replacing the OS and UI Toolkit and doing key mapping itself, with all kind of weird requirements like the user needing to ensure that the keyboard layout set in the OS is something well known. I can say flat out that I am not willing to maintain something like this.

Please go ahead and research, but please share your findings and plan before starting on a patch. Thanks again!

Edit: not very related to the point, but Typst is a simple language to write but despite that (or maybe because of it?) it is an exceedingly difficult language to parse. Traversing from cursor location until a $ doesn't work - it might be escaped, or you can actually be inside a content block, that is inside a math block, that is inside a code block, חד גדיא חד גדיא.

[IgKh]

Some of my research notes on this matter, for anyone who maybe wants to tackle this (pretty desirable) feature.

There are two possible approaches I can see here:

The "Limited" Approach

If we are fine with just a single fixed layout being used inside math mode (say US English), then a possible approach is to re-write key events in Editor::keyPressEvent according to a fixed table. What we want to take from the QKeyEvent is not the key or text but either the nativeVirtualKey or the nativeScanCode depending on the platform. The reason is that we want the emit the correct English letter based on the physical key on whatever layout the user actually has on their keyboard, without requiring them to set the keyboard layout in the system to a known value as in LyX or TyX.

So for example we can map a Qt::Key_A from:

• On Linux (both Wayland and X11 via xkbcommon), nativeScanCode value KEY_A + 8 (Linux evdev code + 8).
• On Windows, nativeVirtualKey value VK_A
• On macOS, nativeVirtualKey value kVK_ANSI_A

The "Flexible" Approach

The above approach probably makes sense for math, but if there is a need for being able to choose any keyboard layout to override the active one when typing in math mode - it won't scale, and I don't want to maintain multiple such mappings.

In this case, it is possible (although much more complex) to use platform specific APIs to override the keyboard layout.

On Linux/Wayland: The keyboard layout to use is fed to the framework via a wl_keyboard::keymap event, in XKB format. We can't actually intercept it, but we can load any installed keyboard layout using libxkbcommon (which Qt depends on anyway due to the prior) and use the proper API to re-map editor key events in the same way as in the "limited" approach.

On Linux/X11: Technically can change the global keyboard layout through the XKB protocol, but it is too coarse. Can use same approach as in Wayland.

On Windows: The only platform with convenient API for changing the keyboard layout just for the calling application, in the form of ActivateKeyboardLayout. Use GetKeyboardLayoutList for discovering installed layouts.

On macOS: There is the Carbon APIs TISCreateInputSourceList and TISSelectInputSource, but they set the keyboard layout globally - which is messy.

For this approach, there needs to be some sort of manager class to abstract the platform APIs and change/restore the keyboard layout based on the cursor position in the editor and the focus entering or leaving the editor component - which is quite complex.