The Disaster Of Wayland Input Methods
Vložit
- čas přidán 17. 02. 2024
- If you write in non latin script you absolutely require an input method, but they're also very important for handwriting recognition and virtual keyboards but on Wayland there are some flaws designed into the protocol and this is the person who do so.
==========Support The Channel==========
► Patreon: brodierobertson.xyz/patreon
► Paypal: brodierobertson.xyz/paypal
► Liberapay: brodierobertson.xyz/liberapay
► Amazon USA: brodierobertson.xyz/amazonusa
==========Resources==========
Input Broken: dorotac.eu/posts/input_broken/
Fosdem Talk: fosdem.org/2024/schedule/even...
2020 Input Methods: dorotac.eu/posts/input_method/
Old Protocol KDE: blog.martin-graesslin.com/blo...
Old Protocol Virtual Keyboard: bugs.launchpad.net/onboard/+b...
=========Video Platforms==========
🎥 Odysee: brodierobertson.xyz/odysee
🎥 Podcast: techovertea.xyz/youtube
🎮 Gaming: brodierobertson.xyz/gaming
==========Social Media==========
🎤 Discord: brodierobertson.xyz/discord
🐦 Twitter: brodierobertson.xyz/twitter
🌐 Mastodon: brodierobertson.xyz/mastodon
🖥️ GitHub: brodierobertson.xyz/github
==========Credits==========
🎨 Channel Art:
Profile Picture:
/ supercozman_draws
#Wayland #OpenSource #Linux #FOSS
🎵 Ending music
Track: Debris & Jonth - Game Time [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Debris & Jonth - Game ...
Free Download / Stream: ncs.io/GameTime
DISCLOSURE: Wherever possible I use referral links, which means if you click one of the links in this video or description and make a purchase I may receive a small commission or other compensation. - Věda a technologie
The other use case that's actually been driving me crazy as a first time linux user is emoji keyboards. All the ones that I've tried just copy the emojis to the clipboard, which is annoying for several reason, including messing up my clipboard history
Developers implementing different versions protocols is another problem. For example, Chromium implements text-input-v1, Qt implements text-input-v2, and others implement text-input-v3. In addition, although wlroots implements text-input-v3 & input-method-v2, Fcitx5 is the only viable input method which implements input-method-v2.
If only you knew the pain of using Linux, or most of the other Unixoids, as a non-English user with weird squiggly text in your language. Things have gotten a bit better but the default in a lot of cases seems to be "If you are not using US/English/US Keyboard, because we don't care" or "Fix it yourself".
We don’t care. Fix it yourself
@@toxiccan175worst take of year
I have found the compose key to be a boon in those cases :D
it works the same as it does on windows for me, i use åäöëéèáàóòãẽõ. Unless you mean non-latin characters
Text rendering is hard in general so a lot of projects choose to support only the bare minimum. You have all the technical font stuff, all the font variants including non monospace ones, all the unicode characters, ligatures (they're a must for some languages), RTL rendering etc. I only use English and Polish so I don't encounter any problems. The only problem I have is that I like to use bitmap fonts and they're so small on a hi-rez display and basically there are no apps that support integer font scaling, such a simple feature yet as with all of this font stuff it has to be implemented on a per app basis because there are no standards.
Not just IME for non-Latin based writing systems, there also is the pervasive use of keyboard-like widgets for emoji and other glyphs. This partially working under X11 and is present on most mobile operating systems
Windows comes with a built-in emoji IME probably added to force devs to test IME support
It's also present on MacOSX and some utilities do it (Luke Smith basically implemented an emoji picker like this in the name of minimalism, instead of using Kitty's)
4:16 - if keyboards and mice should be in Wayland, I'd say joysticks should be too. They're actually used outside of games more than you might think. I've seen CAD software with joystick support (apparently it's a good way to interact with models), and many devices for people with disabilities will use joystick inputs to allow them to interact with their computers in ways that make sense for them. I'm pretty sure VR devices use joystick inputs for their controllers, regardless of whether in a game or not. The inputs normally associated with joysticks are useful anywhere where you need a bunch of axes and some buttons.
Joystick is just another method of analog input. I don't understand why not just let protocol expose what event is happened directly to the app and let user check what to do. Maybe I always want read keys from keyboard as latin keys, but wayland protocol developers decided that "fu, here you have a UTF-8 text that does not make any sence for you, goodbye". Just let inputs be read by the app and let the server control which app gets which inputs.
Yep, broadly as a class they're HIDs (Human Interface Devices), some joysticks had torsion mechanisms even 20+ years ago and other fine control options that yes could be useful for controlling a 3D environment potentially in CAD. It's not something I've personally worked with but I can definitely as an old flight and mech sim fan appreciate why they'd prefer it over a mouse.
@@rogo7330Let it do both. Apps that want text input (normal keyboard use) should be able to register for text input events and apps that use keyboards as button arrays (like games) should be able to register for button press events.
1:09 - Even physical keyboards do not always have a physical button that accounts for all keys. That's why deadkeys exist, and they're not always simple to get to work either. Especially if you need to have both that and a language like Japanese set up. Lord knows how many issues I've had in the past getting both "Ø" and Japanese input working in FFXIV. Deadkeys like ¨ + o for ö for that one time i might need to write German is also a bit of a pain in some applications.
Amen. I just had to buy a new laptop today (as a temporary stop gap) but as yet have not found how to get Ctrl-PgUp/PgDn to work.
As you posit - why is this so hard?
Dude. For one-off-needed characters just use the compose key.
@@smorrowWhat on earth is a compose key?
@@Davixxa I'm not the other guy, but the Compose key is a dedicated key that you press once (and not hold down), and then press a sequence of characters. The input method checks a table of known 'Compose key sequences', and if the following keys you type are partial matches it doesn't type anything, but as soon as you complete one of the matches it'll insert that single character for that sequence. If what you type doesn't match anything, it just starts letting you use the keyboard normally again.
I personally remap CapsLock to Compose, so that I can do things like 'Compose,
@@TynachAh okay that makes sense. And same, I just don't really see the need to make a compose sequence for something my keyboard layout should have by default
7:36
"backwards N with a squiggle on it"
this is a yot (й) from the cyrillic alphabet, and is pronounced /j/, which is the IPA transliteration of 'y' as a consonant - as in the first phoneme of the word "you" (/ju/).
(thus, in cyrillic, one would transliterate "you" as "йу", since 'й' acts as 'y' and the cyrillic upsilon 'у' (not the latin y called " why") can be assimilated to the latin 'u'.)
Except one would translitirate you as ю
you would translate yo-yo as йо-йо. Not you as йy
@@bleack8701 yo-yo would be ё-ё
@@ukyoize this was for illustrations purpose ; although you are right in that i could have picked a better example that didn't involve phoneme compression.
I know that some people on Windows use multiple keyboards where each has a different keymap. Not even to type different character sets, but for hotkeys.
Yes, they somehow managed to create so many hotkeys, that they needed multiple keyboards (including modifiers like Shift, Alt, Ctrl etc.) to actually have all of them.
And then there are people who have a keyboard whose entire purpose is to type emojis.
Having a Japanese keyboard would make my life easier, but I potter along with the one US English layout. That many hotkeys is insanity as someone that has dozens setup for OBS and tools so I don't need to click through.
Setting up korean input is a PAIN, especially on WMs
I've been mentioning IMEs as a big blocker for my switch to wayland for a long time on many "just switch to wayland now" type of videos. It's nice to see someone with more reach bringing this issue some spotlight. On X11 my IME config works flawlessly and reliably. On hyprland and sway i've recently been finally able to generally get fcitx5 working but it does have some issues. Ibus on wayland just didn't work outside of gnome no matter the config and environmental variables
9:47 Funny you mention this, but in the FOSDEM talk you linked, dcz admitted to not looking into how AOSP handled input during the Q&A portion.
In his defense, how would that help if Android isn't already running Wayland? Isn't android still using x11?
@@MrGamelover23 I don't believe Android uses either X11 or Wayland. I believe it would still help as a high level idea of how Android handles input though.
Android uses its own display server called Surface Flinger, not X11 or Wayland. @@MrGamelover23
@@MrGamelover23it uses SurfaceFlinger
@@mmkthecoolest Android has native textbox widgets. Wayland and X11 don't.
Let's get this straight..... he put a backwards procedure into the protocol for a problem that exists on interconnected web documentation & collaboration? You mean the problem that gets solved by literal checkout/lockout functionality of shared document sharing, where it tells you who is & is not interacting with what text/document/place/etc at any given time. X.x
You're telling me that he's designing a single user system to be lag resistant, if not complicate the matter of where a problem is, if for some reason a program is slow to respond or you type to fast.
....
Seriously, Wayland had a complete mental breakdown for what amounts to window Icons.. but good god, having a layer where it handles input devices separate and sends the character (key+map) to the application is to fucking much? like .... what?!
Thank you for continuing to cover things that are a source of friction for a typical business user using Linux!
Please keep doing what you are doing.
This is a severely complicated topic, but at least it *does* work, i primarily use fcitx5 or mozc. As for the development process..... Input from non keyboard devices is necessary, and not just for accessibility, and emulation should have been the first step for a reference implementation, fallback, and a place to merge input, similar to Mesa.
This just underlines that getting one person to bear the weight of critical tooling is a serious issue and we need to take advantage of resources more often.
I thought this was the type of problem Wayland was written to solve in the first place... that X11 was too old to have a proper framework for IMEs, and Wayland would be better for Unicode/mobile and handle this kind of use case out of the box? If this is true, it looks like Wayland is having teething problems even for its intended use cases that it was sold to everyone on, not just for the old X11 desktop use cases. What's weird to me is that Wayland didn't follow the same rules they apply to desktop stuff usually and say that this should be determined by each compositor, leaving it to GNOME and KDE to implement their own IMEs and having this not be part of Wayland. I mean, presumably since X11 also didn't support IMEs, everyone already had their own... so why try to add one into the protocol now? It seems like ironically, the Wayland devs may have wound up in the same situation they were in with the X11 compositor, but with IMEs... they're trying to build one IME into the protocol that isn't very good to fall back on except in limited use cases, but in practice none of the major DEs use it, and the few things that do depend on it are designed around its bugs and it might be hard to fix it without breaking a lot of stuff at this point. At this point I am wondering if we might wind up with GNOME and KDE becoming more and more their own stack and Wayland being sidelined into just a common dependency of those desktops, sort of like Cairo was for a long time. I mean, if they have their own compositors, their own toolkits, and their own IME... then what do they really need Wayland for, other than just to abstract away some kernel-level interfaces and talk to the graphics drivers? At this point so much functionality has been offloaded to libraries external to X11 and Wayland that it almost doesn't matter which "backend" you use to talk to the underlying graphics hardware. It's very possible that the tools built over the years to abstract away the differences between X11 and Wayland during the transition may eventually result in a world where we don't need either one... would be ironic if that were the case.
Wayland is just X11 + second system effect. It doesn't make anything better. It seemed better at the start, because it ignored a bunch of real-world complexities. Now that it has to handle these complexities, it's lost its only advantage and it turns out the redesign from scratch is actually worse than the design with the historical accumulated knowledge behind it.
Wayland isn't really a part of the GNOME or KDE stack, it's just a common protocol for non-GNOME and non-KDE applications to talk to the GNOME and KDE desktop servers.
@@thewhitefalcon8539 I think that's a really good perspective on this. I always thought Wayland was terrible, but I heard all these other people saying it was great. Now that I've looked closer, I'm seeing that the truth is basically that modern Wayland is becoming somewhat usable finally, after 15 years of development, but it doesn't seem substantially better than X11, just different. And if I'm being honest, it seems like the main reason Wayland is starting to be better is simply because X11 hasn't seen major development in 15 years... and something actively developed will eventually be better than something in maintenance mode. I can accept that some people like Wayland and find it suits their needs, but I just don't buy that it is objectively better... it seems like it's just different and comes at things from a fresh perspective, sort of like how Object Oriented was supposed to be better than Procedural programming... but it turns out it really depends on what you're doing and OO isn't a magic bullet that fixes all the issues with Procedural... I think Wayland vs. Xorg is the same thing.
Błąd is easy, try this one 😀 Zażółć gęślą jaźń - it's using all letters from Polish alphabet
Now I understand why i did fail to write a Polish Trzynastozgłoskowiec on wayland session.
all the modified letters, that is
Back when the Russian invasion of Ukraine began, some Russian soldiers came across weapons with strange writing. They were speculating that NATO devised a way to write Ukrainian with Latin letters. It was hilarious when someone pointed out that they were looking at Polish.
7:38 That's a short capital I in Cyrillic the Й, the term for the diacritic mark specifically is a breve.
I specifically use an international English IME for my primary input, because I often have to use diacritic marks if I'm writing German or French in particular, secondary is Japanese which can be a fun time on a regular 103/4 US English layout assuming I even have it in the right mode for the character set I want.
Even the opening statement is incorrect: multiple Latin-alphabet languages, sometimes even including English, use keyboard layouts with dead keys.
0:38 OH that's actually a deep subject. Linguistic complications are what held back Japanese software industry, and is why they focused primarily on video games. Check out "fifth generation computing project" which was an attempt to overcome what was really a fundamentally hard problem.
Hi Brodie! Thank you for putting this on the table! I always have this kind of problem and always have no idea who to blame (I mean, not blaming anyone but always wish to find out where the bug is). I type into the input method and the lag is really strong that I cannot see what my target characters are anymore. In the end what I have to do is to try until it appears the correct combination. Very difficult.
FOSDEM is really fun to attend. i hope to see brodie there someday :)
I'd love to but getting there is a little expensive for me
@@BrodieRobertson Hopefully someday there could be virtual FOSDEM participation?
the issues always start with "on the wayland side"
I was there at fosdom. There was a talk about this topic.
I use mainly English in my day to day life. Then I have to type Spanish for my family, and I have to type Chinese for my partner's. Only GNOME properly supports it and it's still garbage. And nobody is doing anything to fix it cause it's not the GNOME desktop problem, or it's "unsafe" for Wayland, etc.. Sometimes Linux developers make me miss Windows, I swear
It would be that easy if my fingers could keep in line, the damn things don't want to go in order
7:39 it's called short i. it's the same sound as y in the word boy
Strange... In Serbian the sound the y makes in boy is the one the letter J makes lol
@@aqua-bery that's basically the Russian equivalent of j
@@aqua-bery well yeah, that's because the J is supposed to be that sound basically everywhere:) it's only a handful of incompetent languages such as english and spanish don't know how to use it correctly lol
@@user-xe6sm4jv8f As a British person I am not going to let you let French get out of this
@@user-xe6sm4jv8f And then there's that Sj thing in Swedish
So where exactly does the Libei stuff from Peter Hutterer come into it? I had heard that was working somewhat
Here's a fun one, Bulgarian has BDS, Bulgarian (Typewriter) and Bulgarian Phonetic. But devs that don't know this pick one at random
That means that depending onm which you chose the \ key will either do ( and ) when shifting; or „ and “ when shifting
or when you press a you might have ь and Ь when shifting; or ь and ѝ when shifting
or it might just be phonetic, which means some letters go to keys like `[]\' etc.
ibus and mozc used to work in kde, but they recently did something that broke it. I guess I'll wait for the next kde release and try to figure out what they want to make it work.
Can vouch for the bit about software keyboards. Between my PC and phone, using Mozc on my phone
is the only way to directly write in Japanese that I have, might not be my first language. But hey,
more respectful than cheaping out and using Romaji. Speaking of Mozc, maybe I should try it on PC too?
Good to hear it's being looked at. Non-Latin, or just plain not English languages deserve the same respect
We've already had to retrofit a US invention to convey our language plenty before IMEs were expected
The lack of joysticks in X11 is a big reason you often need your game launcher to suppress the screensaver for you... because the game doesn't do it and X11 is blind to the input being sent via the joystick... plus, Flatpak is having trouble with joystick hotplug when applications need to go direct because, for that to work, the host and container D-Bus libraries would need to be split along an API boundary that the D-Bus devs have declared internal and unstable.
Even for latin script, there is some issues : the French layout for example as some swapped keys A and Z are where Q and W are on an English keyboard, Same for M and "," and most special characters are misplaced.
Add a remote KVM into the mix, and chaos begin.
As a consequence, my old job had a requirement for the emergency root password : no a, z, q, w or m, because when you have to log as root on the virtual console you don't want to lose 15 minutes to type a password you cannot see.
That's a keymap issue from my understanding not an input method issue but I could be wrong
@@BrodieRobertsonit's a consequence of mixing up the button board model of a keyboard with the text input model of a keyboard
The backwards n with a squiggle is a short i, part of the Belorussian, Bulgarian, Russian, Ukrainian and Kazakh alphabet. The ordinary backwards n without squiggle is an ordinary i. Slavic languages using the Latin alphabet tend to use j for it, and the Latin j also got imported into some Cyrillic alphabets for that purpose. And as I'm already on wikipedia: There's eight different possible 'i's in Cyrillic, each language using a subset, and that isn't even counting stuff like Ы (which is technically a ligature) and syllable-letters like yu, ya, ye and yo. It's not even really a mess though Latin is worse (especially with English re-assigning all the vowels to other vowels), the only real downside is the lack of proper small letters, Cyrillic isn't easy to read. Italic and hand script is better or worse for some characters, depends.but overall it has the readability of Latin text written in small caps, that is, it's atrocious. There's a reason Romans only used what's now capital letters for stone engravings and something more sensible for writing.
As to the actual topic: I guess this is actually a perfect use-case for a protocol for privileged applications, so that compositors can hand off handling keyboard mapping to a program which can do watever: Let it have the raw keycodes, allow it to display stuff on-screen, take from it a stream of *logical* keypresses, rip all layout handling out of applications that's usually the wrong place to do it unless we're talking games which don't tend to want mapping but react to physical keys, mapping only comes into play when it comes to answering "what's WASD labelled like on the current keyboard so I can display the right symbols?".
When you use a gamepad and press one of the analog sticks to the max into one direction and then let it go, it will wiggle a lot ( in high speed) until it calms down. Many games fail to filter the raw data from the stick properly and detect the first strongest "wiggle" when the stick returns for the first time as a move into the opposite direction. A lazy solution is to implement a far too large dead zone around the center of the stick. I once listened to a podcast about how the clone of a popular game wasn't really playable as the programmers didn't understand all the input magic of the original, e.g. moving a character up an alley although the user slightly missed the time window for pressing the up key. Input is hard ;)
2:57 You seem quite confused. These three protocols are complementary to each other; a compositor that outsources input method handling over Wayland needs all three of them, but those that don't (or do it via other means) only need text-input-unstable-v3, as that's the only one that's facing towards regular client applications.
You run into issues when you have a mismatch of supported protocols. I'm not sure if you even disagree with me
Yeah, and no protocol will work very well if you unplug the power from your PC, but that's hardly insightful. Whether a compositor needs to implement the latter two protocols depends on how it does its input method handling. From the application PoV it's just an implementation detail, as ideally these protocols shouldn't be exposed to non-privileged apps at all even if the compositor supports them.
Text input landscape in Wayland is indeed messy because of protocol mismatches, but in a completely different place than where you pointed to (look at text-input-unstable-v2 if you want to know more).
Considering Google does kinda have a stake in making Wayland better (ChromeOS) it might be worth sending them a message and asking for advice/how they handle it on Android at a high level (yes you could look at AOSP code, but that is most likely less of a high level overview to understand and maybe even improve uppon).
On Android, text editing and the keyboard are both controlled by the OS, and they only have one use case to think about so it's a lot easier. Android IMEs practically have access to the whole text field, even its ID and label and placeholder text. It can do pretty much what it wants with your text: make selections, delete, paste, replace everything, delete last word. There's even an API to send arbitrary data to and from the keyboard so the application can talk with the keyboard directly and bypass everything. It even knows the location of the cursor in global screen coordinates.
It looks fairly decent, supports things like voice input and handwriting and gestures in handwriting. But they also had the advantage of that being the _only_ API available to interact with the IME: applications have to support it, and everyone just extend the built-in EditText class which they can because everything is Java. That's a major difference with how input has been handled on computers with keyboard and mouse for decades, and that's where it gets really messy. How do you tell XWayland "delete last word"? How does that work in say, a terminal where all you have is sending backspace to the program? Keyboard shortcuts? WASD in games?
This one's already a disaster on X11, so it's gonna be a long road, and it's gonna break everything, and it'll make people scream at how complicated the protocol will be just to implement "Ctrl+C" in your app. But it's necessary, and it goes beyond just input but also accessibility. Some people are blind. Some people can't use their hands and need to exclusively talk to a machine. Some people can't talk and can only communicate with symbols like emojis. On Android, it all just works out of the box. But on Linux it's going to be a long fight against a certain crowd that only cares about a keyboard that's roughly shaped the same as a US english keyboard, and display in some form of latin alphabet.
Й is on the same key as Q on russian keyboards. it represents the Y sound like in in You. the letter is somehow related to И that represents EE sound like in chEEse, but now no one would say that й and и sounds have something in common.
I make heavy use of the compose key and I've got quite a bit of stuff in my custom xcompose file that I'm concerned about Wayland breaking that.
This goes along with what I've been saying a lot lately about developers not learning from history. Oh well. I guess we all just love wasted efforts.
I'm studying Korean and I can't even have a Wayland virtual keyboard to compose ㅎㅏㄴ => 한.
my fantasy overlaying a small vim editor over any input field is shattered now...
Doesn't SailfishOS use an on-screen keyboard on Wayland since 2013 with no issues?
I need a Keyboard in Shmargorian, and it has to be made for 4 hands with 6 fingers each, and we have 123 letters, 89 pronunciation marks, and 45 punctuation marks. Oh, and what about dance move input?
If wayland input methods are broken, then how virtual keyboard works in Plasma?
Sure the hardware interface is needlessly different, but semantically, a joystick is just an excited mouse iiuc
I've used joysticks as mice when a cord broke before. Great hammer swing.
If flight sticks should be handled by specific applications then so should game-pads, and arguably so should mice and keyboards. They are all HID after all.
Why dont the people from kde use the mallit keyboard as the emoji picker? The kde emoji picker only copies the emoji to the clipboard, and the mallit keyboard inputs it directly
I wonder if they can implement input methods with pipewire instead of using Wayland
From my years of using the Swype keyboard on android, I'm pretty sure that it keeps an internal buffer/state of recently typed keys, and this buffer is what matters for auto-correction. So yes, it would overwrite the state on the application if it came to that. Which works fine for me. It does lead to some funny desync situations where you are typing into the void, but it's rare.
Please, look at existing solutions, for heaven's sake!
the problem is really annoying. i have the dvorak keymap. in baldurs gate 3, i have to remap my settings to use ,aoe instead of wasd. but minecraft for example uses the keys, not the letters they represent, so when i press , it understands that its the same key as w and allows me to go forward. BUT in settings... i see that W is assigned to moving forward, and if i try to reassign to , it doesnt change as , is w. minecraft knows that my , is w on qwerty. but I dont know where the keys on qwerty layout, and have to guess where is the key, i cant just watch in settings what to press, i have to experiment.
this problem is not BG3 and minecraft specific. some games use BG3 method, other games - minecraft's. so when i enter a new game, i dont know will my , move me forward or do some other stuff
Minecraft solution is better, tbh. Yeah, it's non-trivial that you must reference keycodes of QWERTY keyboards, but that just right thing to do. Many games though solve this by showing a letter that would be printed with your current layout, still saving keys as keycodes. DOOM 2016 does that, other games too.
To be fair that also depends on the app
okey, you said android got it all figured out, isn't android open source and linux? can some code or ideas or philosophy be borrowed on input methods or it's an issue with licence? could help with behaviour being intuitive and some cross compatibility
The Android way of doing this is not easily transferable to Wayland. It has abstracted the input away from the apps in a way that is really difficult to replicate on a regular OS.
Also, since handling multiple inputs was one of the challenges, Android is not good at this. If you have ever used a physical keyboard with an Android device, you know what I mean.
@@enemixiushow Windows does this? Why not just follow them?
Even in latin script getting installers to keep the correct language is often very unreliable which sucks considering special characters are in completely different places.
plasma + fcitx5 + mozc + env variables, I have no clue how this magic works but I'm grateful it has been made this easy
I'm thinking, why does nobody look in the Android source code and see how keyboard inputs are done there, now you dont have to start from scratch and have a general idea how it is done....
Probably because android isn't using Wayland.
@@MrGamelover23 I know that Android isnt using wayland, what I'm saying is at least they have a general idea how it is implemented and maybe they can use that idea to do the same to wayland...
@@ChimeraX0401 It is pretty silly that they didn't look at AOSP, which had already been doing this for decades.
Well, I just thought about what kinds of (physical) input devices actually exists:
- digital working keyboards (a key is either pressed or not)
- analog working keyboards (some mechanical keyboards report how much a key is pressed; and yes, some games take advantage of that to handle walking/running speed)
- mice
- touch pads
- touch screens
- control sticks (1- and 2-dimensional)
- in general analog and digital buttons
- touch pens (some even support how close the pen is to the surface, so yay 3d)
- VR-controllers (so, basically a mouse but with 3 dimensions instead of 2)
Of all these multiple can exist.
The problem of emulating a keyboard is still something that Wayland needs to cover. There's lots of touchscreens which are not a mobile phone but for example Kiosk applications or industrial touchscreens in some production line running Linux. Don't always expect Linux runs only in the cloud, servers or on someone's personal PC/laptop with a physical keyboard.
You say at the start that for people that use latin script can be pretty simple to type on, but it can be a bit annoying if the language that you use has accented latin characters. Like when I first moved to Linux deadkey'+c would return (ć) in some distros and (ç) in others for the us international kb layout. Of course still much more simpler to fix than using a IME. XD
In my opinion, this needs to work from the firmware up, not from the windowing system. You should be able to use this in the bootloader to enter an encrypted boot image using this system, so before kernel, on the kernel command line, and on the normal text mode terminal as well as in the graphical session. So it's a whole stack spanning problem. Add to it that in today's prevalency of multidisplay systems, there's no reason for a user to not expect to be able to have multiple input devices for multiple programs spread over the various devices, each with a separate focus for interaction, as well as for multiple device spanning programs, or in situations where you'd want to spawn the input mode editor on a separate device from the display etc.
I guess this is why Wayland has so much debate before implementing protocols and not version them as improvements
I use Wayland on the desktop and mostly write English but also Irish/a little bit of Scottish Gaelic - these languages are mostly latinised now but do feature unique special characters. The current best solution I have is just to use the Windows Extended UK layout clone, but sometimes I want a real Tironian et instead of an Ampersand!
But what is this I see... Compose mode? Surely this will solve all my problems. NOPE.
Also deadkeys... I don't want to talk about dead keys.
Joysticks should absolutely be a part of Wayland's purview if other common HIDs like m/kb are - it's essential for Accessibility and also just utility; my joystick/throttle/rudder are all standard HID devices and can control CAD software like OpenSCAD just as well as they can game!
If wayland input is the solution, can we have X11 xorg back?
What?
linux is wild. text input is broken, sound is broken, video is broken, yet we all use it and people don't try to fix things properly, just patch over the mistakes without getting rid of the broken stuff
Sadness :(. Why can't I get a window-maker API that isn't insane without paying a huge fee for a wrapper?
Could this apply to controller inputs as well? Cuz damn it's a hit or miss on Wayland lol
@@09f9 which is funny since it breaks wayland "security" for virtual keyboard input (such as steam deck password input, etc)
Show incoming second input in orange to both users.
4:30 Joysticks would probably be important for wayland if you made a UI that works entirely with a game controller.
even German and Spanish have input methods because they use characters that are not part of the English Latin script, like Spanish uses "ñ" & "Ñ" and accents like áéíóú,ü, German uses the beta like letter that is really an F
but yeah, the most problematic would be Japanese because it uses 3 character sets, 2 syllabaries and kanji but their input is usually in Latin script, what they call romaji, some Japanese computers use layers for the syllabaries and thus type with them but others do inpunt Latin script and then the input method has to translate that to a list of suggestions and then let the user choose.
My bad
It's pretty ironic that JP input is actually much more usable for me on Linux (Wayland) than on Windows because it seems "non-QWERTY user on an English system, who would occasionally like to type in Japanese" is just not a normal use-case for the guys at Microsoft...
Й is kind of like an i mixed with a j or a y... lmao. И is I in Cyrillic, Й is basically a modified version of I, similar to how J is basically an I with a funky sound to it.
Valve? Can you hire someone to work on this ?
I hate on-screen keyboards. They would be less bad, I think, if some sort of multi-touch could be implemented instead of having to constantly switch between layouts to get caps, numbers, punctuation, symbols etc.. Entering things like URLs, email addresses etc. is a PITA.
Haha Pretty sure I have come across issues related to this in my dwl and ydotool hacking. It is working at the moment. Sounds like it will break again one day.
I kinda wish a group of decent devs would get together and fork wayland or ditch it and build something better and backwards compatible
I speak several languages none of which need an input method. But I feel like I'm lagging behind the times because people write emojis these days everywhere and you basically need an input method for that.
1. This 'reversed N with squggles' is a capital 'i' in Russian language
2. Translation of word "Błąd" is 'Error
3. Using wayland without such protocols already in place on mobile space = somebody rushed it too much ;)
Yep... trying to use nix, not nixos... install, nix-shell... and... it has problems with my input.
Why wouldn't Wayland want to handle input output managment? That seems like something you'd want your protocol to handle itself. You do have pipewire for audio (at least) and likely video. Seems like you need an HIDIMEwire protocol that handles both virtual and real software and hardware input and acts as a central manager tying together hardware and software with input methods and mappings for low-latency response. Maybe it even gets added to pipewire so you can capture things like syncronized User Input AND video AND audio and direct that to multiple destinations? If the latency is low enough, it just becomes the device you attach to your game/app or pipe into wine or whatever. Seems kind of silly if you ask me. Also having that kind of control and visibility would tell you if something was trying to interject itself (keylogger or something), since you should be able to see where you I/O is going. Make the framework and then let people have at with targets and methods that interest them.
Man... The day wayland doesn't show up in my news or recommendations because of some "drama" will be a glessed day.
I know, it's a pipe dream. XD
But can the wayland people just pull that metaphorical stick out of their youknowhere and just ... do stuff? o.o
I need to figure out why dead keys don't work on my work fedora Plasma on KDE software but works on Discord flatpak somehow.
Have you checked your kde input settings? There's typically multiple keyboard layouts available. (E.g. for german there's de de(no deadkeys) de(some obscure layout)
Discord flatpak runs in XWayland, it's not a native Wayland client
It's time to finally admit the simple fact that developers do not solve problems, they just throw a new fresh manure on top of a huge pile which was thrown before them by others.
Is someone else gonna work on it? Cuz if that hasn't been updated in the last 4 years...oof...
i use arabic on wayland on kubuntu very good
This is one the problem with oss. Important stuff like this dont get worked on because nobody wants to.
like german:
ßö yöü ßäy
yeß
"ø" is not German.
@@thingsiplay you sure? xd
@@fuseteam very
Special german chars are öüäß (source: i mean im german)
You do realise that leaving off the last two words on the video title would have been enough of a description - and saved you keyboard wear.
Just checked - xorg still running fine on all my Gentoo machines and still being update for bug fixes... ho hum.
0:58 Excuse me, who is that on the virtual keyboard!?
Linus Torvalds, dude that started and leads Linux kernel development
@@raw_000 I thought it was, but I didn't know it 100%! Thanks for letting me know!
And btw, that's a hilarious background to the virtual keyboard!
@@raw_000 I don't know if this is a joke and I'm kind of afraid to ask
@@cameronbosch1213it's not i found out, it's actually cropped from a famous photo of Gabe Newell, the two just kinda look like each other (or at least did when that photo was taken)
Oh we're so cooked
you said Garnacho correctly.
Joysticks see the responsibility of sound cards
Couldn't you write a key logger with a IME? I shouldn't give them concerns though or they will remove it or make it worse
I use an IME all the time, and English is my only human language. I don't have a … key. I don't have a § key. Nothing on my keyboard is made to type ∴ or µ or 😀 or ©. I can't afford a keyboard that will support ⠃⠗⠇ patterns. I didn't need a GUI to type any of those things. I did need to switch from English to type the English code contraction "brl" for Braille. My caps lock key does a lot of heavy lifting for me serving as a compose key, a caps lock (combined with shift), and a Fn key for certain other keys.
I don't think I can do most of that on Wayland yet.
I know their argument is that *they* ruined it, but really Wayland didn't cause any issues here. In fact, Wayland's strength is that people can create new protocols for their own use-cases, and these can be integrated upstream so others can start using it ASAP. The politics and stuff are obviously downsides, but there would simply be no solution at all (albeit a limited and "broken" one) if Wayland wasn't designed to be an open and modifiable protocol. It lends itself to the Linux FOSS mindset of "fork, fix, upstream" and I'm pretty confident that with more publicity, this will be improved all too soon!
I didn't realize we had a amish community in Australia
Proper Input is incredibly hard, and i *know* that.
That's why i don't think it's something i want to touch again.
This is even worse if you are using flatpak or other container-for-each-app garbage.
йцукен gang mentioned
3:30
0:14 Only that German has the 3 Umlaute Ä, Ö and Ü as well letter that's not quite a real letter ß, though it's not really a problem since those are in UTF-8 (though it does leave out the capitalized version of ß, ẞ, since it wasn't added to unicode until 2008)
btw I just hate that Y and Z are swapped on the QWERTZ layout, makes the keyboard shortcut Ctrl+Z a pain in the ass. Also why can't you set the language to English but set your keyboard layout to German in most Linux installers?
I think at least the Debian installer should let you do that. You can pick display language, locale and keyboard layout independent of each other, though locales and keyboard layouts that don't match your display language are hidden behind the "others" submenu at the bottom of the list.
What do you mean? UTF-8 can encode all valid code points and your comment that contains ẞ clearly arrived just fine encoded using UTF-8.
want to bet that someone will fix it, only for wayland team to deny it 20 times for stupid reasons, approve the worst solution, and then gnome refuse to implement it?
This video about Wayland input methods is very interesting and informative. May the algorithm bless this video with many views.
Here to help engagement!
This feels like a comment written by chat GPT
Blud got the ChatGPT comment
@@yorimirus I was hoping for that effect
@@esra_erimez oh ok lmao