Fix UTF-8 input handling in interactive CLI - #11
Conversation
|
@gilsonolegario Thanks for the PR. Before merging: Also: The Windows path still uses msvcrt.getch() and isn't covered by the new integration test. It would be good to either verify that path or open a follow-up issue/PR to improve UTF-8 handling there. |
|
Thanks for the review — both points addressed. Manual verification (macOS): Drove the real CLI capture flow through a pty (the same Also confirmed paste/fast-typed input survives (whole string written in one shot, exercising the Windows path: Correct — |
Summary
Typing accented characters (e.g.
água) in the interactive CLI input stores them without the accent (gua).Root cause
_getch()reads a single byte and decodes it in isolation withdecode("utf-8", "ignore"). Multi-byte UTF-8 characters (e.g.á=0xC3 0xA1) arrive as separate bytes; each lone byte is invalid UTF-8 and is silently discarded.tty.setraw()usesTCSAFLUSH, which discards queued input on every keypress — breaking pasted or fast-typed multi-byte text.Fix
_getch(_utf8_sequence_len+_decode_key) before decoding.tty.setraw(fd, termios.TCSANOW)so pending input is not flushed between reads.Tests
Added
tests/test_cli_input.pywith unit tests for the UTF-8 helpers plus a pty-based integration test that drives_interactive_inputwithágua para as plantasand asserts the full string is returned.Full suite: 211 passed.