Skip to content

char_(from, to) and char_("set") should reject invalid code units #45

Description

@saki7

This issue is not just about the implementation quality, it is actually about the validity of the design.

There are currently multiple layers of misfeature:

  1. char_('Z', 'A') -- should be rejected
  2. char_(from, to) with non-single code unit param -- should be rejected
  3. char_("Z-A") -- should be rejected
  4. char_("あ") -- should be rejected, while char_(U'あ') should be maintained
  5. char_("abcあdef") -- should be rejected, while char_(U"abcあdef") should be maintained

The real problem is that we can fix x4::unicode but can't do anything about x4::standard and x4::standard_wide.

Non-unicode variants operate on the execution character set, and moreover, our implementation currently uses <cctype>, which utilizes the C locale. So they don't have well-defined meaning for "character ranges" or "character sets".

Note that I used the term "code units", which is the Unicode terminology, merely for explanation, and C locales may or may not be Unicode in practice. Even if some C locale happens to be Unicode, the case like (5.) cannot be resolved for x4::standard because multi-code-unit character like u8"あ" is not representable by the character parser; i.e., (char)u8'あ' is invalid even when char is UTF-8.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is neededtests neededUnit tests are required for all public API

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions