Skip to content

IDNA per-label ASCII leniency doesn't recognize the other UTS-46 dot-separator variants #69

Description

@OmarAlJarrah

Summary

Idna.domainToAsciiForUrl (via its asciiLenientDomainToAscii helper) decides, label by label,
whether an all-ASCII label gets kept verbatim (the WHATWG web-compat leniency for an invalid xn--
label) or has to run the full UTS-46 pipeline. That decision is made by splitting the domain on a
literal . before UTS-46 mapping runs.

The UTS-46 mapping table also maps three other code points to . (U+002E):

  • U+3002 IDEOGRAPHIC FULL STOP
  • U+FF0E FULLWIDTH FULL STOP
  • U+FF61 HALFWIDTH IDEOGRAPHIC FULL STOP

These are legitimate, browser-accepted domain-label separators — that's exactly why the mapping
table maps them to . in the first place. Because the per-label split runs on the raw text before
that mapping happens, a label boundary formed by one of these three characters is invisible to the
leniency decision: the two labels either side of it get merged into a single non-ASCII "label" by
the naive splitter and handed to the strict pipeline as one unit, which fails outright the moment it
hits an invalid xn-- label instead of applying the intended per-label leniency to it.

Trigger

// literal ASCII dot: per-label leniency applies correctly
Idn.toAscii("xn--a.bücher") // => Ok(xn--a.xn--bcher-kva)

// ideographic full stop (U+3002) instead of '.': leniency doesn't kick in
Idn.toAscii("xn--a。bücher") // => Err(InvalidHost(..., IdnaFailed))

xn--a is invalid Punycode either way; the only difference is which separator joins it to the
neighboring Unicode label. The first case is the correctly-fixed one; the second rejects the whole
domain over the same invalid label, which is the exact failure mode the per-label leniency is meant
to avoid when a sibling label is non-ASCII.

Where it comes from

Idna.asciiLenientDomainToAscii splits on LABEL_SEPARATOR (.) directly over the raw domain
parameter, before any UTS-46 mapping has run. domainToAscii (the strict pipeline it delegates
non-ASCII spans to) does the equivalent split correctly, because by the time it calls
splitLabels, mapAll has already turned any of the three variants above into a literal .. The
mismatch is that the outer per-label ASCII/non-ASCII gate runs before that normalization, while the
inner strict-path split runs after it.

This isn't a new regression — the pre-existing whole-domain-ASCII gate had the identical outcome for
this input (a domain containing any non-ASCII text, including these separators, went through the
strict pipeline with no leniency at all). It's a gap in the per-label leniency fix's completeness for
these three characters specifically.

Proposal

Normalize (or at least recognize) the full UTS-46 label-separator set — U+002E, U+3002, U+FF0E, and
U+FF61 — when deciding label boundaries for the leniency decision, rather than splitting on . only
before mapping has run. The simplest fix is probably to run mapAll over the domain first and split
on the result, mirroring what domainToAscii already does internally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions