feat(mojibake): new rule MD083 for mojibake detection

[guillp]

What

Add a rule for mojibake detection, as discussed in #687

I made the rule opt-in, but it is probably worth enabling it by default. You decide @rvben :)

[rvben]

Thanks for this, the detection genuinely works. I built the branch and ran it over a deliberately mojibaked corpus and it caught every case, including the mixed é/â€" families and doubly-encoded Cyrillic. Opt-in by default and FixCapability::Unfixable are both the right calls, and 45 tests is a solid base.

Before it can land I need the false positives closed. Clean prose in Portuguese, Spanish, German, Greek, Russian, Danish and Icelandic is untouched, and so are symbols, maths, box drawing and typographic punctuation, so this is narrow rather than pervasive. Three things:

1. [Œœ][^A-Za-z] fires on ordinary French. Any œ or Œ not followed by an ASCII letter matches, so a sentence discussing the ligature itself trips it: La ligature œ est utilisée, Le caractère œ se compose, and the same for the capital. Four hits in one page of normal French typography prose.
2. [a-z]\s?[ÃÂ][\s] fires on letter enumerations. Romanian sunt cuvinte cu  și Î and Vietnamese nguyên âm có dấu mũ: a  e Ê o Ô both match. Naming a letter is a normal thing to do in a document about that language.
3. No code-block exemption. Mojibake is flagged inside inline code spans and fenced and indented blocks, which is a problem for exactly the documents that discuss encoding. The clearest evidence is that this PR's own docs/md083.md trips MD083 ten times.

There is a structural reason for 1 and 2. These patterns come from ftfy's BADNESS_RE, where they are weighted contributions to a score compared against a threshold. Ported to binary matching, a single weak signal becomes a finding on its own, so the false positive rate goes up by construction. Either restore something threshold-like, or drop the individual patterns that cannot carry a finding alone.

Also: from_config ignores the config, so MD083 has no options. Since the rule is heuristic, users need an escape hatch narrower than disabling it outright.

One process note: no CI has run yet, since fork PRs need workflow approval. I'll approve a run once the above is addressed.

[guillp]

@rvben 👍🏻 Thanks for the feedback, all very valid points.

• about french œ : I should know, I'm french 🇫🇷 . I edited the pattern [Œœ][^A-Za-z] to allow a more characters after the ligature than just letters, i.e. spaces and most kinds of quotes.
• about single letters: I removed the pattern [a-z]\s?[ÃÂ][\s] completely.
• I made the rule configurable with both an allow-list of ignored mojibake, and an option to ignore code blocks, that is enabled by default.

[rvben]

Thanks, fast turnaround. Approved CI and verified against a build. Letter enumerations and the config (ignore + ignore-code-blocks) are both solid now.

CI is red on two new-rule bookkeeping items:

1. Rule-count docs out of sync (MD083 makes it 77 rules, docs still say 76, RULE_MAX still MD082). make sync-rule-docs updates the sentinels; then add the MD083 rows to docs/rules.md by hand.
2. test_all_53_rules_systematic_coverage fails: MD083 has no arm in get_test_content_for_rule (tests/cli/cli_lsp_fix_consistency.rs), tripping assert!(test_content_missing < 15). Add e.g. "MD083" => Some("Broken dash – here"),.

Two false positives still open:

3. French œ: [Œœ][^A-Za-z\s"'»”’] still fires before ordinary punctuation, e.g. la ligature (œ), œ,, œ., œ;. Extending the exclusion set will keep leaking. Since real œ mojibake only appears inside “ (already caught elsewhere), I'd drop the standalone œ pattern rather than enumerate trailing chars. Your call.
4. Inline code spans aren't exempted (only blocks are), so `Café` still flags and docs/md083.md itself trips 9 times. Exempt inline code, or move the doc examples into fenced blocks.

Once 1-3 land and you've decided on 4, I'll merge. Happy to push the two bookkeeping fixes myself if you'd rather focus on the regex.

[guillp]

1. fixed (please review the opt-in rationale in docs/rules.md)
2. fixed
3. I removed the oe specific pattern
4. I am actually not sure how to exclude inline code blocks. Is there some built-in features in Rumdl LintContext to obtain those code blocks, or should I detect them myself in the rule ? This looks a bit too complex when I look at rule MD048...

[rvben]

Thanks for this, @guillp. MD083 is a genuinely useful addition and the character-class detection approach is clean.

I've merged it and finished the last review item (exempting inline code spans) directly on the branch so it didn't need another round-trip. Mojibake inside a backtick span is now skipped when ignore-code-blocks is set, with regression tests covering the single-line, mixed prose/code, and multi-line-span cases. I also updated the rule-count guards and cleared a few clippy/rustfmt nits to get CI green.

This ships in v0.2.42. Thanks again for the contribution!