Skip to content

l10n.md

Latest commit

 

History

History
215 lines (138 loc) · 5.49 KB

l10n.md

File metadata and controls

215 lines (138 loc) · 5.49 KB
Raw

🌍 Localization (L10n) in uutils coreutils

This guide explains how localization (L10n) is implemented in the Rust-based coreutils project, detailing the use of Fluent files, runtime behavior, and developer integration.

🏗️ Architecture Overview

English (US) locale files (en-US.ftl) are embedded directly in the binary, ensuring that English always works regardless of how the software is installed. Other language locale files are loaded from the filesystem at runtime.

Source Repository Structure

  • Main repository: Contains English (en-US.ftl) locale files embedded in binaries
  • Translation repository: uutils/coreutils-l10n contains all other language translations
  • Online Translation: weblate/rust-coreutils to translate the strings.

📁 Fluent File Layout

Each utility has its own set of translation files under:

    src/uu/<utility>/locales/<locale>.ftl

Examples:

    src/uu/ls/locales/en-US.ftl    # Embedded in binary
    src/uu/ls/locales/fr-FR.ftl    # Loaded from filesystem

These files follow Fluent syntax and contain localized message patterns.

The French translation is the only locale with English to be part of the tree. The goal is to be able to run tests with a different locale to verify that they work.

⚙️ Initialization

Localization must be explicitly initialized at runtime using:

    setup_localization(path)

This is typically done:

  • In src/bin/coreutils.rs for multi-call binaries
  • In src/uucore/src/lib.rs for single-call utilities

The string parameter determines the lookup path for Fluent files. English always works because it's embedded, but other languages need their .ftl files to be available at runtime.

🌐 Locale Detection

Locale selection is automatic and performed via:

    fn detect_system_locale() -> Result<LanguageIdentifier, LocalizationError>

It reads the LANG environment variable (e.g., fr-FR.UTF-8), strips encoding, and parses the identifier.

If parsing fails or LANG is not set, it falls back to:

    const DEFAULT_LOCALE: &str = "en-US";

You can override the locale at runtime by running:

    LANG=ja-JP ./target/debug/ls

📥 Retrieving Messages

We have a single macro to handle translations. It can be used in two ways:

translate!(id: &str) -> String

Returns the message from the current locale bundle.

    let msg = translate!("id-greeting");

If not found, falls back to en-US. If still missing, returns the ID itself.

translate!(id: &str, args: key-value pairs) -> String

Supports variable interpolation and pluralization.

    let msg = translate!(
        "error-io",
        "error" => std::io::Error::last_os_error()
    );

Fluent message example:

    error-io = I/O error occurred: { $error }

Variables must match the Fluent placeholder keys ($error, $name, $count, etc.).

📦 Fluent Syntax Example

    id-greeting = Hello, world!
    welcome = Welcome, { $name }!
    count-files = You have { $count ->
        [one] { $count } file
       *[other] { $count } files
    }

Use plural rules and inline variables to adapt messages dynamically.

🧪 Testing Localization

Run all localization-related unit tests with:

    cargo test --lib -p uucore

Tests include:

  • Loading bundles
  • Plural logic
  • Locale fallback
  • Fluent parse errors
  • Thread-local behavior
  • ...

🧵 Thread-local Storage

Localization is stored per thread using a OnceLock. Each thread must call setup_localization() individually. Initialization is one-time-only per thread — re-initialization results in an error.

🧪 Development vs Release Mode

During development (cfg(debug_assertions)), paths are resolved relative to the crate source:

    $CARGO_MANIFEST_DIR/../uu/<utility>/locales/

In release mode, paths are resolved relative to the executable:

    <executable_dir>/locales/<utility>/
    <prefix>/share/locales/<utility>/
    ~/.local/share/coreutils/locales/<utility>/
    ~/.cargo/share/coreutils/locales/<utility>/
    /usr/share/coreutils/locales/<utility>/

If external locale files aren't found, the system falls back to embedded English locales.

🔤 Unicode Isolation Handling

By default, the Fluent system wraps variables with Unicode directional isolate characters (U+2068, U+2069) to protect against visual reordering issues in bidirectional text (e.g., mixing Arabic and English).

In this implementation, isolation is disabled via:

    bundle.set_use_isolating(false);

This improves readability in CLI environments by preventing extraneous characters around interpolated values:

Correct (as rendered):

    "Welcome, Alice!"

Fluent default (disabled here):

    "\u{2068}Alice\u{2069}"

🔧 Embedded English Locales

English locale files are always embedded directly in the binary during the build process. This ensures that:

  • English always works regardless of installation method (e.g., cargo install)
  • No runtime dependency on external .ftl files for English
  • Fallback behavior when other language files are missing

The embedded English locales are generated at build time and included in the binary, providing a reliable fallback while still supporting full localization for other languages when their .ftl files are available.