Skip to content

fix(app): make instances with non-UTF8 text file encodings launcheable and importable #3721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AlexTMjugador
Copy link
Member

Overview

As reported in issue #794, the Modrinth App fails to launch a Minecraft instance if its options.txt file is not encoded in UTF-8, and settings that require modifying such file, such as launching the game in fullscreen mod, are enabled.

While non-UTF-8 encodings are becoming less common every day, and UTF-8's backward compatibility with 7-bit ASCII often masks encoding mismatch issues effectively, older Minecraft versions used the JVM's default system encoding when manipulating this file, and on Windows, this default is often a locale-dependent extended ASCII encoding (i.e., a code page), unlike (in my opinion, superior) Unix-like systems (Linux, macOS, BSDs), which default to UTF-8.

This PR addresses the issue by enhancing the internal utility function responsible for reading files into strings so that the file's encoding is detected, and text on it automatically decoded with the guessed encoding. Experimentally, I've found that UTF-8, Windows-1252/ISO 8859-1, and Shift JIS encodings are accurately identified with no relevant false positives. That function now also returns the detected encoding, allowing the file to be modified and saved without changing its original encoding, which is necessary because Minecraft may rely on that encoding staying the same.

Additionally, since this utility function is also used when reading metadata files while importing instances from another launchers, these changes also fix similar issues during that process.

Testing

I manually tested the changes by generating several variations of an options.txt file in different encodings. In each case, not only was the reported issue resolved, but the modified file also retained its original encoding.

@AlexTMjugador AlexTMjugador requested review from Gaming32 and Copilot May 30, 2025 21:07
@AlexTMjugador AlexTMjugador added bug Something isn't working app Relates to Modrinth App labels May 30, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses launch and import issues for Minecraft instances with non-UTF8 encoded files by enhancing file reading utilities to detect and preserve file encodings. Key changes include:

  • Introducing a new async function (read_any_encoding_to_string) in io.rs to detect and decode file encodings.
  • Refactoring various modules (launcher, mmc, gdlauncher, curseforge, atlauncher) to use the new function.
  • Adding necessary dependencies (chardetng and encoding_rs) in Cargo.toml for encoding detection and conversion.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/app-lib/src/util/io.rs Replaces read_to_string with read_any_encoding_to_string to support non-UTF8 encodings.
packages/app-lib/src/launcher/mod.rs Adapts options.txt handling to detect encoding and safely write back using the original encoding.
packages/app-lib/src/api/pack/import/mmc.rs Updates file reading calls to use the new encoding-aware function.
packages/app-lib/src/api/pack/import/gdlauncher.rs Applies the new file reading function with a default fallback for config.json.
packages/app-lib/src/api/pack/import/curseforge.rs Replaces direct string reads with encoding-aware reads for minecraftinstance.json.
packages/app-lib/src/api/pack/import/atlauncher.rs Updates instance.json reading to support multiple encodings.
packages/app-lib/Cargo.toml & Cargo.toml Adds chardetng and encoding_rs dependencies.

…e and importable

Previous to these changes, the app always assumed that Minecraft and
other launchers always use UTF-8, which is not necessarily always true.
@AlexTMjugador AlexTMjugador force-pushed the alex/fix-app-non-utf8-mc-launch-and-import branch from 6a61f44 to d2f4eef Compare June 5, 2025 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
app Relates to Modrinth App bug Something isn't working
Development

Successfully merging this pull request may close these issues.

UTF-8 Error when launching 1.8.9 Forge Modpack An error occurred I/O error: stream did not contain valid UTF-8, path:
1 participant