Skip to content

fix: Support reading from files that have an UTF-8 Byte Order Mark #670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 9, 2025

Conversation

ldemidov
Copy link
Contributor

@ldemidov ldemidov commented Jul 8, 2025

Adds support for reading files with UTF-8 BOM. This is commonly created by Windows text editors and should be skipped because serde deserialization will not handle those bytes.

We have encountered this issue with our Windows customers who may create UTF-8 BOM files without their knowledge. Although we fixed it with a custom FileSource implementation, it would be nice to have this in the upstream to help others who may run into this issue.

This PR came from discussion in #565
Unlike that PR, this one handles only UTF-8 Boms, and not other encodings, and does not pull in any new dependencies.

  • Adds a test with a UTF-8 BOM text file.
  • Updates FileSourceFile to skip the 3 BOM bytes if they are detected.

@ldemidov ldemidov changed the title Support-utf8-bom-files fix: Support reading from files that have an UTF-8 Byte Order Mark Jul 8, 2025
@coveralls
Copy link

Pull Request Test Coverage Report for Build 16156062432

Details

  • 6 of 6 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.1%) to 64.77%

Totals Coverage Status
Change from base Build 16154172080: 0.1%
Covered Lines: 945
Relevant Lines: 1459

💛 - Coveralls

@epage epage merged commit d8bdf0f into rust-cli:main Jul 9, 2025
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants