Support for 8-bit ASCII and unescaped UTF8 characters #28
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
October 14, 2019 13:41
…-parser into 8bit-ascii-support Conflicts: JsonStreamingParser.cpp
|
It was just something quick I put together while trying to use your library in a PoC, but I see it needs some polishing to be able to pass through tests. If you are interested in merging this patch, I can update the PR when I have some spare time. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The original code uses signed
chartype for variables that hold the character being processed. It breaks 8-bit ASCII characters used in many European languages, but also prevents the library from correctly handling unescaped UTF8 characters. The attached patch addresses both these problems and further improves support for UTF8-encoded strings. I am aware that updating all methods wasn't necessary to make this work, but I thought perhaps it was a good idea to still do so for consistency.