Fix parser silently accepting a lone trailing surrogate in string values

The C parser guards leading-first surrogate pairs but silently accepts
a lone trailing surrogate, encoding U+DC00..U+DFFF as three UTF-8 bytes
into the output buffer. The returned Ruby String is tagged UTF-8 but
fails valid_encoding? and raises misleading errors from downstream
String operations (upcase, split, regex, encode, JSON.generate).

Add a symmetric branch alongside the leading-surrogate check to raise
JSON::ParserError at the parse boundary. Extend test_invalid_surogates
with the new cases and move the three JSONTestSuite fixtures that this
fix newly rejects from INVALID_ENCODING_TESTS into UNDEFINED_FAILING,
closing the CRuby/JRuby parser parity gap documented in the file.

Author: elang2

CHANGES.md

@@ -3,6 +3,7 @@
 ### Unreleased
 
 * Add `JSON::ParserError#json_path` to locate parse errors in the document as a JSONPath-style string (e.g. `$.foo[0].bar`). For duplicate key errors it points at the duplicated key itself.
+* Fix the parser to also reject lone trailing UTF-16 surrogates (`\uDCxx` with no leading partner), symmetric to the leading-surrogate case. The Java parser already rejected these; this closes the CRuby/JRuby parity gap.
 
 ### 2026-08-11 (3.0.0.rc1)
 

ext/json/ext/parser/parser.c

@@ -1041,6 +1041,9 @@ NOINLINE(static) VALUE json_string_unescape(JSON_ParserState *state, JSON_Parser
                         raise_syntax_error_at("incomplete surrogate pair at %s", state, p);
                         break;
                     }
+                } else if ((ch & 0xFC00) == 0xDC00) {
+                    raise_syntax_error_at("unpaired trailing surrogate at %s", state, p);
+                    break;
                 }
 
                 int unescape_len = convert_UTF32_to_UTF8(buffer, ch);

test/json/json_minefield_parser_test.rb

@@ -22,12 +22,9 @@ def define_test(name, &block)
     i_string_overlong_sequence_2_bytes
     i_string_not_in_unicode_range
     i_string_lone_utf8_continuation_byte
-    i_string_lone_second_surrogate
     i_string_iso_latin_1
     i_string_invalid_utf-8
-    i_string_incomplete_surrogate_pair
     i_string_UTF-8_invalid_sequence
-    i_object_key_lone_2nd_surrogate
   )
 
   COMMENT_TESTS = %w(
@@ -56,6 +53,9 @@ def define_test(name, &block)
     i_string_utf16BE_no_BOM
     i_string_utf16LE_no_BOM
     i_structure_UTF-8_BOM_empty_object
+    i_string_lone_second_surrogate
+    i_string_incomplete_surrogate_pair
+    i_object_key_lone_2nd_surrogate
   )
 
   if RUBY_ENGINE == 'jruby'

test/json/json_parser_test.rb

@@ -403,6 +403,15 @@ def test_invalid_surogates
     assert_raise(JSON::ParserError) { parse('"\\uD800_________________"') }
     assert_raise(JSON::ParserError) { parse('"\\uD800\\u0041"') }
     assert_raise(JSON::ParserError) { parse('"\\uD800\\u004') }
+    # Lone trailing surrogate (issue #1069): parser previously returned an
+    # invalid-UTF-8 String instead of raising. Symmetric to the leading cases
+    # above.
+    assert_raise(JSON::ParserError) { parse('"\\uDC00"') }
+    assert_raise(JSON::ParserError) { parse('"\\uDC00_________________"') }
+    assert_raise(JSON::ParserError) { parse('"\\uDC00\\uD800"') }
+    # Valid pair still parses to the astral codepoint U+10000.
+    assert_predicate JSON.parse('"\\uD800\\uDC00"'), :valid_encoding?
+    assert_equal "\u{10000}", JSON.parse('"\\uD800\\uDC00"')
   end
 
   def test_parse_big_integers