Repository navigation
Commit 9dbfeb8
Fix ResumableParser treating unterminated line comments as complete
A `//` line comment split across a feed boundary was treated as fully
consumed, so comment body delivered in a later chunk leaked out as parsed
JSON values.
## Reproduction
```ruby
require "json"
parser = JSON::ResumableParser.new(allow_comments: true)
documents = []
"[1]//[999]\n[3]".each_char do |char|
parser << char
documents << parser.value while parser.parse
end
p documents
```
## Expected Behavior
```
[[1], [3]]
```
The `[999]` is inside the `//` comment (which runs until the newline), so it
is ignored, leaving the two documents `[1]` and `[3]`. This is what feeding
the whole string in a single chunk produces.
## Actual Behavior
```
[[1], [999], [3]]
```
The commented-out `[999]` leaked out as a real value. When the comment body
does not form valid JSON the split feed raises a ParserError instead. This
happens with the default configuration too, since comments are accepted
(with a deprecation warning) unless disabled.
## Description
A `//` line comment is only terminated by a newline. `json_eat_comments`
searched for that newline and, when none was present in the buffer, moved
the cursor to the end of the buffer and returned as if the comment had been
fully consumed.
For the one-shot `JSON.parse` this is correct: the buffer holds the whole
document, so a `//` comment with no trailing newline genuinely runs to the
end of input. For `ResumableParser`, however, the end of the buffer is only
a chunk boundary, not the end of input. Reaching it does not mean the comment
is over -- the newline (and possibly more comment body) may still arrive in a
later `<<`. Swallowing the buffered bytes as a finished comment loses the
"still inside a comment" state, so any comment body delivered in the next
chunk was parsed as JSON instead of being ignored. Because the result then
depended on where the input happened to be split, the same byte stream could
produce different values.
The block-comment branch already handles this correctly: when it cannot find
the closing `*/` it rewinds to the comment start and raises an EOS-tagged
error, which `ResumableParser#parse` swallows into a `false` return so the
comment is retried once more input is available. This change makes the
line-comment branch behave the same way, but only in resumable mode
(detected via `state->parser`, the same discriminator `raise_parse_error`
already uses). In non-resumable mode the previous behaviour is preserved: a
`//` comment with no trailing newline is still consumed to the end of input.
As a consequence, a stream that ends with an unterminated line comment now
stays incomplete (`parse` keeps returning `false`) until a newline arrives,
which mirrors how a trailing bare number is only considered complete once a
following separator is seen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PpeQSEFkF1X1Uuzjx9BsYe1 parent ea008e8 commit 9dbfeb8
3 files changed
Lines changed: 53 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
779 | 779 | | |
780 | 780 | | |
781 | 781 | | |
782 | | - | |
783 | | - | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
784 | 791 | | |
785 | 792 | | |
786 | | - | |
| 793 | + | |
787 | 794 | | |
788 | 795 | | |
789 | 796 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
526 | 526 | | |
527 | 527 | | |
528 | 528 | | |
| 529 | + | |
529 | 530 | | |
530 | 531 | | |
531 | 532 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
202 | 244 | | |
203 | 245 | | |
204 | 246 | | |
| |||
0 commit comments