-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[improve][client]: add --streaming option to topics partitioned-stats-internal #23380
base: master
Are you sure you want to change the base?
Conversation
…-internal (apache#261) (cherry picked from commit 471669a)
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #23380 +/- ##
============================================
+ Coverage 73.57% 74.52% +0.94%
- Complexity 32624 34468 +1844
============================================
Files 1877 1934 +57
Lines 139502 145158 +5656
Branches 15299 15874 +575
============================================
+ Hits 102638 108176 +5538
+ Misses 28908 28679 -229
- Partials 7956 8303 +347
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of an InputStream
, a Reader
would be more appropriate for the interface.
The reason for this is in this case, the payload is most likely UTF-8 encoded which uses multi-byte characters for most non-ascii characters.
When reading in 2048 chunks, a multi-byte character might get split at the boundary which will corrupt the output.
Using a Reader
would address this without any additional logic.
A long time ago I worked on such a fix for Gradle, https://issues.gradle.org/browse/GRADLE-3329. In that case there was custom buffering and fixing the issue wasn't trivial. I ended up using java.nio.charset.CharsetDecoder
and java.nio.charset.CoderResult#isUnderflow
to detect when a byte is missing.
A Reader
will buffer the underlying bytes when there's a multi-byte character case and the remaining byte is not yet readable. That's why it's better to rely on a Reader
in the interface.
slightly related #18263 |
@dlg99 please address the review feedback, #23380 (review). |
Motivation
Streaming the stats output directly to the standard output without parsing the response in memory.
Modifications
added
--streaming
flag fortopics stats-internal/partitioned-stats-internal
commands to stream the stats.Verifying this change
This change added unit tests.
Does this pull request potentially affect one of the following parts:
added
--streaming
flag fortopics stats-internal/partitioned-stats-internal
commandsDocumentation
doc
doc-required
doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: dlg99#21