Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions changelog/25.0/25.0.0/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
- **[Minor Changes](#minor-changes)**
- **[VReplication](#minor-changes-vreplication)**
- [Default data protection for `_reverse` workflow cancel/complete](#vreplication-reverse-workflow-data-protection)
- [`vdiff show --only-summary` omits the per-table row-sample report](#vreplication-vdiff-only-summary)
- **[VTGate](#minor-changes-vtgate)**
- [Ingress bytes in query LogStats](#vtgate-logstats-ingress-bytes)
- [New controls for cross-keyspace reads](#vtgate-cross-keyspace-reads)
Expand Down Expand Up @@ -170,6 +171,18 @@ The `--keep-data` flag help text has been updated to note this default explicitl

See [#19906](https://github.com/vitessio/vitess/pull/19906) for details.

#### <a id="vreplication-vdiff-only-summary"/>`vdiff show --only-summary` omits the per-table row-sample report</a>

`vtctldclient vdiff ... show` now accepts an `--only-summary` flag. When set, the per-table diff report has its sampled-row arrays (`MismatchedRowsSample`, `ExtraRowsSourceSample`, `ExtraRowsTargetSample`) stripped on the tablet before the response is built, while the scalar counters (processed, matching, mismatched, and extra rows) and all other summary fields are preserved.

The sampled-row arrays carry actual row data, including large `BLOB`/`JSON` columns, and `vdiff show` aggregates them across every target shard. For diffs over tables with large rows this could push the aggregated response past gRPC message limits and make `vdiff show` fail outright, leaving no way to read the summary or mismatch state. `--only-summary` lets callers that only need progress and the mismatch state avoid transferring the samples while keeping the reported counts accurate.

The option is exposed as `only_summary` on the `VDiffShowRequest` (vtctld) and `VDiffReportOptions` (tablet) protobuf messages. It is opt-in and backward compatible: without the flag, the full report is returned as before.

`vdiff create --wait` also uses `only_summary` for its internal progress polls, so the wait loop no longer transfers the sampled-row arrays on every interval and cannot be killed by the gRPC message limit on large diffs. The text output is unchanged; with `--format json`, the per-interval progress output no longer includes the sampled rows (the full samples remain available via `vdiff show --verbose` once the diff completes).

See [#20870](https://github.com/vitessio/vitess/pull/20870) for details.

### <a id="minor-changes-vtgate"/>VTGate</a>

#### <a id="vtgate-logstats-ingress-bytes"/>Ingress bytes in query LogStats</a>
Expand Down
14 changes: 12 additions & 2 deletions go/cmd/vtctldclient/command/vreplication/vdiff/vdiff.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,9 @@ var (
}{}

showOptions = struct {
Arg string
Verbose bool
Arg string
Verbose bool
OnlySummary bool
}{}

stopOptions = struct {
Expand Down Expand Up @@ -318,10 +319,17 @@ func commandCreate(cmd *cobra.Command, args []string) error {
case <-ctx.Done():
return vterrors.Errorf(vtrpcpb.Code_CANCELED, "context has expired")
case <-tkr.C:
// This is a progress poll that only renders a single non-verbose
// summary (state, counts, has_mismatch), so request only_summary:
// the per-row sample arrays are never shown here and fetching them
// on every interval can push the aggregated response past the gRPC
// message limit for large diffs, killing the wait. The authoritative
// samples remain available via `vdiff show --verbose`.
resp, err := vtctldClient.VDiffShow(ctx, &vtctldatapb.VDiffShowRequest{
Workflow: common.BaseOptions.Workflow,
TargetKeyspace: common.BaseOptions.TargetKeyspace,
Arg: uuidStr,
OnlySummary: true,
Comment thread
pedroalb marked this conversation as resolved.
Outdated
Comment thread
pedroalb marked this conversation as resolved.
Outdated
})
if err != nil {
return err
Expand Down Expand Up @@ -645,6 +653,7 @@ func commandShow(cmd *cobra.Command, args []string) error {
Workflow: common.BaseOptions.Workflow,
TargetKeyspace: common.BaseOptions.TargetKeyspace,
Arg: showOptions.Arg,
OnlySummary: showOptions.OnlySummary,
})
if err != nil {
return err
Expand Down Expand Up @@ -709,6 +718,7 @@ func registerCommands(root *cobra.Command) {
base.AddCommand(resume)

show.Flags().BoolVar(&showOptions.Verbose, "verbose", false, "Show verbose output in summaries")
show.Flags().BoolVar(&showOptions.OnlySummary, "only-summary", false, "Omit the per-table diff report body and return only the vdiff and per-table summary state. Useful for large diffs where the report can exceed gRPC message limits.")
Comment thread
pedroalb marked this conversation as resolved.
Outdated
Comment thread
pedroalb marked this conversation as resolved.
Outdated
base.AddCommand(show)

stop.Flags().StringSliceVar(&stopOptions.TargetShards, "target-shards", nil, "The target shards to stop the vdiff on; default is all shards.")
Expand Down
23 changes: 19 additions & 4 deletions go/vt/proto/tabletmanagerdata/tabletmanagerdata.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 34 additions & 0 deletions go/vt/proto/tabletmanagerdata/tabletmanagerdata_vtproto.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 16 additions & 3 deletions go/vt/proto/vtctldata/vtctldata.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 34 additions & 0 deletions go/vt/proto/vtctldata/vtctldata_vtproto.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions go/vt/vtctl/workflow/vdiff.go
Original file line number Diff line number Diff line change
Expand Up @@ -566,6 +566,11 @@ func (s *Server) VDiffShow(ctx context.Context, req *vtctldatapb.VDiffShowReques
Workflow: req.Workflow,
Action: string(vdiff.ShowAction),
ActionArg: req.Arg,
Options: &tabletmanagerdatapb.VDiffOptions{
ReportOptions: &tabletmanagerdatapb.VDiffReportOptions{
OnlySummary: req.GetOnlySummary(),
},
},
}

ts, err := s.buildTrafficSwitcher(ctx, req.TargetKeyspace, req.Workflow)
Expand Down
Loading