vtgate: make sql_mode a session-owned setting

changelog/25.0/25.0.0/summary.md

@@ -32,6 +32,8 @@
         - [Stricter validation of SQL-level PREPARE statements](#vtgate-prepare-stricter-validation)
         - [Stricter PROXY protocol v1 header validation](#vtgate-proxy-protocol-v1-strictness)
         - [MySQL-faithful `sql_mode` validation, and support for the parse-relevant modes](#vtgate-sql-mode-validation)
+        - [Sessions start with a well-defined default `sql_mode` (new `--sql-mode` flag)](#vtgate-sql-mode-session-default)
+        - [The session's `sql_mode` is applied to every backend query](#vtgate-sql-mode-always-sent)
         - [MySQL-faithful lexing of built-in function names](#sqlparser-function-name-keywords)
         - [New `VEXPLAIN MYSQLPLAN` statement](#vtgate-vexplain-mysqlplan)
         - [Multi-statement queries are split the way MySQL splits them](#vtgate-multi-statement-splitting)
@@ -365,6 +367,28 @@ Incoming v25-canonical query text is unaffected: it parses identically under v24
 
 **Impact**: Clients can now `SET sql_mode` to any value their MySQL version accepts, and VTGate honors it for parsing. Invalid values fail with the same errors MySQL returns.
 
+**Impact**: Clients that issue `SET sql_mode` with an unsupported mode now receive an error, also when the `SET` is a no-op that matches the backend's existing `sql_mode`. Clients that set mode names the backend MySQL would itself reject receive an error as well. Such sessions were already unreliable, because VTGate parses queries without honoring these modes. Clients that read `@@sql_mode` back will see MySQL's canonical form instead of their original spelling.
+
+#### <a id="vtgate-sql-mode-session-default"/>Sessions start with a well-defined default `sql_mode` (new `--sql-mode` flag)</a>
+
+VTGate sessions now start with a well-defined `sql_mode`. They no longer implicitly inherit whatever each backend happens to be configured with. A new `--sql-mode` flag sets this session default. The flag is named after MySQL's flag and is validated the same way as a `SET sql_mode` statement. Its default is MySQL 8.x's factory default, the mode Vitess-managed `mysqld` instances run with. As a result:
+
+- VTGate answers `SELECT @@sql_mode`, `SHOW VARIABLES LIKE 'sql_mode'`, and expressions that reference `@@sql_mode` from the session. It never asks an arbitrary shard. Previously, reads and `SET sql_mode` change-detection queried one shard and assumed all backends agreed on `sql_mode`. Nothing guarantees that in a multi-backend cluster.
+- `SET sql_mode = <expr>` is evaluated at the VTGate, with no shard round trip for what VTGate can compute itself. This includes expressions such as `CONCAT(@@sql_mode, '...')`, `@@global.sql_mode`, and user variables. A sub-expression VTGate cannot compute, such as `RAND()` or a system variable the session never set, is fetched from a shard, with the session's `@@sql_mode` passed along, and the result is validated and stored like any other value. `SET` statements no longer force a reserved connection on MySQL 8.0+, where `SET_VAR` is available.
+- `@@global.sql_mode` resolves to the configured default (the `--sql-mode` value), never to a backend's global. In this model the VTGate owns the global `sql_mode`, so `SET sql_mode = @@global.sql_mode` restores the session default.
+
+**Impact**: Sessions that never set `sql_mode` read the configured session default. They no longer read the value of whichever shard happened to answer. Deployments whose backends intentionally run a non-default global `sql_mode` should set `--sql-mode` to that value, so that VTGate's session semantics match their backends. They can also opt out entirely with `--enable-system-settings=false` (see the next section).
+
+#### <a id="vtgate-sql-mode-always-sent"/>The session's `sql_mode` is applied to every backend query</a>
+
+The session's `sql_mode` is now applied to every query sent to the backends, whether or not the session ever set it. Statements that can carry optimizer hints get a `SET_VAR(sql_mode = ...)` hint. Statements that cannot, such as DDL, run on a connection with the session's settings applied through the tablet's settings pool. Previously, sessions that never set `sql_mode` ran queries under whatever mode each backend happened to be configured with, while VTGate reported a different mode to the client. Relatedly, the `SET_VAR` comment is now placed only in the top-level statement comment, where MySQL honors it, instead of also being duplicated into subqueries. `vexplain` and `explain` hint the statement they wrap.
+
+The value sent has exactly one mode removed, `NO_BACKSLASH_ESCAPES`, as described in [the validation section](#vtgate-sql-mode-validation): a session's `sql_mode` governs how the client's SQL is interpreted, the queries a backend receives are always serialized in VTGate's canonical format, and that format is inert under every mode but the backslash-escaping one. Every other mode is forwarded, the parse-relevant ones included, for the backend to enforce its resolution- and execution-time semantics.
+
+Deployments that run with `--enable-system-settings=false` are exempt. Their sessions are not seeded with a `sql_mode`, and no `SET_VAR` hint is sent, so queries keep running under each backend's configured mode, as before. This is the opt-out for deployments that want each backend's configured `sql_mode` to keep governing its own queries.
+
+**Impact**: Queries now run under the `sql_mode` VTGate reports to the client, instead of each backend's configured mode. Note one warning-related effect: a `sql_mode` without a strict mode, such as bare `NO_ZERO_DATE`, makes MySQL attach deprecation warning 3135 to every query that carries the mode as a `SET_VAR` hint. This was already the case for sessions that `SET` such a mode. It now also applies to every session when `--sql-mode` is configured to a non-strict combination.
+
 #### <a id="vtgate-vexplain-mysqlplan"/>New `VEXPLAIN MYSQLPLAN` statement</a>
 
 A new `VEXPLAIN MYSQLPLAN <query>` statement runs MySQL's `EXPLAIN FORMAT=JSON` against the shards a `SELECT` would target, **without executing the query itself**. It resolves each `Route`'s target shards from its vindex at resolution time and issues `EXPLAIN` against every resolved shard, attaching the per-shard MySQL plan to the VTGate plan tree keyed by shard, so per-shard plan and cost differences are visible.

go/flags/endtoend/vtcombo.txt

@@ -355,6 +355,7 @@ Flags:
       --slow-query-threshold duration                                    Mark vtgate queries as slow when their total execution time meets or exceeds this duration. 0 disables slow-query detection.
       --sql-max-length-errors int                                        truncate queries in error logs to the given length (default unlimited)
       --sql-max-length-ui int                                            truncate queries in debug UIs to the given length (default 512) (default 512)
+      --sql-mode string                                                  The sql_mode every session starts with. The value is validated like a SET sql_mode statement. (default "ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION")
       --srv-topo-cache-refresh duration                                  how frequently to refresh the topology for cached entries (default 1s)
       --srv-topo-cache-ttl duration                                      how long to use cached entries for topology (default 1s)
       --srv-topo-timeout duration                                        topo server timeout (default 5s)

go/flags/endtoend/vtexplain.txt

@@ -74,6 +74,7 @@ Flags:
       --sql-file string                                             Identifies the file that contains the SQL commands to analyze
       --sql-max-length-errors int                                   truncate queries in error logs to the given length (default unlimited)
       --sql-max-length-ui int                                       truncate queries in debug UIs to the given length (default 512) (default 512)
+      --sql-mode string                                             The sql_mode every session starts with. The value is validated like a SET sql_mode statement. (default "ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION")
   -v, --version                                                     print binary version
       --vschema string                                              Identifies the VTGate routing schema
       --vschema-file string                                         Identifies the VTGate routing schema file

go/flags/endtoend/vtgate.txt

@@ -194,6 +194,7 @@ Flags:
       --slow-query-threshold duration                                    Mark vtgate queries as slow when their total execution time meets or exceeds this duration. 0 disables slow-query detection.
       --sql-max-length-errors int                                        truncate queries in error logs to the given length (default unlimited)
       --sql-max-length-ui int                                            truncate queries in debug UIs to the given length (default 512) (default 512)
+      --sql-mode string                                                  The sql_mode every session starts with. The value is validated like a SET sql_mode statement. (default "ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION")
       --srv-topo-cache-refresh duration                                  how frequently to refresh the topology for cached entries (default 1s)
       --srv-topo-cache-ttl duration                                      how long to use cached entries for topology (default 1s)
       --srv-topo-timeout duration                                        topo server timeout (default 5s)

go/flags/endtoend/vtgateclienttest.txt

@@ -64,6 +64,7 @@ Flags:
       --purge-logs-interval duration                                     how often try to remove old logs (default 1h0m0s)
       --security-policy string                                           the name of a registered security policy to use for controlling access to URLs - empty means allow all for anyone (built-in policies: deny-all, read-only)
       --service-map strings                                              comma separated list of services to enable (or disable if prefixed with '-') Example: grpc-queryservice
+      --sql-mode string                                                  The sql_mode every session starts with. The value is validated like a SET sql_mode statement. (default "ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION")
       --table-refresh-interval int                                       interval in milliseconds to refresh tables in status page with refreshRequired class
   -v, --version                                                          print binary version
       --vschema-ddl-authorized-users string                              List of users authorized to execute vschema ddl operations, or '%' to allow all users.

go/mysql/sqlmode/sqlmode.go

@@ -329,6 +329,20 @@ func checkRemoved(mode Mode) error {
 	return vterrors.NewErrorf(vtrpcpb.Code_INVALID_ARGUMENT, vterrors.UnsupportedSQLMode, "sql_mode=0x%08x is not supported.", uint64(removed))
 }
 
+// wrongValueErrorValueLen is the length MySQL truncates the offending value to in
+// ER_WRONG_VALUE_FOR_VAR, verified against MySQL 8.0.46: a longer value is cut, with no
+// marker, so an arbitrarily long expression result does not swell the error.
+const wrongValueErrorValueLen = 200
+
 func wrongValueError(value string) error {
+	if n := 0; len(value) > wrongValueErrorValueLen {
+		for i := range value {
+			if n == wrongValueErrorValueLen {
+				value = value[:i]
+				break
+			}
+			n++
+		}
+	}
 	return vterrors.NewErrorf(vtrpcpb.Code_INVALID_ARGUMENT, vterrors.WrongValueForVar, "Variable 'sql_mode' can't be set to the value of '%s'", value)
 }

go/mysql/sqlmode/sqlmode_test.go

@@ -17,6 +17,7 @@ limitations under the License.
 package sqlmode
 
 import (
+	"strings"
 	"testing"
 
 	"github.com/stretchr/testify/assert"
@@ -211,6 +212,21 @@ func TestValidate(t *testing.T) {
 	}
 }
 
+// A long invalid value is truncated in the error the way MySQL truncates it: to 200
+// characters, with no marker (verified against MySQL 8.0.46 with a 1 MB value).
+func TestWrongValueErrorTruncatesTheValue(t *testing.T) {
+	long := strings.Repeat("a", 1000000)
+	_, err := Parse(long)
+	require.EqualError(t, err, "Variable 'sql_mode' can't be set to the value of '"+strings.Repeat("a", 200)+"'")
+
+	// the cut falls on a character boundary
+	_, err = Parse(strings.Repeat("é", 201))
+	require.EqualError(t, err, "Variable 'sql_mode' can't be set to the value of '"+strings.Repeat("é", 200)+"'")
+
+	_, err = Parse(strings.Repeat("a", 200))
+	require.EqualError(t, err, "Variable 'sql_mode' can't be set to the value of '"+strings.Repeat("a", 200)+"'")
+}
+
 func TestValidateNoLexerModes(t *testing.T) {
 	// the check covers exactly the LexerModes, reporting combination modes under their
 	// own name, and passes everything else

go/test/endtoend/reparent/utils/utils.go

@@ -23,6 +23,7 @@ import (
 	"os"
 	"os/exec"
 	"path"
+	"regexp"
 	"strings"
 	"testing"
 	"time"
@@ -844,6 +845,10 @@ func WaitForTabletToBeServing(ctx context.Context, t *testing.T, clusterInstance
 	}
 }
 
+// optimizerHintRE matches optimizer hint comments (`/*+ ... */`) that vtgate
+// injects after the leading verb of queries sent to vttablet.
+var optimizerHintRE = regexp.MustCompile(`\s*/\*\+.*?\*/`)
+
 // WaitForQueryWithStateInProcesslist waits for a query to be present in the processlist with a specific state.
 func WaitForQueryWithStateInProcesslist(ctx context.Context, t *testing.T, tablet *cluster.Vttablet, sql, state string, timeout time.Duration) {
 	require.Eventually(t, func() bool {
@@ -855,7 +860,7 @@ func WaitForQueryWithStateInProcesslist(ctx context.Context, t *testing.T, table
 			if strings.EqualFold(row[0].ToString(), "Query") {
 				continue
 			}
-			if strings.EqualFold(row[1].ToString(), state) && strings.EqualFold(row[2].ToString(), sql) {
+			if strings.EqualFold(row[1].ToString(), state) && strings.EqualFold(optimizerHintRE.ReplaceAllString(row[2].ToString(), ""), sql) {
 				return true
 			}
 		}

go/test/endtoend/schemadiff/vrepl/schemadiff_vrepl_suite_test.go

@@ -91,6 +91,12 @@ func TestMain(m *testing.M) {
 			"--migration-check-interval", "5s",
 		}
 
+		// The suite's test tables use zero-date defaults, so vtgate sessions
+		// must run with a sql_mode that allows them.
+		clusterInstance.VtGateExtraArgs = []string{
+			"--sql-mode", sqlModeAllowsZeroDate,
+		}
+
 		if err := clusterInstance.StartTopo(); err != nil {
 			return 1, err
 		}