Change the way types for prepared statement parameters are calculated - #1407
Change the way types for prepared statement parameters are calculated#1407sidorares wants to merge 7 commits into
Conversation
…n inferring from input variables
|
ref #1239 |
|
@vlasky when you have time can you try this brach to see if it fixes It's a potentially backwards incompatible change in some scenarios, I'm thinking to release it as a major version bump. Maybe alpha release initially |
|
Hello @sidorares |
|
Any updates on this? |
|
@mlucic unfortunately no progress here (but its quite high in my priority) The way I want to implement this currently is different from this PR though. The plan is:
const mediumintparam = mysql.TypedParameter.MEDIUMINT(123);
const floatparam = mysql.TypedParameter.FLOAT(123);
conn.execute('select ?+? as sum', [mediumintparam, floatparam]);when used via type helper the type sent with COM_EXECUTE is always what you specify
step 1) can be done now and would help solving current edge cases with a bit of manually added typings ( as in mysql types metadata in users code, not typescript ) after that we can add step 2 to make default map work with no surprises most of the time. Mysql is quite capable of casting back to a correct value on a server side, in earlier versions of this driver all example table:
I'm starting to think simple "always send (parameter.toString()) as VAR_STRING" unless the type is explicitly specified by user" might be actually the best behaviour |
Two changes to how a bind parameter picks its MySQL type.
TypedParameter carries the type alongside the value and is accepted
anywhere execute() takes one:
conn.execute('SELECT * FROM t LIMIT ?', [mysql.TypedParameter.BIGINT(10)])
It expresses what a JavaScript value cannot: integer width, signedness, and
binary versus text. The declared type is written into COM_STMT_EXECUTE, so
the unsigned flag is now sent rather than hardcoded to zero, and a typed
null keeps its declared type instead of collapsing to MYSQL_TYPE_NULL.
Values that do not fit are rejected at the call site, including numbers that
already lost precision. Types no server accepts as a bind type (MEDIUMINT,
YEAR, ENUM, SET, and JSON on MariaDB) travel as the nearest accepted type.
For untyped values the driver now adopts the type COM_STMT_PREPARE reports,
but only when that type is an integer type and the value is already an
integer that fits it. That is enough to make 'LIMIT ?' accept a number, and
to stop a cached statement being fixed as DOUBLE by its first execution.
Every other combination keeps the type inferred from JavaScript, so a bare
'SELECT ?' still round-trips a number as a number and server-side string
coercions are left alone. MariaDB reports MYSQL_TYPE_NULL for every
parameter, which fails the same gate, so nothing changes there.
Closes #1239
Refs #1407, #548, #446, #1483, #1173
…4488) * test: reproduce integer parameters MySQL refuses or silently rounds Every JS number is bound as MYSQL_TYPE_DOUBLE. MySQL rejects a DOUBLE where the statement needs an integer, so `LIMIT ?` and `LIMIT ? OFFSET ?` fail with ER_WRONG_ARGUMENTS for the most natural call a user can write. Since MySQL 8.0.22 a prepared statement is no longer re-prepared when a later execution changes a parameter type, so the first type a cached statement sees decides how every later execution is read. Binding a number once fixes the position as DOUBLE, and an exact BIGINT string bound afterwards is silently rounded through it. These tests fail on MySQL 8.3 and 9.7 and pass on MariaDB, which accepts a DOUBLE for LIMIT and re-optimises each execution. Refs #1239, #1407 * feat: typed parameters, and adopt integer types the server reports Two changes to how a bind parameter picks its MySQL type. TypedParameter carries the type alongside the value and is accepted anywhere execute() takes one: conn.execute('SELECT * FROM t LIMIT ?', [mysql.TypedParameter.BIGINT(10)]) It expresses what a JavaScript value cannot: integer width, signedness, and binary versus text. The declared type is written into COM_STMT_EXECUTE, so the unsigned flag is now sent rather than hardcoded to zero, and a typed null keeps its declared type instead of collapsing to MYSQL_TYPE_NULL. Values that do not fit are rejected at the call site, including numbers that already lost precision. Types no server accepts as a bind type (MEDIUMINT, YEAR, ENUM, SET, and JSON on MariaDB) travel as the nearest accepted type. For untyped values the driver now adopts the type COM_STMT_PREPARE reports, but only when that type is an integer type and the value is already an integer that fits it. That is enough to make 'LIMIT ?' accept a number, and to stop a cached statement being fixed as DOUBLE by its first execution. Every other combination keeps the type inferred from JavaScript, so a bare 'SELECT ?' still round-trips a number as a number and server-side string coercions are left alone. MariaDB reports MYSQL_TYPE_NULL for every parameter, which fails the same gate, so nothing changes there. Closes #1239 Refs #1407, #548, #446, #1483, #1173 * fix: handle servers that report no parameter types, and cover the encoders MySQL 5.7 answers VAR_STRING with a zero length for every parameter, the same non-answer MariaDB gives as MYSQL_TYPE_NULL. Neither reaches the integer gate, so nothing is adopted on either and no version check is needed, but the test asserting what the server reports assumed every MySQL resolves types. It now detects that capability instead of naming versions. Also formats the new docs page with the website's own prettier config, adds the 5.7 and MariaDB rows to it, and covers the remaining parameter encoders (FLOAT, DATE, TIMESTAMP, TIME in all four forms, length coded text, and the unsupported-type path), taking lib/packets/typed_parameter.js from 92% to 99% statement coverage.
Currently types are inferred based on what is in the input (Date -> mysql date, js number -> mysql double, everything else - string ). This used to work ok until mysql server version 8.0.22
Calling
execute('SELECT * from foo limit ?' , [1])results inIncorrect arguments to mysqld_stmt_executeerror. The type of the parameter server expects ( LONGINT ) is incompatible to what is actually sent ( DOUBLE ).The fix is to use parameters column definitions returned from the previous
prepare()call and use types from there instead of inferring type from the provided input