Skip to content

Change the way types for prepared statement parameters are calculated - #1407

Open
sidorares wants to merge 7 commits into
masterfrom
feature/bind-parameter-types-hints
Open

Change the way types for prepared statement parameters are calculated#1407
sidorares wants to merge 7 commits into
masterfrom
feature/bind-parameter-types-hints

Conversation

@sidorares

@sidorares sidorares commented Oct 11, 2021

Copy link
Copy Markdown
Owner

Currently types are inferred based on what is in the input (Date -> mysql date, js number -> mysql double, everything else - string ). This used to work ok until mysql server version 8.0.22

Calling execute('SELECT * from foo limit ?' , [1]) results in Incorrect arguments to mysqld_stmt_execute error. The type of the parameter server expects ( LONGINT ) is incompatible to what is actually sent ( DOUBLE ).

The fix is to use parameters column definitions returned from the previous prepare() call and use types from there instead of inferring type from the provided input

@sidorares

Copy link
Copy Markdown
Owner Author

ref #1239

@sidorares

Copy link
Copy Markdown
Owner Author

@vlasky when you have time can you try this brach to see if it fixes Incorrect arguments to mysqld_stmt_execute error for you?

It's a potentially backwards incompatible change in some scenarios, I'm thinking to release it as a major version bump. Maybe alpha release initially

@kimf

kimf commented Apr 27, 2022

Copy link
Copy Markdown

Hello @sidorares
For what it's worth, I had the exact same issue as mentioned in #1239 and tried this branch with great success.

@mlucic

mlucic commented Nov 23, 2022

Copy link
Copy Markdown

Any updates on this?

@sidorares

sidorares commented Nov 24, 2022

Copy link
Copy Markdown
Owner Author

@mlucic unfortunately no progress here (but its quite high in my priority)

The way I want to implement this currently is different from this PR though.

The plan is:

  1. Introduce a "typed parameter" helpers which would hold the value you pass together with type information. Not sure about the exact api, but it might look like this:
const mediumintparam = mysql.TypedParameter.MEDIUMINT(123);
const floatparam =  mysql.TypedParameter.FLOAT(123);

conn.execute('select ?+? as sum', [mediumintparam, floatparam]);

when used via type helper the type sent with COM_EXECUTE is always what you specify

  1. when untyped js value is passed we have 2 options: infer from current value ( string / date / number / null / non-null object ) or use response from COM_PREPARE. The problem "type hint from the server" is that its not always accurate ( for example, there are scenarios when the server has no way of telling parameter type - select ?+? as sum ). I'll need to think when this hint might be useful and when not ( any feedback is appreciated! ), ideally we need a matrix where there is js type on one axis, COM_EXECUTE type hint on another axis mapped to what the driver will actually use for type

step 1) can be done now and would help solving current edge cases with a bit of manually added typings ( as in mysql types metadata in users code, not typescript )

after that we can add step 2 to make default map work with no surprises most of the time. Mysql is quite capable of casting back to a correct value on a server side, in earlier versions of this driver all .execute() parameters were converted to a string ( sometimes its even more efficient, "1" takes 2 bytes over the wire where double precision number is 8 )

example table:

Mysql type hint \ JS type string number bigint Date
LONGLONG VAR_STRING VAR_STRING VAR_STRING DATETIME
VAR_STRING VAR_STRING DOUBLE VAR_STRING DATETIME

I'm starting to think simple "always send (parameter.toString()) as VAR_STRING" unless the type is explicitly specified by user" might be actually the best behaviour

@wellwelwel wellwelwel added the needs rebase For internal organization purpose label Mar 5, 2025
sidorares added a commit that referenced this pull request Aug 19, 2026
Two changes to how a bind parameter picks its MySQL type.

TypedParameter carries the type alongside the value and is accepted
anywhere execute() takes one:

  conn.execute('SELECT * FROM t LIMIT ?', [mysql.TypedParameter.BIGINT(10)])

It expresses what a JavaScript value cannot: integer width, signedness, and
binary versus text. The declared type is written into COM_STMT_EXECUTE, so
the unsigned flag is now sent rather than hardcoded to zero, and a typed
null keeps its declared type instead of collapsing to MYSQL_TYPE_NULL.
Values that do not fit are rejected at the call site, including numbers that
already lost precision. Types no server accepts as a bind type (MEDIUMINT,
YEAR, ENUM, SET, and JSON on MariaDB) travel as the nearest accepted type.

For untyped values the driver now adopts the type COM_STMT_PREPARE reports,
but only when that type is an integer type and the value is already an
integer that fits it. That is enough to make 'LIMIT ?' accept a number, and
to stop a cached statement being fixed as DOUBLE by its first execution.
Every other combination keeps the type inferred from JavaScript, so a bare
'SELECT ?' still round-trips a number as a number and server-side string
coercions are left alone. MariaDB reports MYSQL_TYPE_NULL for every
parameter, which fails the same gate, so nothing changes there.

Closes #1239

Refs #1407, #548, #446, #1483, #1173
sidorares added a commit that referenced this pull request Aug 23, 2026
…4488)

* test: reproduce integer parameters MySQL refuses or silently rounds

Every JS number is bound as MYSQL_TYPE_DOUBLE. MySQL rejects a DOUBLE where
the statement needs an integer, so `LIMIT ?` and `LIMIT ? OFFSET ?` fail
with ER_WRONG_ARGUMENTS for the most natural call a user can write.

Since MySQL 8.0.22 a prepared statement is no longer re-prepared when a
later execution changes a parameter type, so the first type a cached
statement sees decides how every later execution is read. Binding a number
once fixes the position as DOUBLE, and an exact BIGINT string bound
afterwards is silently rounded through it.

These tests fail on MySQL 8.3 and 9.7 and pass on MariaDB, which accepts a
DOUBLE for LIMIT and re-optimises each execution.

Refs #1239, #1407

* feat: typed parameters, and adopt integer types the server reports

Two changes to how a bind parameter picks its MySQL type.

TypedParameter carries the type alongside the value and is accepted
anywhere execute() takes one:

  conn.execute('SELECT * FROM t LIMIT ?', [mysql.TypedParameter.BIGINT(10)])

It expresses what a JavaScript value cannot: integer width, signedness, and
binary versus text. The declared type is written into COM_STMT_EXECUTE, so
the unsigned flag is now sent rather than hardcoded to zero, and a typed
null keeps its declared type instead of collapsing to MYSQL_TYPE_NULL.
Values that do not fit are rejected at the call site, including numbers that
already lost precision. Types no server accepts as a bind type (MEDIUMINT,
YEAR, ENUM, SET, and JSON on MariaDB) travel as the nearest accepted type.

For untyped values the driver now adopts the type COM_STMT_PREPARE reports,
but only when that type is an integer type and the value is already an
integer that fits it. That is enough to make 'LIMIT ?' accept a number, and
to stop a cached statement being fixed as DOUBLE by its first execution.
Every other combination keeps the type inferred from JavaScript, so a bare
'SELECT ?' still round-trips a number as a number and server-side string
coercions are left alone. MariaDB reports MYSQL_TYPE_NULL for every
parameter, which fails the same gate, so nothing changes there.

Closes #1239

Refs #1407, #548, #446, #1483, #1173

* fix: handle servers that report no parameter types, and cover the encoders

MySQL 5.7 answers VAR_STRING with a zero length for every parameter, the
same non-answer MariaDB gives as MYSQL_TYPE_NULL. Neither reaches the
integer gate, so nothing is adopted on either and no version check is
needed, but the test asserting what the server reports assumed every MySQL
resolves types. It now detects that capability instead of naming versions.

Also formats the new docs page with the website's own prettier config, adds
the 5.7 and MariaDB rows to it, and covers the remaining parameter encoders
(FLOAT, DATE, TIMESTAMP, TIME in all four forms, length coded text, and the
unsupported-type path), taking lib/packets/typed_parameter.js from 92% to
99% statement coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs rebase For internal organization purpose

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants