Skip to content

Commit 3d4636b

Browse files
committed
feat: add HandleEmtpyInListsPlugin. (#925)
* feat: empty where in plugin * test: add new tests * chore: remove unneccesary typeguards * fix: change to binary operator node * test: update tests to do both in and not in * test: for having * chore: rm test * test: nullable tests * chore: nit * chore: condense suite * chore: db config override * chore: extra console log * chore: empty arr plugin docs * HandleEmptyInListsPlugin initial commit. Co-authored-by: Austin Woon Quan <[email protected]> --------- Co-authored-by: Austin Woon <[email protected]> Co-authored-by: igalklebanov <[email protected]> remove only.
1 parent 02c8a56 commit 3d4636b

9 files changed

+740
-14
lines changed

site/docs/plugins.md

+5-1
Original file line numberDiff line numberDiff line change
@@ -20,4 +20,8 @@ A plugin that converts snake_case identifiers in the database into camelCase in
2020

2121
### Deduplicate joins plugin
2222

23-
Plugin that removes duplicate joins from queries. You can read more about it in the [examples](/docs/recipes/deduplicate-joins) section or check the [API docs](https://kysely-org.github.io/kysely-apidoc/classes/DeduplicateJoinsPlugin.html).
23+
A plugin that removes duplicate joins from queries. You can read more about it in the [examples](/docs/recipes/deduplicate-joins) section or check the [API docs](https://kysely-org.github.io/kysely-apidoc/classes/DeduplicateJoinsPlugin.html).
24+
25+
### Handle `in ()` and `not in ()` plugin
26+
27+
A plugin that allows handling `in ()` and `not in ()` with a chosen strategy. [Learn more](https://kysely-org.github.io/kysely-apidoc/classes/HandleEmptyWhereInListsPlugin.html).

src/index.ts

+2
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,8 @@ export * from './plugin/camel-case/camel-case-plugin.js'
110110
export * from './plugin/deduplicate-joins/deduplicate-joins-plugin.js'
111111
export * from './plugin/with-schema/with-schema-plugin.js'
112112
export * from './plugin/parse-json-results/parse-json-results-plugin.js'
113+
export * from './plugin/handle-empty-in-lists/handle-empty-in-lists-plugin.js'
114+
export * from './plugin/handle-empty-in-lists/handle-empty-in-lists.js'
113115

114116
export * from './operation-node/add-column-node.js'
115117
export * from './operation-node/add-constraint-node.js'
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
import { QueryResult } from '../../driver/database-connection.js'
2+
import { RootOperationNode } from '../../query-compiler/query-compiler.js'
3+
import {
4+
KyselyPlugin,
5+
PluginTransformQueryArgs,
6+
PluginTransformResultArgs,
7+
} from '../kysely-plugin.js'
8+
import { UnknownRow } from '../../util/type-utils.js'
9+
import { HandleEmptyInListsTransformer } from './handle-empty-in-lists-transformer.js'
10+
import { HandleEmptyInListsOptions } from './handle-empty-in-lists.js'
11+
12+
/**
13+
* A plugin that allows handling `in ()` and `not in ()` expressions.
14+
*
15+
* These expressions are invalid SQL syntax for many databases, and result in runtime
16+
* database errors.
17+
*
18+
* The workarounds used by other libraries always involve modifying the query under
19+
* the hood, which is not aligned with Kysely's philosophy of WYSIWYG. We recommend manually checking
20+
* for empty arrays before passing them as arguments to `in` and `not in` expressions
21+
* instead, but understand that this can be cumbersome. Hence we're going with an
22+
* opt-in approach where you can choose if and how to handle these cases. We do
23+
* not want to make this the default behavior, as it can lead to unexpected behavior.
24+
* Use it at your own risk. Test it. Make sure it works as expected for you.
25+
*
26+
* Using this plugin also allows you to throw an error (thus avoiding unnecessary
27+
* requests to the database) or print a warning in these cases.
28+
*
29+
* ### Examples
30+
*
31+
* The following strategy replaces the `in`/`not in` expression with a noncontingent
32+
* expression. A contradiction (falsy) `1 = 0` for `in`, and a tautology (truthy) `1 = 1` for `not in`),
33+
* similarily to how {@link https://github.com/knex/knex/blob/176151d8048b2a7feeb89a3d649a5580786d4f4e/docs/src/guide/query-builder.md#L1763 | Knex.js},
34+
* {@link https://github.com/prisma/prisma-engines/blob/99168c54187178484dae45d9478aa40cfd1866d2/quaint/src/visitor.rs#L804-L823 | PrismaORM},
35+
* {@link https://github.com/laravel/framework/blob/8.x/src/Illuminate/Database/Query/Grammars/Grammar.php#L284-L291 | Laravel},
36+
* {@link https://docs.sqlalchemy.org/en/13/core/engines.html#sqlalchemy.create_engine.params.empty_in_strategy | SQLAlchemy}
37+
* handle this.
38+
*
39+
* ```ts
40+
* import Sqlite from 'better-sqlite3'
41+
* import {
42+
* HandleEmptyInListsPlugin,
43+
* Kysely,
44+
* replaceWithNoncontingentExpression,
45+
* SqliteDialect,
46+
* } from 'kysely'
47+
* import type { Database } from 'type-editor' // imaginary module
48+
*
49+
* const db = new Kysely<Database>({
50+
* dialect: new SqliteDialect({
51+
* database: new Sqlite(':memory:'),
52+
* }),
53+
* plugins: [
54+
* new HandleEmptyInListsPlugin({
55+
* strategy: replaceWithNoncontingentExpression
56+
* })
57+
* ],
58+
* })
59+
*
60+
* const results = await db
61+
* .selectFrom('person')
62+
* .where('id', 'in', [])
63+
* .where('first_name', 'not in', [])
64+
* .selectAll()
65+
* .execute()
66+
* ```
67+
*
68+
* The generated SQL (SQLite):
69+
*
70+
* ```sql
71+
* select * from "person" where 1 = 0 and 1 = 1
72+
* ```
73+
*
74+
* The following strategy does the following:
75+
*
76+
* When `in`, pushes a `null` value into the empty list resulting in `in (null)`,
77+
* similiarly to how {@link https://github.com/typeorm/typeorm/blob/0280cdc451c35ef73c830eb1191c95d34f6ce06e/src/query-builder/QueryBuilder.ts#L919-L922 | TypeORM}
78+
* and {@link https://github.com/sequelize/sequelize/blob/0f2891c6897e12bf9bf56df344aae5b698f58c7d/packages/core/src/abstract-dialect/where-sql-builder.ts#L368-L379 | Sequelize}
79+
* handle `in ()`. `in (null)` is logically the equivalent of `= null`, which returns
80+
* `null`, which is a falsy expression in most SQL databases. We recommend NOT
81+
* using this strategy if you plan to use `in` in `select`, `returning`, or `output`
82+
* clauses, as the return type differs from the `SqlBool` default type for comparisons.
83+
*
84+
* When `not in`, casts the left operand as `char` and pushes a unique value into
85+
* the empty list resulting in `cast({{lhs}} as char) not in ({{VALUE}})`. Casting
86+
* is required to avoid database errors with non-string values.
87+
*
88+
* ```ts
89+
* import Sqlite from 'better-sqlite3'
90+
* import {
91+
* HandleEmptyInListsPlugin,
92+
* Kysely,
93+
* pushValueIntoList,
94+
* SqliteDialect
95+
* } from 'kysely'
96+
* import type { Database } from 'type-editor' // imaginary module
97+
*
98+
* const db = new Kysely<Database>({
99+
* dialect: new SqliteDialect({
100+
* database: new Sqlite(':memory:'),
101+
* }),
102+
* plugins: [
103+
* new HandleEmptyInListsPlugin({
104+
* strategy: pushValueIntoList('__kysely_no_values_were_provided__') // choose a unique value for not in. has to be something with zero chance being in the data.
105+
* })
106+
* ],
107+
* })
108+
*
109+
* const results = await db
110+
* .selectFrom('person')
111+
* .where('id', 'in', [])
112+
* .where('first_name', 'not in', [])
113+
* .selectAll()
114+
* .execute()
115+
* ```
116+
*
117+
* The generated SQL (SQLite):
118+
*
119+
* ```sql
120+
* select * from "person" where "id" in (null) and cast("first_name" as char) not in ('__kysely_no_values_were_provided__')
121+
* ```
122+
*
123+
* The following custom strategy throws an error when an empty list is encountered
124+
* to avoid unnecessary requests to the database:
125+
*
126+
* ```ts
127+
* import Sqlite from 'better-sqlite3'
128+
* import {
129+
* HandleEmptyInListsPlugin,
130+
* Kysely,
131+
* SqliteDialect
132+
* } from 'kysely'
133+
* import type { Database } from 'type-editor' // imaginary module
134+
*
135+
* const db = new Kysely<Database>({
136+
* dialect: new SqliteDialect({
137+
* database: new Sqlite(':memory:'),
138+
* }),
139+
* plugins: [
140+
* new HandleEmptyInListsPlugin({
141+
* strategy: () => {
142+
* throw new Error('Empty in/not-in is not allowed')
143+
* }
144+
* })
145+
* ],
146+
* })
147+
*
148+
* const results = await db
149+
* .selectFrom('person')
150+
* .where('id', 'in', [])
151+
* .selectAll()
152+
* .execute() // throws an error with 'Empty in/not-in is not allowed' message!
153+
* ```
154+
*/
155+
export class HandleEmptyInListsPlugin implements KyselyPlugin {
156+
readonly #transformer: HandleEmptyInListsTransformer
157+
158+
constructor(readonly opt: HandleEmptyInListsOptions) {
159+
this.#transformer = new HandleEmptyInListsTransformer(opt.strategy)
160+
}
161+
162+
transformQuery(args: PluginTransformQueryArgs): RootOperationNode {
163+
return this.#transformer.transformNode(args.node)
164+
}
165+
166+
async transformResult(
167+
args: PluginTransformResultArgs,
168+
): Promise<QueryResult<UnknownRow>> {
169+
return args.result
170+
}
171+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import { BinaryOperationNode } from '../../operation-node/binary-operation-node.js'
2+
import { OperationNodeTransformer } from '../../operation-node/operation-node-transformer.js'
3+
import { PrimitiveValueListNode } from '../../operation-node/primitive-value-list-node.js'
4+
import { OperatorNode } from '../../operation-node/operator-node.js'
5+
import {
6+
EmptyInListNode,
7+
EmptyInListsStrategy,
8+
} from './handle-empty-in-lists.js'
9+
import { ValueListNode } from '../../operation-node/value-list-node.js'
10+
11+
export class HandleEmptyInListsTransformer extends OperationNodeTransformer {
12+
readonly #strategy: EmptyInListsStrategy
13+
14+
constructor(strategy: EmptyInListsStrategy) {
15+
super()
16+
this.#strategy = strategy
17+
}
18+
19+
protected transformBinaryOperation(
20+
node: BinaryOperationNode,
21+
): BinaryOperationNode {
22+
if (this.#isEmptyInListNode(node)) {
23+
return this.#strategy(node)
24+
}
25+
26+
return node
27+
}
28+
29+
#isEmptyInListNode(node: BinaryOperationNode): node is EmptyInListNode {
30+
const { operator, rightOperand } = node
31+
32+
return (
33+
(PrimitiveValueListNode.is(rightOperand) ||
34+
ValueListNode.is(rightOperand)) &&
35+
rightOperand.values.length === 0 &&
36+
OperatorNode.is(operator) &&
37+
(operator.operator === 'in' || operator.operator === 'not in')
38+
)
39+
}
40+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
import { BinaryOperationNode } from '../../operation-node/binary-operation-node.js'
2+
import { CastNode } from '../../operation-node/cast-node.js'
3+
import { DataTypeNode } from '../../operation-node/data-type-node.js'
4+
import { OperatorNode } from '../../operation-node/operator-node.js'
5+
import { ParensNode } from '../../operation-node/parens-node.js'
6+
import { PrimitiveValueListNode } from '../../operation-node/primitive-value-list-node.js'
7+
import { ValueListNode } from '../../operation-node/value-list-node.js'
8+
import { ValueNode } from '../../operation-node/value-node.js'
9+
import { freeze } from '../../util/object-utils.js'
10+
11+
export interface HandleEmptyInListsOptions {
12+
/**
13+
* The strategy to use when handling `in ()` and `not in ()`.
14+
*
15+
* See {@link HandleEmptyInListsPlugin} for examples.
16+
*/
17+
strategy: EmptyInListsStrategy
18+
}
19+
20+
export type EmptyInListNode = BinaryOperationNode & {
21+
operator: OperatorNode & {
22+
operator: 'in' | 'not in'
23+
}
24+
rightOperand: (ValueListNode | PrimitiveValueListNode) & {
25+
values: Readonly<[]>
26+
}
27+
}
28+
29+
export type EmptyInListsStrategy = (
30+
node: EmptyInListNode,
31+
) => BinaryOperationNode
32+
33+
let contradiction: BinaryOperationNode
34+
let eq: OperatorNode
35+
let one: ValueNode
36+
let tautology: BinaryOperationNode
37+
/**
38+
* Replaces the `in`/`not in` expression with a noncontingent expression (always true or always
39+
* false) depending on the original operator.
40+
*
41+
* This is how Knex.js, PrismaORM, Laravel, and SQLAlchemy handle `in ()` and `not in ()`.
42+
*
43+
* See {@link pushValueIntoList} for an alternative strategy.
44+
*/
45+
export function replaceWithNoncontingentExpression(
46+
node: EmptyInListNode,
47+
): BinaryOperationNode {
48+
const _one = (one ||= ValueNode.createImmediate(1))
49+
const _eq = (eq ||= OperatorNode.create('='))
50+
51+
if (node.operator.operator === 'in') {
52+
return (contradiction ||= BinaryOperationNode.create(
53+
_one,
54+
_eq,
55+
ValueNode.createImmediate(0),
56+
))
57+
}
58+
59+
return (tautology ||= BinaryOperationNode.create(_one, _eq, _one))
60+
}
61+
62+
let char: DataTypeNode
63+
let listNull: ValueListNode
64+
let listVal: ValueListNode
65+
/**
66+
* When `in`, pushes a `null` value into the list resulting in `in (null)`. This
67+
* is how TypeORM and Sequelize handle `in ()`. `in (null)` is logically the equivalent
68+
* of `= null`, which returns `null`, which is a falsy expression in most SQL databases.
69+
* We recommend NOT using this strategy if you plan to use `in` in `select`, `returning`,
70+
* or `output` clauses, as the return type differs from the `SqlBool` default type.
71+
*
72+
* When `not in`, casts the left operand as `char` and pushes a literal value into
73+
* the list resulting in `cast({{lhs}} as char) not in ({{VALUE}})`. Casting
74+
* is required to avoid database errors with non-string columns.
75+
*
76+
* See {@link replaceWithNoncontingentExpression} for an alternative strategy.
77+
*/
78+
export function pushValueIntoList(
79+
uniqueNotInLiteral: '__kysely_no_values_were_provided__' | (string & {}),
80+
): EmptyInListsStrategy {
81+
return function pushValueIntoList(node) {
82+
if (node.operator.operator === 'in') {
83+
return freeze({
84+
...node,
85+
rightOperand: (listNull ||= ValueListNode.create([
86+
ValueNode.createImmediate(null),
87+
])),
88+
})
89+
}
90+
91+
return freeze({
92+
...node,
93+
leftOperand: CastNode.create(
94+
node.leftOperand,
95+
(char ||= DataTypeNode.create('char')),
96+
),
97+
rightOperand: (listVal ||= ValueListNode.create([
98+
ValueNode.createImmediate(uniqueNotInLiteral),
99+
])),
100+
})
101+
}
102+
}

test/node/src/controlled-transaction.test.ts

+6-4
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,12 @@ for (const dialect of DIALECTS) {
4444
>
4545

4646
before(async function () {
47-
ctx = await initTest(this, dialect, (event) => {
48-
if (event.level === 'query') {
49-
executedQueries.push(event.query)
50-
}
47+
ctx = await initTest(this, dialect, {
48+
log(event) {
49+
if (event.level === 'query') {
50+
executedQueries.push(event.query)
51+
}
52+
},
5153
})
5254
})
5355

0 commit comments

Comments
 (0)