Index that uses B+ tree

[kevin-dp]

Currently, we have an OrderedIndex that internally keeps an ordered array of indexed values. While this allows for efficient range queries, it introduces significant overhead on inserts and deletes because inserting/deleting in the array is in linear time.

This PR introduces a BTreeIndex which internally keeps a B+ tree of indexed values. This still allows for efficient range queries but also makes insertions/deletions efficient (logarithmic time). I think it makes the OrderedIndex completely obsolete and we should remove it in favor of BTreeIndex.

Note that the B+ tree is only needed for range queries but some collections might never use range queries. So it would be good to introduce also a HashIndex in a follow up PR which only keeps a map of indexed values to PKs. That will reduce bookkeeping overhead on inserts/deletes further from logarithmic to constant time.

[changeset-bot]

🦋 Changeset detected

Latest commit: 0b2875e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 7 packages

Name	Type
@tanstack/db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch
@tanstack/db-example-react-todo	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

@tanstack/db-example-react-todo

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@302

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@302

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@302

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@302

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@302

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@302

commit: 0b2875e

[github-actions]

Size Change: +5.49 kB (+10.44%) ⚠️

Total Size: 58.1 kB

Filename	Size	Change
./packages/db/dist/esm/collection.js	9.7 kB	+4 B (+0.04%)
./packages/db/dist/esm/index.js	1.52 kB	-1 B (-0.07%)
./packages/db/dist/esm/indexes/auto-index.js	689 B	-1 B (-0.14%)
./packages/db/dist/esm/indexes/ordered-index.js	0 B	-1.65 kB (removed)	🏆
./packages/db/dist/esm/utils/array-utils.js	0 B	-262 B (removed)	🏆
./packages/db/dist/esm/indexes/btree-index.js	1.47 kB	+1.47 kB (new file)	🆕
./packages/db/dist/esm/utils/btree.js	5.93 kB	+5.93 kB (new file)	🆕

ℹ️ View Unchanged

Filename	Size
./packages/db/dist/esm/change-events.js	1.12 kB
./packages/db/dist/esm/deferred.js	230 B
./packages/db/dist/esm/errors.js	2.98 kB
./packages/db/dist/esm/indexes/base-index.js	605 B
./packages/db/dist/esm/indexes/lazy-index.js	1.25 kB
./packages/db/dist/esm/local-only.js	827 B
./packages/db/dist/esm/local-storage.js	2.03 kB
./packages/db/dist/esm/optimistic-action.js	294 B
./packages/db/dist/esm/proxy.js	4.19 kB
./packages/db/dist/esm/query/builder/functions.js	575 B
./packages/db/dist/esm/query/builder/index.js	3.67 kB
./packages/db/dist/esm/query/builder/ref-proxy.js	890 B
./packages/db/dist/esm/query/compiler/evaluators.js	1.48 kB
./packages/db/dist/esm/query/compiler/expressions.js	631 B
./packages/db/dist/esm/query/compiler/group-by.js	2.04 kB
./packages/db/dist/esm/query/compiler/index.js	1.74 kB
./packages/db/dist/esm/query/compiler/joins.js	1.56 kB
./packages/db/dist/esm/query/compiler/order-by.js	705 B
./packages/db/dist/esm/query/compiler/select.js	657 B
./packages/db/dist/esm/query/ir.js	318 B
./packages/db/dist/esm/query/live-query-collection.js	2.45 kB
./packages/db/dist/esm/query/optimizer.js	2.44 kB
./packages/db/dist/esm/SortedMap.js	1.24 kB
./packages/db/dist/esm/transactions.js	2.28 kB
./packages/db/dist/esm/utils.js	419 B
./packages/db/dist/esm/utils/comparison.js	539 B
./packages/db/dist/esm/utils/index-optimization.js	1.62 kB

_{compressed-size-action::db-package-size}

[github-actions]

Size Change: 0 B

Total Size: 1.05 kB

ℹ️ View Unchanged

Filename	Size
./packages/react-db/dist/esm/index.js	152 B
./packages/react-db/dist/esm/useLiveQuery.js	902 B

_{compressed-size-action::react-db-package-size}

[kevin-dp]

Some quick microbenchmarks confirm that the B+ tree index is faster for inserts/updates/deletes and similar for range queries:

==== INDEX BUILDING ====
OrderedIndex: 2004.6950840000002ms for 100000 users
BTreeIndex: 63.54179199999999ms for 100000 users

==== INDEX INSERTION ====
OrderedIndex 0.056916999999884865ms per insert
BTreeIndex 0.004040999999688211ms per insert

==== INDEX DELETION ====
OrderedIndex 0.07920799999919836ms per delete
BTreeIndex 0.01920799999970768ms per delete

==== INDEX UPDATE ====
OrderedIndex 0.09737500000028376ms per update
BTreeIndex 0.010333000000173342ms per update

==== INDEX RANGE QUERY ====
OrderedIndex range query: 10.354792000000089ms per query
BTreeIndex range query: 11.601208000000042ms per query

All operations were performed on an index containing 100K rows.