Repository navigation
Conversation
|
@dvd233 is attempting to deploy a commit to the Evidence Team on Vercel. A member of the Team first needs to authorize it. |
tojacob03
left a comment
There was a problem hiding this comment.
I ran this on macOS (arm64) with the native chdb build. The description says the ClickHouse assertions only ran through a temporary adapter on Windows, so I wanted to check them against the real one.
- Rebased onto current
main(c2dd7c2): the cherry-pick applies cleanly. table.test.ts: 24/24 pass, including the 4 new tests.- With
build-table-sql.tsreverted tomain, "applies a column sort before limiting grouped rows" fails, so the regression test does catch the bug. - Full
coresuite: 3990 passed, 3 skipped.
Two questions:
1. Which column wins when several have sort?
For the initial client-side sort, Table.svelte uses the first column in columnMeta that has sort. This PR pushes down the first column with sort that can be pushed down. The two differ when the first sorted column is excluded (sparkline, derived, pivot) and a later one isn't. For example:
cols(dim('category'), measure('sum(total_sales)'), measure('count(*)'))
// [1]: sort = 'desc', viz = 'sparkline'
// [2]: sort = 'asc'
// limit: 5This gives order = '"count" asc', so the top 5 rows are picked by count but displayed sorted by the sparkline column. Would it be safer to push the sort down only when the column the client sorts by can itself be pushed down, and leave order undefined otherwise?
2. Paging together with limit (probably out of scope)
With limit + page_size/offset, the generated SQL is:
SELECT * FROM (... ORDER BY "sum_total_sales" desc LIMIT 5) AS evidence_paged LIMIT 2 OFFSET 2The outer query has no ORDER BY, and most engines don't promise to keep a subquery's order, so pages could overlap or skip rows. This comes from the existing wrapper in sql-options.ts (it happens with an explicit order too), so it isn't caused by this PR. I'm only mentioning it in case it's worth a follow-up.
|
/upstream |
|
Imported for internal review, this PR will be updated when it merges. |
Summary
sortmetadata whenlimitis setorderProblem
Table
limitis applied in SQL, while declarative column sorting normally happens after the query. When a grouped result has more thanlimitrows, the warehouse can therefore truncate an arbitrary subset before the client sorts it, silently excluding the actual top rows.This addresses the top-N ordering part of #3334. It intentionally does not change the existing subtotal suppression under
limitor the separate null-dimension rendering behavior described in that issue.Testing
mainimplementation and passes with this changenode_modules/.bin/svelte-check --threshold error --tsconfig core/tsconfig.json— 0 errorsThe npm
chdbpackage does not provide its native binary on Windows, so the local Vitest run used a temporary, uncommitted transport adapter to execute the existing ClickHouse assertions againstplay.clickhouse.com. The committed test and test utilities are unchanged in that respect; normal Linux CI will continue to use the repository's nativechdbpath.