Commit e1dd0bf
feat(011): Model Management API — validate, refresh, delete, single-cube, diff, rollback (#45)
* fix(cubejs): match checkSqlAuth callback signature to Cube.js v1.6
Cube.js v1.6 invokes checkSqlAuth as (request, user, password) — three
positional args — see
@cubejs-backend/api-gateway/dist/src/sql-server.js:291,105.
Our implementation declared (_, user) and did:
password = typeof user === "string" ? user : user?.password
username = typeof user === "string" ? _ : user?.username
With the v1.6 wire server, user arrives as a plain string (the Postgres
username), so the code took the username as the password AND used the
request metadata object as the username. findSqlCredentials then
received the {protocol, method, apiType} object, and Hasura rejected
the query with:
parsing Text failed, expected String, but encountered Object
path: $.selectionSet.sql_credentials.args.where.username._eq
Every SQL API login failed before any password comparison ran
(reproduced via `psql -U <valid> -h <cubejs>` → 28P01).
Fix: match the documented v1.6 signature and keep a defensive branch
for the legacy object-shape call. Also reject non-string username
early so the Hasura GraphQL layer cannot receive a non-string variable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cubejs): propagate teamProperties/memberProperties for SQL API logins
queryRewrite's rule-based row filtering relies on
`securityContext.userScope.teamProperties` and `.memberProperties` to
look up per-rule property values (e.g. `partition` from team settings).
defineUserScope populates both from the member's team settings and
member properties. buildSqlSecurityContext (the SQL API path) never
did, so userScope for SQL logins had no team/member properties. Every
rule whose property lookup returned undefined blocked the whole query
and queryRewrite replaced query.filters with:
[{ member: allMembers[0], operator: "equals",
values: ["__blocked_by_access_control__"] }]
When the first member was a numeric measure (e.g. `count`), ClickHouse
tried to cast the sentinel to Float64:
Cannot parse string '__blocked_by_access_control__' as Float64
Fix: buildSqlSecurityContext now resolves the member for the
datasource's team and passes the team settings + member properties
into the scope (matching defineUserScope). Team settings also flow
into buildSecurityContext so the content hash includes them, keeping
cache isolation consistent between REST and SQL paths.
Reproduced via `SELECT count(*) FROM stockout_event` over the Postgres
wire — rule `semantic_events.partition` couldn't resolve
team.partition, blocked fired, sentinel filter crashed ClickHouse.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cubejs): include team.settings and member.properties in membersFragment
Follow-up to #42. buildSqlSecurityContext now resolves teamProperties
and memberProperties from sqlCredentials.user.members, but the GraphQL
query used to load sql_credentials (sqlCredentialsQuery →
membersFragment) never selected those fields. At runtime teamMember
was found but team and properties were undefined, so teamProperties
stayed empty and queryRewrite rules that look up
teamProperties.<key> still blocked every query.
Observed via:
SELECT count(*) FROM stockout_event
→ still rewritten to:
HAVING count(*) = toFloat64('__blocked_by_access_control__')
Add team { id settings } and properties to membersFragment so the SQL
API path has the same shape defineUserScope consumes on the REST path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(cubejs): bump @cubejs-backend/* from 1.6.19 to 1.6.37
Brings in 16 patch releases of upstream Cube.js fixes and features,
including cubesql improvements (Tableau format codes, Talend compat,
MEASURE function panic fix, LAG/LEAD pushdown, TO_TIMESTAMP formats,
SET TIMEZONE, FETCH directions, CASE/LIKE planning, pg_catalog.pg_collation,
SAVEPOINT/ROLLBACK TO/RELEASE, SET ROLE auth context, and more).
Does not close the DataGrip introspection gap (regclass in functions,
pg_get_userbyid coercion, OPERATOR(schema.~), CHAR[] arrays,
SHOW server_version, pg_description.objoid mapping, empty pg_index/
pg_constraint — none addressed upstream between 1.6.21 and 1.6.37),
but keeps us current before we build or wait on a JetBrains-friendly
fix.
yarn.lock will regenerate on CI build (Dockerfile runs `yarn --network-timeout 100000`).
No breaking changes relevant to us (server-core access-policy row
filtering breaking change doesn't affect our usage — we don't use
cube's access_policy feature; row filtering is via queryRewrite.js).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(cubejs): bump CUBESTORE_VERSION in .env.example to v1.6.37
Keeps local docker-compose in sync with the Cube.js backend bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(011): Model Management API — validate, refresh, delete, single-cube, diff, rollback
Adds six authenticated REST endpoints that let an agent own the full
author-to-publish lifecycle of a cube model without operator assistance:
POST /api/v1/validate-in-branch (US1)
POST /api/v1/internal/refresh-compiler (US2)
DELETE /api/v1/dataschema/:dataschemaId (US3)
GET /api/v1/meta/cube/:cubeName (US4)
POST /api/v1/version/diff (US5)
POST /api/v1/version/rollback (US5)
Delete and rollback emit durable audit rows via Hasura event triggers
(`audit_dataschema_delete`, `audit_version_rollback`) into a new
`audit_logs` table with 90-day retention via a daily cron trigger. Refresh
is cache-only and emits a non-durable structured log line only.
The Hasura migration (`1713600000000_dataschemas_delete_permission`) adds
`versions.origin` + `versions.is_current` with a statement-level trigger
that uses a NEW TABLE transition table so multi-row inserts cannot break
the invariant. A transaction-scoped advisory lock keyed on affected branches
serialises concurrent inserts on the same branch while different branches
still proceed in parallel. Backfill runs in 1 000-row batches. `/meta-all`
now enriches every cube summary with `dataschema_id` + `file_name` by
parsing each schema once per call (cube-name keyed, since Cube.js v1.6
`metaConfig` omits `fileName`).
All five mutating handlers write a durable audit row on every failure path
(partition mismatch, insufficient role, historical version, blocking
references, Hasura rejection). Success is captured by the event triggers.
New utilities: `compilerCacheInvalidator`, `referenceScanner` (FR-008 seven
kinds), `directVerifyAuth`, `requireOwnerOrAdmin`, `mapHasuraErrorCode`,
`auditWriter`, `metaForBranch`, `versionDiff`, `errorCodes` (FR-017
single-source-of-truth enum). `graphql.js` gains a `preserveErrors` option
so handlers can surface Hasura extension codes as stable FR-017 codes.
Spec + runbook: `specs/011-model-mgmt-api/` (spec.md, plan.md, tasks.md,
research.md, data-model.md, contracts/, quickstart.md, DEPLOYMENT.md,
migration README). `scripts/lint-error-codes.mjs` fails the build if the
error-code enum drifts across `errorCodes.js` and any of the six contracts.
Tests: 49 Vitest-style `node:test` unit tests for the new utilities +
`summarizeCube` + versionDiff adapter + SC-003 fixture corpus; 5
integration tests for the Actions RPC handlers; 8 StepCI workflows under
`tests/workflows/model-management/` including an end-to-end flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 9bfc8bd commit e1dd0bf
72 files changed
Lines changed: 7570 additions & 29 deletions
File tree
- scripts
- services
- actions/src/rpc
- __tests__
- cubejs
- src
- routes
- __tests__
- utils
- __tests__
- hasura
- metadata
- migrations/1713600000000_dataschemas_delete_permission
- specs/011-model-mgmt-api
- checklists
- contracts
- tests/workflows/model-management
- fixtures
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
99 | | - | |
| 99 | + | |
100 | 100 | | |
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
104 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
105 | 119 | | |
106 | 120 | | |
107 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
108 | 125 | | |
109 | 126 | | |
110 | 127 | | |
| |||
273 | 290 | | |
274 | 291 | | |
275 | 292 | | |
| 293 | + | |
| 294 | + | |
276 | 295 | | |
277 | 296 | | |
278 | 297 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
Lines changed: 53 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
Lines changed: 35 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
Lines changed: 64 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
0 commit comments