Repository navigation
A SET applies its writes in the order they were written (#1896) - #1916
Conversation
`SetClause` held three vectors by kind (property, label, whole-entity), so the
order between kinds was lost at parse time, and `SetPropertyOperator` ran every
`n.k = v` before every `n = {…}`. `SET n.k = 7, n = {…}` came out right and
`SET n = {…}, n.k = 7` silently lost `k`, which is the bulk-import shape
`SET n = row.props, n.source_id = row.id`.
- ast: `SetClause { ops: Vec<SetOp> }`, one ordered sequence; `items()`,
`label_items()` and `entity_items()` read it by kind.
- operator: `SetPropertyOperator` holds `Vec<SetWrite>` and applies it in
order. Property values are still all evaluated before any write, so
`SET a.x = a.y, a.y = a.x` still swaps.
- planner: `attach_set` builds operators per clause instead of flattening
every clause into one list, so `SET n = {…} SET n.k = 7` keeps `k`.
`Clause::Set` and the bare SET after MERGE go through it too; the MERGE
path used to keep only `n.prop = v` items and drop `n = {…}` and labels.
- tests/set_order.rs pins both orders, two clauses, the import shape, `+=`,
the swap, and SET after MERGE.
Kalyanpulikanti-01
left a comment
There was a problem hiding this comment.
Approving. This is the fix the diagnosis pointed at, and it went further than the diagnosis did.
One source of truth, not an ordinal. SetClause { ops: Vec<SetOp> } with items()/label_items()/entity_items() as filtered iterators is the right shape: kind-only readers (validation, parameter substitution) keep reading by kind, and nothing has to know to sort. An order: usize beside three vectors would have left the invariant unenforced.
The new doc comment also directly replaces the old one that argued against this, which is the honest way to retire a wrong rationale rather than quietly deleting it.
The evaluation split is preserved correctly, and that's the subtle part. evaluated pre-computes only the SetWrite::Property right-hand sides (entity entries map to None), then the single apply loop zips writes with evaluated in source order — properties from the pre-read values, entity assignments evaluated where they stand. So SET a.x = a.y, a.y = a.x still swaps while ordering is now positional. Doing the obvious thing instead — evaluating each RHS at its write — would have fixed #1896 and broken the swap, and property_values_are_still_read_before_any_write is what pins it.
No production caller retains the old order. I checked: all three sites are gone — planner.rs:1844 (SetPropertyOperator::new), :8814 and :9358 (with_entity_items) — replaced by one from_writes. with_entity_items survives as public API for the cov tests and still appends entity items last, which is correct for a constructor that is handed two kind-separated lists and has no ordering to preserve. It is now the only way to get the old behaviour, and its doc comment points at from_writes; worth removing outright once the cov tests move over, since a future caller reaching for it reintroduces #1896 silently.
The third bug is the one worth calling out. MERGE (m) SET m = {…} and MERGE (m) SET m:Label being dropped without an error is a separate silent write loss from the one in the issue — not a reordering, a total discard — and it was found only because this path was being consolidated. That belongs in the release note as its own line, not folded into "SET ordering".
The tests can fail. Every row asserts x is None, so the replace demonstrably happened, alongside k surviving. An implementation that simply stopped replacing would satisfy "the later property is kept" and fail on x — which is exactly the shortcut a narrower test would have let through. the_bulk_import_shape_keeps_its_source_id then reads the node back by {k: 7}, which is the failure the issue was actually about rather than a proxy for it.
One note: SetOp::Labels is carried in ops in source order but labels are still applied by a separate LabelMutationOperator after the property/entity writes, so label position relative to the others is not honoured. That is correct and worth a sentence somewhere, because it reads like an oversight: SET n = {…} replaces properties only and never touches labels, so no ordering between a label item and an entity item is observable. If whole-entity assignment ever gained label semantics, that would stop being true.
I've filed the deferred half as #1918 — ON CREATE SET / ON MATCH SET keeping the same by-kind split. Your read is almost certainly right that ON CREATE SET n = row.props, n.id = row.id loses id, and leaving it inside this PR would have widened it across several operators that each keep their own copy of the set logic. Splitting it was the right call; it just needed to exist somewhere other than a PR description.
Thanks for crediting the trace — and for not stopping at the row in the issue table.
|
Correction to my review above: the follow-up issue is #1919, not #1918 — another issue took that number while I was writing. Everything else stands. #1919 confirms the diagnosis against the code: |
#1916 (#1896) made `SetPropertyOperator` apply its writes in source order, which changed the value-evaluation closure to return `Option<PropertyValue>` over the new `writes` vector. This branch's fix for #1897 makes a failed evaluation the statement's error rather than `Null` -- and a null SET removes the property, so an expression that failed reported success and deleted what was there. Both are needed, and taking either side of the conflict alone would drop the other silently: - the closure now returns `ExecutionResult<Option<PropertyValue>>`, `None` for a non-property write, and the collect is `collect::<ExecutionResult<_>>()?`, so `Err(e) => return Err(e)` still propagates; - `SET a.x = a.y, a.y = a.x` still swaps, because every property value is read before any write; - writes still apply in the order they were written. `validate.rs`'s `write_expressions` reads `SetClause` through the `items()` / `entity_items()` accessors that replaced the by-kind fields. `cargo check --lib --tests` is clean.
Fixes #1896.
The bug
SET n = {name: 'A'}, n.k = 7stored{name: 'A'}and reported success. Any property assignment written afterSET n = mapon the same variable was lost, in the same SET clause or a later one. This is the bulk-import shapeSET n = row.props, n.source_id = row.id: the id was dropped, and every laterMATCH (n {source_id: …})found nothing.Root cause
As Kalyanpulikanti-01 traced in the issue, there were two separate losses of order:
SetClausekept three vectors by kind (items,label_items,entity_items), so the order between kinds was discarded at parse time.SetPropertyOperatorthen ran everyn.k = vbefore everyn = {…}. That madeSET n.k = 7, n = {…}right andSET n = {…}, n.k = 7wrong. Any rule based on kind gets exactly one of the two orders right.attach_setflattened every SET clause of a query into one list, soSET n = {…} SET n.k = 7was reordered too.The fix
src/query/ast.rs):SetClause { ops: Vec<SetOp> }, withSetOp::{Property, Labels, Entity}in source order. This is one source of truth rather than an ordinal beside three vectors.items(),label_items()andentity_items()read it by kind for the places that only care about one kind (validation and parameter substitution).src/query/executor/operator.rs):SetPropertyOperatorholdsVec<SetWrite>and applies it in order. Property values are still all evaluated before any write, as before, soSET a.x = a.y, a.y = a.xstill swaps. A whole-entity value is evaluated where it stands, as it always was.src/query/executor/planner.rs):attach_setbuilds operators per clause.Clause::Setnow goes through it instead of keeping its own copy.SETafterMERGE(MERGE (m) SET …) had its own path that kept only then.prop = vitems.MERGE (m) SET m = {…}andMERGE (m) SET m:Labelwere dropped without an error. That path now usesattach_settoo.Tests
tests/set_order.rschecks every row of the issue's table:UNWIND … SET n = row.props, n.k = row.fidimport shape, including that a laterMATCH (n {k: …})finds the node;+=alongside a property;Every test also reads back a property that should have been removed, so an implementation that never replaced anything would fail.
cargo check --workspace --tests --examples --benchesis clean, and the lint ratchet is under its ceiling.Not done here
MERGE … ON CREATE SET/ON MATCH SEThave the same by-kind split (on_create_setbesideon_create_entity_set), and the merge operators apply the property items first. SoON CREATE SET n = row.props, n.id = row.idprobably losesidin the same way. Those operators keep their own copies of the set logic in several places, so I've left them to a follow-up rather than widen this PR.