Releases: HelixDB/helix-db
Releases · HelixDB/helix-db
v2.2.8
v2.2.7
impr (hql+core): nested ids and math ops (#845)
v2.2.6
impr(cli + core + hql): ordered return values, helix restart, hql fix…
v2.2.5
impr(hql+cli+core): nested traversal fixes, CLI logs TUI, and upsert…
v2.2.4
fix(oops): hotfix version bump (#817)
v2.2.3
impr (cli + core + hql): fixing upserts, fixing bugs, improving logs …
v2.2.2
feat (cli + core): Adding logs to CLI and allowing working number set…
v2.2.1
fix (core): upserts bug (#803)
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
Fixed critical bug where `upsert_v` created vectors that were not
searchable. The old code used `put_vector` which only stored vectors in
the database without adding them to the HNSW index, while the fix
properly uses the `insert` method to make vectors searchable.
Key changes:
- Replaced manual vector creation and `put_vector` call with
`vectors.insert()` method that properly adds vectors to HNSW index
- Added comprehensive test `test_upsert_v_new_vector_is_searchable` to
verify searchability
- Improved error handling: changed error type from `VectorNotFound` to
`EntryPointNotFound` in `get_raw_vector_data` for more accurate error
messages
- Minor formatting improvements to secondary index handling code
- Version bumps: helix-db 1.2.0 → 1.2.1, helix-cli 2.2.0 → 2.2.1
Issue found:
- Pre-existing bug (not introduced by this PR): when creating new
vectors, secondary indices use `APPEND_DUP` for all index types without
checking if they're `Unique` or `Index`, which could allow duplicate
values in unique indices
<details><summary><h3>Important Files Changed</h3></summary>
| Filename | Overview |
|----------|----------|
| helix-db/src/helix_engine/traversal_core/ops/util/upsert.rs | Fixed
upsert to use HNSW insert for new vectors (making them searchable), but
pre-existing bug with unique secondary indices remains |
| helix-db/src/helix_engine/tests/traversal_tests/upsert_tests.rs |
Added test to verify upserted vectors are searchable via HNSW |
| helix-db/src/helix_engine/vector_core/vector_core.rs | Improved error
type accuracy in get_raw_vector_data (EntryPointNotFound instead of
VectorNotFound) |
</details>
</details>
<details><summary><h3>Sequence Diagram</h3></summary>
```mermaid
sequenceDiagram
participant Client
participant UpsertAdapter
participant VectorCore
participant HNSW
participant SecondaryIndices
participant BM25
Client->>UpsertAdapter: upsert_v(query, label, props)
UpsertAdapter->>VectorCore: get_vector_by_label(label)
alt Vector exists
VectorCore-->>UpsertAdapter: Some(existing_vector)
UpsertAdapter->>VectorCore: update_vector(with new data)
UpsertAdapter->>SecondaryIndices: put_with_flags (NO_OVERWRITE or APPEND_DUP)
UpsertAdapter->>BM25: update_doc()
UpsertAdapter->>VectorCore: put_vector()
else Vector does not exist (NEW CODE PATH)
VectorCore-->>UpsertAdapter: None
Note over UpsertAdapter,HNSW: OLD: Created vector manually + put_vector<br/>NEW: Use vectors.insert()
UpsertAdapter->>VectorCore: insert(label, query, properties)
VectorCore->>HNSW: insert vector into HNSW index
HNSW-->>VectorCore: vector added to index
VectorCore-->>UpsertAdapter: Ok(vector)
UpsertAdapter->>SecondaryIndices: put_with_flags(APPEND_DUP)
Note over UpsertAdapter,SecondaryIndices: Bug: Should check Unique vs Index type
UpsertAdapter->>BM25: insert_doc()
end
UpsertAdapter-->>Client: TraversalValue::Vector(result)
```
</details>
<!-- greptile_other_comments_section -->
<!-- /greptile_comment -->
Release v2.2.0
major update across entire codebase (#795)
<!-- greptile_comment -->
<h2>Greptile Overview</h2>
### Greptile Summary
This PR implements comprehensive upsert functionality and unique index
support across the entire HelixDB codebase. The changes span 171 files
and add ~7,457 lines while removing ~2,175 lines.
## Major Features Added
**1. Upsert Operations (UpsertN, UpsertE, UpsertV)**
- Node upserts: Creates new nodes or updates existing ones by merging
properties
- Edge upserts: Creates or updates edges between nodes
- Vector upserts: Creates or updates vector embeddings with associated
properties
- Full grammar, parser, analyzer, and code generation support
**2. Unique Index Support**
- `UNIQUE INDEX` field prefix for node/vector properties
- `UNIQUE` modifier for edge definitions
- Enforces uniqueness constraints via `PutFlags::NO_OVERWRITE`
- Returns `DuplicateKey` error on constraint violations
**3. Secondary Index Enhancements**
- New `SecondaryIndex` enum distinguishes `Unique`, `Index`, and `None`
types
- Updated storage methods to accept typed indices
- Proper index management during upsert operations
## Key Implementation Details
The upsert logic handles three scenarios:
1. **No existing node**: Creates new node with properties and indices
2. **Existing node without properties**: Adds properties and creates
indices
3. **Existing node with properties**: Deletes old indices, creates new
ones, merges properties
The PR includes extensive test coverage with unit tests, integration
tests, and HQL test cases validating unique constraints and upsert
behavior.
## Code Quality
- Comprehensive formatting applied via `rust fmt`
- Error handling improvements throughout
- Well-structured test suites for new functionality
<details><summary><h3>Important Files Changed</h3></summary>
File Analysis
| Filename | Score | Overview |
|----------|-------|----------|
| helix-db/src/helix_engine/traversal_core/ops/util/upsert.rs | 3/5 |
New file implementing upsert operations for nodes, edges, and vectors
with secondary index support. Contains logic for merging properties and
updating indices. |
| helix-db/src/helix_engine/tests/traversal_tests/upsert_tests.rs | 5/5
| Comprehensive test suite for upsert operations covering node, edge,
and vector upserts with various scenarios including empty properties and
updates. |
| helix-db/src/helix_engine/types.rs | 5/5 | Added SecondaryIndex enum
to distinguish unique vs regular indices and DuplicateKey error variant
for unique constraint violations. |
| helix-db/src/helixc/parser/types.rs | 5/5 | Added UpsertNode,
UpsertEdge, UpsertVector types and UniqueIndex field prefix. Extended
ExpressionType and StepType enums to support upsert operations. |
| helix-db/src/helixc/generator/source_steps.rs | 4/5 | Added code
generation for UpsertN, UpsertE, UpsertV operations. Updated AddE to
include is_unique parameter for unique edge constraints. |
| helix-db/src/helix_engine/traversal_core/ops/source/add_e.rs | 4/5 |
Added is_unique parameter to add_edge method. Uses NO_OVERWRITE flag for
unique edges instead of APPEND_DUP to enforce uniqueness. |
| helix-db/src/grammar.pest | 5/5 | Added UNIQUE modifier for edge
definitions and UNIQUE INDEX for field definitions. Added UpsertN,
UpsertE, UpsertV grammar rules. |
| helix-db/src/helixc/analyzer/methods/infer_expr_type.rs | 3/5 | Major
expansion with type inference for upsert operations. Handles validation
of upsert node, edge, and vector expressions with proper type checking.
|
| helix-db/tests/upsert_integration_test.rs | 5/5 | New integration
tests verifying upsert parsing and compilation for nodes, edges, and
vectors across various scenarios. |
| helix-db/src/helixc/parser/schema_parse_methods.rs | 5/5 | Updated
edge parsing to support UNIQUE modifier. Parses edge_modifier rule and
sets unique flag on EdgeSchema. |
| helix-db/src/helix_engine/storage_core/storage_methods.rs | 5/5 |
Updated create_secondary_index signature to accept SecondaryIndex enum
instead of plain string for better type safety. |
| hql-tests/tests/add_n_unique/queries.hx | 5/5 | New HQL test
demonstrating UNIQUE INDEX on node fields and UNIQUE modifier on edge
definitions. |
</details>
</details>
<details><summary><h3>Sequence Diagram</h3></summary>
```mermaid
sequenceDiagram
participant User
participant Parser
participant Analyzer
participant Generator
participant Runtime
participant Storage
User->>Parser: UpsertN<Person>({name: "Alice"})
Parser->>Parser: Parse UNIQUE INDEX fields
Parser->>Analyzer: AST with UpsertNode
Analyzer->>Analyzer: Validate node type exists
Analyzer->>Analyzer: Validate field types match schema
Analyzer->>Analyzer: Check unique index constraints
Analyzer->>Generator: Validated AST
Generator->>Generator: Generate upsert_n() call
Generator->>Generator: Generate property slice
Generator->>Runtime: Compiled Rust code
Runtime->>Runtime: Check if node exists in iterator
alt Node exists with properties
Runtime->>Storage: Delete old secondary indices
Runtime->>Storage: Insert new secondary indices with NO_OVERWRITE (unique)
Runtime->>Runtime: Merge old and new properties
else Node exists without properties
Runtime->>Storage: Insert secondary indices
Runtime->>Runtime: Create properties map
else No node exists
Runtime->>Storage: Create new node
Runtime->>Storage: Insert secondary indices
end
Runtime->>Storage: Update BM25 index
Runtime->>Storage: Save node
Storage-->>User: Node with merged properties
```
</details>
<!-- greptile_other_comments_section -->
<!-- /greptile_comment -->
v2.1.10
impr (hql + cli): improve dx for interactive cli and fix hql error (#…