Added support for Redis Query Engine by vladvildanov · Pull Request #1356 · redis/redis-rb

vladvildanov · 2026-06-25T07:43:51Z

Summary

This PR adds first-class support for the Redis Query Engine and aligns it with the module-command structure established by the JSON module. It started from an earlier standalone RQE implementation and was adapted to fit the current lib/redis/commands/modules/** layout, then extended with reply reshaping, protocol compatibility, broad test coverage, and runnable examples.

Kudos to @bsbodden for the initial RQE implementation

All FT.* commands are available on any Redis (and, by inheritance, Redis::Cluster) instance. The change is purely additive — no existing behavior changes.

What's included

30 ft_* command methods (lib/redis/commands/modules/search/miscellaneous.rb) covering:

Index lifecycle: ft_create, ft_alter, ft_dropindex, ft_info
Querying: ft_search, ft_aggregate, ft_cursor_read/ft_cursor_del, ft_explain, ft_profile
Hybrid (lexical + vector): ft_hybrid_search
Suggestions: ft_sugadd/ft_sugget/ft_suglen/ft_sugdel
Dictionaries, synonyms, aliases, tag values, spellcheck, config

A high-level builder/object layer (the recommended entry point):

Index — bundles schema + prefix + connection; create_index returns it and it exposes #add/#search/#aggregate/#hybrid_search/#info/#drop/…
Schema / Field (Text/Numeric/Tag/Geo/GeoShape/Vector incl. FLAT/HNSW/SVS‑VAMANA)
Query (fluent + predicate DSL), IndexDefinition/IndexType
AggregateRequest + Reducers + Asc/Desc + Cursor
Hybrid builders: HybridSearchQuery, HybridVsimQuery, HybridQuery, CombineResultsMethod, HybridPostProcessingConfig, HybridCursorQuery

Typed, protocol-independent result objects (result.rb): SearchResult/Document, AggregateResult, and HybridResult, produced by a ResultParser that normalizes both RESP2 (flat arrays) and RESP3 (native maps) to the same Ruby objects. FT.INFO/FT.CONFIG GET/FT.SYNDUMP/FT.SPELLCHECK return plain Hashes.

Both styles interoperate — you can call a raw ft_* method with a hand-built query string and still get a reshaped result back.

Layering

Index (high-level) → Builders (Schema/Query/AggregateRequest/Hybrid*) → ft_* commands → ResultParser (RESP2/RESP3 → objects).

See specs/query-engine.md for the full design and an abstractions-vs-raw-commands guide.

Behavior notes

Default dialect = 2 (Search::DEFAULT_DIALECT). Applied on every path — Query, AggregateRequest, and the raw ft_search/ft_aggregate string forms — overridable per query.
Protocol: works under both RESP3 (the gem default) and RESP2.
Redis::Distributed: not supported (FT.* is index-scoped, not key-shardable) — methods are simply absent. Redis::Cluster inherits the commands via the Commands mixin; the module test suite exercises standalone only (consistent with the JSON module).

Examples & docs

examples/search/ — 10 runnable examples, Index-API-first (quickstart, ft_queries, range, aggregations, geo, vector_similarity, with_hashes, with_json, hybrid), plus search_raw_ft_commands.rb showing the low-level alternative.
specs/query-engine.md — architecture + usage guide.

Note

Medium Risk
Large additive API surface and reply parsing across RESP2/RESP3; incorrect parsing or command wiring could break search workloads, but existing non-FT behavior is unchanged and Distributed is explicitly excluded.

Overview
Adds first-class Redis Query Engine (RediSearch FT.*) support on Redis and Redis::Cluster, mirroring the JSON module layout under lib/redis/commands/modules/search/.

Callers get ~30 ft_* methods (create/search/aggregate/hybrid, suggestions, synonyms, dictionaries, aliases, spellcheck, config, cursors) plus a high-level Index API via create_index: schema/field builders, fluent Query with a predicate DSL, AggregateRequest/Reducers, and FT.HYBRID builders (lexical + vector fusion). Replies are normalized by ResultParser into SearchResult/Document, AggregateResult, and HybridResult for both RESP2 and RESP3; dialect 2 is the default on search/aggregate paths.

CHANGELOG documents the feature (not on Redis::Distributed). CI adds an unstable Redis image to the test matrix. examples/search/ adds runnable quickstart, geo, vector, hybrid, aggregations, and raw-ft_* samples.

^{Reviewed by Cursor Bugbot for commit 5663b85. Bugbot is set up for automated code reviews on this repo. Configure here.}

Implement modular Search architecture with complete feature parity to redis-py: Core Components: - Schema and field definitions (TextField, NumericField, TagField, GeoField, VectorField, GeoShapeField) - Query builder with fluent API and advanced query syntax - Index management and operations (create, drop, alter, info) - Aggregation framework with reducers and grouping - Hybrid search combining text and vector queries - Result parsing and formatting Search Features: - Full-text search with stemming, phonetic matching, and stop words - Vector similarity search supporting FLAT, HNSW, and SVS-VAMANA algorithms - Geospatial search with radius and polygon queries - Numeric and tag filtering - Aggregations with grouping, sorting, applying, and reducing - Hybrid search with score combination (RRF, linear) - Auto-complete/suggestions with fuzzy matching - Spell checking and synonym management - Query profiling and explain Field Types: - TextField: full-text search with weights, phonetic matching, withsuffixtrie - NumericField: range queries with sortable option - TagField: exact-match filtering with case sensitivity and separators - GeoField: geospatial queries with radius search - VectorField: vector similarity with multiple algorithms and distance metrics - GeoShapeField: polygon-based geospatial queries Advanced Features: - IndexDefinition for fine-grained index control - Query parameters for dynamic queries - Multiple dialect support (1, 2, 3, 4) - Cursor-based pagination for large result sets - Score explanations and custom scorers - Highlighting and summarization Testing: - Add comprehensive test suite with 44 tests across 8 test files - Test coverage for all search features and edge cases - Hybrid search tests with vector + text queries - Aggregation tests with complex pipelines - Vector similarity tests with multiple algorithms Documentation: - Add 6 example files demonstrating all features: - search_quickstart.rb: basic search operations - search_ft_queries.rb: advanced query syntax - search_aggregations.rb: aggregation pipelines - search_geo.rb: geospatial queries - search_range.rb: numeric range queries - search_vector_similarity.rb: vector search - search_with_hashes.rb: search with Redis hashes

jit-ci · 2026-06-25T07:44:49Z

🛡️ Jit Security Scan Results

✅ No security findings were detected in this PR

^{Security scan by Jit}

Copilot

Pull request overview

This PR adds first-class Redis Query Engine (RediSearch FT.*) support to redis-rb, following the existing “module commands + higher-level object layer” pattern used for RedisJSON. It introduces a Redis::Commands::Search command surface, builder/DSL objects for schemas and queries, and protocol-normalizing reply parsing (RESP2/RESP3), plus tests and runnable examples.

Changes:

Add Redis::Commands::Search with ~30 ft_* command methods and a higher-level Index API (schema/query/aggregation/hybrid builders).
Add ResultParser + typed result objects to reshape RESP2/RESP3 replies into consistent Ruby objects.
Add documentation (specs/query-engine.md), tests (online lint suite runner + offline unit tests), examples, and changelog entry.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
test/modules/search_offline_test.rb	Adds offline unit tests for builders and reply reshaping.
test/modules/commands_on_search_test.rb	Hooks the shared `Lint::Search` suite into module-capable server tests.
specs/query-engine.md	Documents architecture/layers and usage guidance for builders vs raw `ft_*`.
lib/redis/commands/modules/search/schema.rb	Implements schema DSL (`Schema`, `SchemaDefinition`, vector-field DSL).
lib/redis/commands/modules/search/result.rb	Adds typed result objects and RESP2/RESP3 reply reshaping (`ResultParser`).
lib/redis/commands/modules/search/query.rb	Implements fluent query builder and predicate DSL.
lib/redis/commands/modules/search/miscellaneous.rb	Implements `ft_*` command methods and reply reshaping hooks.
lib/redis/commands/modules/search/index.rb	Adds high-level `Index` wrapper API around schema/prefix/search/aggregate/etc.
lib/redis/commands/modules/search/index_definition.rb	Adds `IndexDefinition` and `IndexType` for FT.CREATE “definition” clause.
lib/redis/commands/modules/search/hybrid.rb	Adds FT.HYBRID builders and post-processing/cursor helpers.
lib/redis/commands/modules/search/field.rb	Adds field classes for schema rendering and query predicate helpers.
lib/redis/commands/modules/search/dialect.rb	Introduces `DEFAULT_DIALECT = 2`.
lib/redis/commands/modules/search/aggregation.rb	Adds `AggregateRequest`, reducers, sort wrappers, and cursor helper.
lib/redis/commands/modules/search.rb	Loads all Search module components.
lib/redis/commands.rb	Wires `Search` into the top-level `Redis::Commands` mixin.
examples/search/search_with_json.rb	Example using Search with RedisJSON documents.
examples/search/search_with_hashes.rb	Example using Search with HASH documents and the Index API.
examples/search/search_vector_similarity.rb	Example demonstrating vector similarity search patterns and KNN usage.
examples/search/search_raw_ft_commands.rb	Example showing direct use of raw `ft_*` methods.
examples/search/search_range.rb	Example of numeric range querying patterns.
examples/search/search_quickstart.rb	Quickstart example for basic indexing and querying.
examples/search/search_hybrid.rb	Example of FT.HYBRID via builder objects and Index API.
examples/search/search_geo.rb	Example of geo/geoshape queries and dialect usage.
examples/search/search_ft_queries.rb	Example suite showing various FT.SEARCH query patterns.
examples/search/search_aggregations.rb	Example suite showing aggregation pipelines and reducers.
CHANGELOG.md	Notes the new Query Engine feature in the Unreleased changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…refix

Copilot

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 13 comments.

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 90ca351. Configure here.}

petyaslavova · 2026-06-29T07:52:13Z

+          @index_type = index_type
+          # Array() makes the prefix handling nil-safe (nil -> []) and wraps a lone String prefix
+          # into a one-element list, so PREFIX always emits the correct count.
+          @prefixes = Array(prefix)


In case a list with prefixes is provided, the call to Arrays(prefix) will return the same list instance, and if the caller later changes the values in the original prefix instance - the changes will affect the current object as well. Maybe it worths duplicating the input list to ensure that the content of the IndexDefinition won't change due to some external object update.

petyaslavova · 2026-06-29T11:33:33Z

+        # @return [Integer] the cursor id
+        # @return [Integer] the +MAXIDLE+ in milliseconds (0 to omit)
+        # @return [Integer] the +COUNT+ (batch size, 0 to omit)
+        attr_accessor :cid, :max_idle, :count


I think the max_idle should be removed from this class. According to the official docs, max_idle is not supported for ft.cursor-read, and this object seems to be used only for this command (the max_idle for ft.aggregate is specified above).

petyaslavova · 2026-06-29T12:18:33Z

+        #
+        # @param [String, Symbol] name the document attribute the field indexes
+        # @param [String] coord_system the coordinate system, {FLAT} or {SPHERICAL}
+        def initialize(name, coord_system = FLAT, **options)


The default server value is SPHERICAL. By providing here FLAT as default coord_system we never allow the default server behaviour to be used.

petyaslavova · 2026-06-29T13:20:02Z

+          end
+
+          begin
+            json? ? @redis.json_set(key, "$", fields) : @redis.hset(key, fields)


add for JSON indexes currently writes the provided kwargs directly at the JSON root via json_set(key, "$", fields).
That only works when schema paths are root-level ($.name AS name), but not for nested paths like $.user.name AS name.
index.add("1", name: "Ann") stores {"name":"Ann"}, while the index expects {"user":{"name":"Ann"}}.
We should either document/test that JSON add requires the full JSON document shape, or reject alias-shaped writes for nested JSON paths with a clear error.

petyaslavova · 2026-06-29T14:53:41Z

+    def wait_for_index(index_name, timeout = 5.0)
+      deadline = now + timeout
+      loop do
+        info = r.ft_info(index_name)


You should have some error handling around this command, but still inside the loop - it will raise an error if the index is not yet existing and will basically silently just exit and won't wait.

petyaslavova · 2026-06-29T15:24:15Z

+      r.hset("doc1", embedding: [0.1, 0.9, 0.2, 0.8].pack("f*"), tag: "a")
+      r.hset("doc2", embedding: [0.2, 0.8, 0.3, 0.7].pack("f*"), tag: "b")
+      r.hset("doc3", embedding: [0.8, 0.2, 0.7, 0.3].pack("f*"), tag: "c")
+


You can include wait_for_index() here as well.

petyaslavova · 2026-06-29T15:24:52Z

+
+      r.hset("doc1", title: "foo", embedding: [0.1, 0.9].pack("f*"))
+      r.hset("doc2", title: "bar", embedding: [0.8, 0.2].pack("f*"))
+


petyaslavova · 2026-07-01T09:23:05Z

+          @return_fields_decode ||= {}
+
+          @return_fields << field
+          @return_fields_decode[field] = decode_field


When as_field is used, Redis returns the value under the alias, but decode_fields is keyed by the original field/path here. For return_field("$.user", as_field: "user"), the parser sees "user" and won’t decode because the map contains "$.user". This should key decoding by as_field || field.

Brian Sam-Bodden and others added 5 commits January 2, 2026 12:38

Merge branch 'master' of github.com:redis/redis-rb into redis/search

773b350

Adapt existing RQE approach to current module structure

68992f1

Removed local config

ab1bcac

Moved search examples

8b16d62

vladvildanov added the feature label Jun 25, 2026

vladvildanov requested a review from Copilot June 25, 2026 07:44

Copilot started reviewing on behalf of vladvildanov June 25, 2026 07:44 View session

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/field.rb Outdated

Comment thread lib/redis/commands/modules/search/miscellaneous.rb

Comment thread lib/redis/commands/modules/search/index_definition.rb

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Added correct version constraints

02dfa89

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/index.rb Outdated

Comment thread lib/redis/commands/modules/search/index.rb

Comment thread lib/redis/commands/modules/search/miscellaneous.rb Outdated

Comment thread lib/redis/commands/modules/search/miscellaneous.rb

Fixed bugs with incorrect alias field and missing search query arguments

692cd6b

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/index.rb Outdated

Comment thread examples/search/search_with_hashes.rb

vladvildanov added 2 commits June 25, 2026 11:18

Fixed bugs with incorrect FIELD processing and not respecting index p…

5632b3c

…refix

Fixed adressed bugs

b37029d

vladvildanov requested a review from Copilot June 25, 2026 08:38

Copilot started reviewing on behalf of vladvildanov June 25, 2026 08:38 View session

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/index.rb

Comment thread lib/redis/commands/modules/search/index.rb Outdated

Comment thread lib/redis/commands/modules/search/miscellaneous.rb

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Fixed bugs

9e50400

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/miscellaneous.rb

Comment thread lib/redis/commands/modules/search/field.rb

Fixed inconsistencies

fed2a01

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/index.rb

Fixed non-used arguments

7dcc7ce

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/result.rb

vladvildanov added 2 commits June 25, 2026 13:50

Added testing with 8.10-milestone1

9b6a38a

Removed deprecated ft.add, ft.mget, ft.del commands

ef7fb44

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/index.rb

Added index type resolution to support JSON indices management

90ca351

cursor Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread lib/redis/commands/modules/search/index.rb

Fixed bugs with index

5663b85

petyaslavova reviewed Jun 29, 2026

View reviewed changes

petyaslavova reviewed Jul 1, 2026

View reviewed changes


		r.hset("doc1", title: "foo", embedding: [0.1, 0.9].pack("f*"))
		r.hset("doc2", title: "bar", embedding: [0.8, 0.2].pack("f*"))

Uh oh!

Conversation

vladvildanov commented Jun 25, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's included

Layering

Behavior notes

Examples & docs

Uh oh!

jit-ci Bot commented Jun 25, 2026

🛡️ Jit Security Scan Results

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

vladvildanov commented Jun 25, 2026 •

edited by cursor Bot

Loading