From 5621bb13697c7f92bb3e72248a8ffa80e1051aff Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Tue, 10 Jun 2025 15:39:03 -0500 Subject: [PATCH 1/3] DOC-4271 RS: Adjusted best practices for scalable Redis query engine & add benchmarks --- .../search/scalable-query-best-practices.md | 117 +++++++++++++++++- 1 file changed, 112 insertions(+), 5 deletions(-) diff --git a/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md b/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md index e32dbbe77e..7c64df00a0 100644 --- a/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md +++ b/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md @@ -10,7 +10,7 @@ linkTitle: Best practices for scalable Redis Query Engine weight: 25 --- -[Vertical scaling of Redis Query Engine]({{}}) requires configuring query performance factors. With careful crafting of search indices and queries, query performance factors allow throughput scaling up to 16X. The following recommendations help queries avoid accessing the keyspace and enable Redis Query Engine to benefit from additional CPUs allocated by query performance factors. +[Vertical scaling of Redis Query Engine]({{}}) requires configuring query performance factors. With careful crafting of search indices and queries, query performance factors allow throughput scaling up to 16X. The following recommendations can help optimize your indices and queries to maximize the performance benefits from additional CPUs allocated by query performance factors. ## Best candidates for query performance factor improvements @@ -22,13 +22,29 @@ weight: 25 - [Vector]({{}}) -- Result set types: + - [Numeric]({{}}) + + - [Geo]({{}}) + +- Result set types: - Small result sets - Document subsets that are indexed in their [non-normalized]({{}}) form -## Indexing best practices +## Best practices + +If query performance factors have not boosted the performance of your queries as much as expected: + +1. Verify your index includes all queried and returned fields. + +1. Identify and avoid query [anti-patterns]({{}}) that limit scalability. + +1. Follow best practices to [improve indexing](#improve-indexing). + +1. Follow best practices to [improve queries](#improve-queries). + +### Improve indexing Follow these best practices for [indexing]({{}}): @@ -40,9 +56,9 @@ Follow these best practices for [indexing]({{}}): +Follow these best practices to optimize [queries]({{}}): - Specify the result set fields in the `RETURN` or `LOAD` clauses and include them in the index definition. Don’t just return the default result set from [`FT.SEARCH`]({{< relref "commands/ft.search/" >}}) or `LOAD *` from [`FT.AGGREGATE`]({{< relref "commands/ft.aggregate/" >}}). @@ -97,3 +113,94 @@ FT.AGGREGATE jsonidx:profiles '@t:[1299 1299]' LIMIT 0 10 DIALECT 3 ``` + +## Performance results + +The following benchmarks show the performance improvements for different query types achieved with query performance factors. Vector, tag, and text queries strongly benefit, while numeric and geographic queries show more limited improvements. + +### Vector schema type + +#### Vector ingest + +| Shards | Threads per shard | CPUs | Speedup factor | +|--------|-------------------|------|----------------| +| 1 | 0 | 1 | 0 | +| 6 | 0 | 6 | 6.6 | +| 1 | 6 | 6 | 2.5 | +| 2 | 6 | 12 | 6.1 | +| 4 | 6 | 24 | 24.3 | + +#### Vector query + +| Shards | Threads per shard | CPUs | Speedup factor | +|--------|-------------------|------|----------------| +| 1 | 0 | 1 | 0 | +| 6 | 0 | 6 | 0.8 | +| 1 | 6 | 6 | 4.7 | +| 2 | 6 | 12 | 5.1 | +| 4 | 6 | 24 | 5.6 | + +### Tag schema type + +| Worker threads | % change | +|----------------|----------| +| 0 | 0 | +| 6 | 135.88 | + +### Text schema type + +#### Two-word union queries + +| Worker threads | Queries per second | % change | +|----------------|--------------------|----------| +| 0 | 188 | 0 | +| 6 | 1,072 | 470 | +| 12 | 1,995 | 961 | +| 18 | 2,834 | 1,407 | + +#### Two-word intersection queries + +| Worker threads | Queries per second | % change | +|----------------|--------------------|----------| +| 0 | 2,373 | 0 | +| 6 | 12,396 | 422 | +| 12 | 17,506 | 638 | +| 18 | 19,764 | 733 | + +#### Simple one-word match + +| Worker threads | Queries per second | % change | +|----------------|--------------------|----------| +| 0 | 476 | 0 | +| 6 | 2,837 | 496 | +| 12 | 5,292 | 1,012 | +| 18 | 7,512 | 1,478 | + +### Numeric schema type + +| Worker threads | Queries per second | % change | +|----------------|--------------------|----------| +| 0 | 33,584 | 0 | +| 1 | 36,993 | 10.15 | +| 3 | 36,504 | 8.69 | +| 6 | 36,897 | 9.86 | + +### Geo schema type + +#### Geo queries without UNF + +| Worker threads | Queries per second | % change | +|----------------|--------------------|----------| +| 0 | 48 | 0 | +| 6 | 96 | 100 | +| 12 | 96 | 100 | +| 18 | 98 | 104 | + +#### Geo queries with UNF + +| Worker threads | Queries per second | % change | +|----------------|--------------------|----------| +| 0 | 61 | 0 | +| 6 | 227 | 272 | +| 12 | 217 | 256 | +| 18 | 217 | 256 | From 443564724c54f56144cdc70c2084e8f7f4c4456a Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Tue, 10 Jun 2025 17:07:45 -0500 Subject: [PATCH 2/3] Style guide fix for indexes --- .../search/scalable-query-best-practices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md b/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md index 7c64df00a0..58124d1a56 100644 --- a/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md +++ b/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md @@ -10,7 +10,7 @@ linkTitle: Best practices for scalable Redis Query Engine weight: 25 --- -[Vertical scaling of Redis Query Engine]({{}}) requires configuring query performance factors. With careful crafting of search indices and queries, query performance factors allow throughput scaling up to 16X. The following recommendations can help optimize your indices and queries to maximize the performance benefits from additional CPUs allocated by query performance factors. +[Vertical scaling of Redis Query Engine]({{}}) requires configuring query performance factors. With careful crafting of search indexes and queries, query performance factors allow throughput scaling up to 16X. The following recommendations can help optimize your indexes and queries to maximize the performance benefits from additional CPUs allocated by query performance factors. ## Best candidates for query performance factor improvements From 0f71158b863ca05489e998ae95b629a53b15e8fb Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Wed, 2 Jul 2025 15:07:57 -0500 Subject: [PATCH 3/3] Fixed broken relrefs after AI/search reorg --- .../search/scalable-query-best-practices.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md b/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md index 156f0b2ab9..c50aeb9a1e 100644 --- a/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md +++ b/content/operate/oss_and_stack/stack-with-enterprise/search/scalable-query-best-practices.md @@ -22,9 +22,9 @@ weight: 25 - [Vector]({{}}) - - [Numeric]({{}}) + - [Numeric]({{}}) - - [Geo]({{}}) + - [Geo]({{}}) - Result set types: @@ -38,7 +38,7 @@ If query performance factors have not boosted the performance of your queries as 1. Verify your index includes all queried and returned fields. -1. Identify and avoid query [anti-patterns]({{}}) that limit scalability. +1. Identify and avoid query [anti-patterns]({{}}) that limit scalability. 1. Follow best practices to [improve indexing](#improve-indexing).