From 63505c7c22d03fe42c10a1505b7e60b5746ea133 Mon Sep 17 00:00:00 2001 From: Arad Zilberstein Date: Mon, 21 Jul 2025 08:27:19 +0000 Subject: [PATCH 1/2] Add more configurations Signed-off-by: Arad Zilberstein --- topics/search.md | 57 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/topics/search.md b/topics/search.md index cae85cb8..a29be8f0 100644 --- a/topics/search.md +++ b/topics/search.md @@ -181,6 +181,21 @@ The following list of configurations can be passed to the `loadmodule` command: sizes may improve performance by enhancing CPU cache efficiency, but come at the cost of increased memory usage due to pre-allocation for potential future growth. (Default: 10K) 5. `--log-level` Controls the log verbosity level. Possible values are: `debug`, `verbose`, `notice` and `warning`. (Default: Valkey's log level) +6. `--query-string-bytes`: (Optional) Controls the maximum length in bytes of the query string for FT.SEARCH commands. (Default: 10240) +7. `--query-string-depth`: (Optional) Controls the maximum depth of query string parsing for FT.SEARCH commands. (Default: 1000) +8. `--query-string-terms-count`: (Optional) Controls the maximum number of terms allowed in a query string. (Default: 16) +9. `--max-indexes`: (Optional) Controls the maximum number of search indexes that can be created. (Default: 10) +10. `--max-prefixes`: (Optional) Controls the maximum number of key prefixes per index during FT.CREATE. (Default: 8) +11. `--max-tag-field-length`: (Optional) Controls the maximum length of tag field identifiers during index creation. (Default: 256) +12. `--max-numeric-field-length`: (Optional) Controls the maximum length of numeric field identifiers during index creation. (Default: 128) +13. `--max-vector-attributes`: (Optional) Controls the maximum number of attributes per index. (Default: 50) +14. `--max-vector-dimensions`: (Optional) Controls the maximum number of dimensions for vector indexes. (Default: 32768) +15. `--max-vector-m`: (Optional) Controls the maximum M parameter for HNSW algorithm. (Default: 2000000) +16. `--max-vector-ef-construction`: (Optional) Controls the maximum EF_CONSTRUCTION parameter for HNSW algorithm. (Default: 4096) +17. `--max-vector-ef-runtime`: (Optional) Controls the maximum EF_RUNTIME parameter for HNSW algorithm. (Default: 4096) +18. `--max-vector-knn`: (Optional) Controls the maximum K value for K-nearest neighbor searches. (Default: 128) +19. `--max-search-result-record-size`: (Optional) Controls the maximum size in bytes for search result records. (Default: 5242880) +20. `--max-search-result-fields-count`: (Optional) Controls the maximum number of fields in search result records. (Default: 500) ### Runtime configuration @@ -190,4 +205,46 @@ The following list of configurations can be modified at runtime using the `CONFI sizes may improve performance by enhancing CPU cache efficiency, but come at the cost of increased memory usage due to pre-allocation for potential future growth. (Default: 10K) +2. `search.query-string-bytes`: Controls the maximum length in bytes of the query string for FT.SEARCH commands. +3. `search.query-string-depth`: Controls the maximum depth of query string parsing for FT.SEARCH commands, preventing + overly complex nested queries. (Default: 1000, Min: 1, Max: UINT_MAX) + +4. `search.query-string-terms-count`: Controls the maximum number of terms (nodes in the predicate tree) allowed in a + query string for FT.SEARCH commands. (Default: 16, Min: 1, Max: 32) + +5. `search.max-indexes`: Controls the maximum number of search indexes that can be created. + (Default: 10, Min: 1, Max: 10) + +6. `search.max-prefixes`: Controls the maximum number of key prefixes that can be specified per index during FT.CREATE. + (Default: 8, Min: 1, Max: 16) + +7. `search.max-tag-field-length`: Controls the maximum length of tag field identifiers during index creation. + (Default: 256, Min: 1, Max: 10000) + +8. `search.max-numeric-field-length`: Controls the maximum length of numeric field identifiers during index creation. + (Default: 128, Min: 1, Max: 256) + +9. `search.max-vector-attributes`: Controls the maximum number of attributes that can be defined per index. + (Default: 50, Min: 1, Max: 100) + +10. `search.max-vector-dimensions`: Controls the maximum number of dimensions allowed for vector indexes. + (Default: 32768, Min: 1, Max: 64000) + +11. `search.max-vector-m`: Controls the maximum M parameter for HNSW algorithm, which affects the connectivity of the graph. + Higher values improve recall but increase memory usage. (Default: 2000000, Min: 1, Max: 2000000) + +12. `search.max-vector-ef-construction`: Controls the maximum EF_CONSTRUCTION parameter for HNSW algorithm during index + building. Higher values improve index quality but slow down construction. (Default: 4096, Min: 1, Max: 4096) + +13. `search.max-vector-ef-runtime`: Controls the maximum EF_RUNTIME parameter for HNSW algorithm during search. Higher + values improve recall but slow down search. (Default: 4096, Min: 1, Max: 4096) + +14. `search.max-vector-knn`: Controls the maximum K value for K-nearest neighbor searches. + (Default: 128, Min: 1, Max: 1000) + +15. `search.max-search-result-record-size`: Controls the maximum size in bytes for individual search result records. + Records exceeding this limit will be dropped from results. (Default: 5MB, Min: 100, Max: 10MB) + +16. `search.max-search-result-fields-count`: Controls the maximum number of fields allowed in individual search result records. + Records exceeding this limit will be dropped from results. (Default: 500, Min: 1, Max: 1000) From 5108c80d515989d5b339d1904a4ba9cc78169f1f Mon Sep 17 00:00:00 2001 From: Arad Zilberstein Date: Thu, 31 Jul 2025 11:57:44 +0000 Subject: [PATCH 2/2] Add info metrics Signed-off-by: Arad Zilberstein --- topics/search.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/topics/search.md b/topics/search.md index a29be8f0..48c20db1 100644 --- a/topics/search.md +++ b/topics/search.md @@ -167,6 +167,21 @@ The following metrics are added to the `INFO` command's output: - `search_add_subscription_skipped_count`: Count of skipped subscription adding processes - `search_modify_subscription_failure_count`: Count of failed subscription modifications - `search_modify_subscription_successful_count`: Count of successful subscription modifications +- `search_ingest_hash_keys`: Count of hash keys processed for ingestion across all indexes +- `search_ingest_hash_blocked`: Count of currently blocked clients waiting for hash key ingestion operations +- `search_ingest_json_keys`: Count of JSON keys processed for ingestion across all indexes +- `search_ingest_json_blocked`: Count of currently blocked clients waiting for JSON key ingestion operations +- `search_ingest_field_vector`: Count of vector fields processed during ingestion +- `search_ingest_field_numeric`: Count of numeric fields processed during ingestion +- `search_ingest_field_tag`: Count of tag fields processed during ingestion +- `search_ingest_last_batch_size`: Size of the most recent ingestion batch processed +- `search_ingest_total_batches`: Count of total ingestion batches processed +- `search_ingest_total_failures`: Count of ingestion failures across all operations +- `search_coordinator_bytes_out`: Total bytes sent by the coordinator to other nodes +- `search_coordinator_bytes_in`: Total bytes received by the coordinator from other nodes +- `search_coordinator_server_listening_port`: Port number on which the coordinator server is listening +- `search_used_read_cpu`: Average CPU percentage used by the reader thread pool +- `search_used_write_cpu`: Average CPU percentage used by the writer thread pool ## Configuration