Migrate Hybrid Search Labs Notebook from RRF to Retrievers API (Elasticsearch 9.x)

[mridula-s109]

Overview

This PR updates notebooks/search/02-hybrid-search.ipynb to use the new Retrievers API for hybrid search, replacing the deprecated rank={"rrf": {}} syntax with the modern retriever={"rrf": {...}} structure. The notebook is now fully compatible with Elasticsearch 9.x.

Key Changes

• Replaced all usage of the legacy rank field with the new retriever field in search requests.
• Updated hybrid search examples to use the rrf retriever, combining both BM25 and kNN retrievers in the new syntax.
• Modified the pretty_response function and result parsing to use _score and positional rank, reflecting the new API response structure.
• Cleaned up notebook outputs and execution counts for a clean diff.
• Added explanatory markdown on the new retriever API and how ranking works in Elasticsearch 9.x.

Testing & Validation

• Verified that hybrid search queries using the retriever API work as expected on Elasticsearch 9.1.0-SNAPSHOT.
• Confirmed that kNN and standard retrievers are combined using RRF and results are displayed correctly.

Test Output

[gitnotebooks]

Found 1 changed notebook. Review the changes at https://app.gitnotebooks.com/elastic/elasticsearch-labs/pull/459

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull Request Overview

This PR migrates the hybrid search notebook to Elasticsearch 9.x by replacing the deprecated rank syntax with the new Retrievers API.

• Replaced legacy "rank" field with the "retriever" field in search requests.
• Updated the pretty_response function to enumerate results and display both ranking position and _score.
• Refined the search call structure to combine BM25 and kNN retrievers using the new API.

Comments suppressed due to low confidence (1)

notebooks/search/02-hybrid-search.ipynb:208

• [nitpick] The variable name 'id' shadows Python's built-in function. Consider renaming it to 'document_id' to improve clarity.

            id = hit["_id"]

[kderusso]

LGTM, thanks!