documentation

Author: miguelgrinberg

docs/reference/dsl_how_to_guides.md

@@ -1425,6 +1425,125 @@ print(response.took)
 If you want to inspect the contents of the `response` objects, just use its `to_dict` method to get access to the raw data for pretty printing.
 
 
+## ES|QL Queries
+
+When working with `Document` classes, you can use the ES|QL query language to retrieve documents. Consider the following `Employee` document definition:
+
+```python
+from elasticsearch.dsl import Document, InnerDoc, M
+
+class Address(InnerDoc):
+    address: M[str]
+    city: M[str]
+    zip_code: M[str]
+
+class Employee(Document):
+    emp_no: M[int]
+    first_name: M[str]
+    last_name: M[str]
+    height: M[float]
+    still_hired: M[bool]
+    address: M[Address]
+
+    class Index:
+        name = 'employees'
+```
+
+All instances of `Document` include the `esql_from()` method, which creates a base query for the associated index. The following example creates a base query for employees:
+
+```python
+query = Employee.esql_from()
+```
+
+This query includes a `FROM` command with the index name, and a `KEEP` command that retrieves all the document attributes. To run this query, you can pass it to the `esql_execute()` method:
+
+```python
+for emp in Employee.esql_execute(query):
+    print(f"{emp.name} from {emp.address.city} is {emp.height:.2f}m tall")
+```
+
+In this example, the `esql_execute()` class method executes the query and returns all the documents in the index, up to the maximum of 1000 results allowed by ES|QL. Here is a possible output from this example:
+
+```
+Kevin Macias from North Robert is 1.60m tall
+Drew Harris from Boltonshire is 1.68m tall
+Julie Williams from Maddoxshire is 1.99m tall
+Christopher Jones from Stevenbury is 1.98m tall
+Anthony Lopez from Port Sarahtown is 2.42m tall
+Tricia Stone from North Sueshire is 2.39m tall
+Katherine Ramirez from Kimberlyton is 1.83m tall
+...
+```
+
+To search for specific documents you can extend the base query with additional ES|QL commands that narrow the search. The next example searches for documents that include specific employees that are 2m tall or more, sorted by their last name. It also limits the results 4:
+
+```python
+query = (
+    Employee.esql_from()
+    .where(Employee.height >= 2)
+    .sort(Employee.last_name)
+    .limit(4)
+)
+```
+
+When running this query, with the same for-loop shown above, possible results would be:
+
+```
+Michael Adkins from North Stacey is 2.48m tall
+Kimberly Allen from Toddside is 2.24m tall
+Crystal Austin from East Michaelchester is 2.30m tall
+Rebecca Berger from Lake Adrianside is 2.40m tall
+```
+
+### Additional fields
+
+ES|QL provides a few ways to add new fields to a query, for example through the `EVAL` command. The following example shows a query that adds a calculated field:
+
+```python
+from elasticsearch.esql import E, functions
+
+query = (
+    Employee.esql_from()
+    .eval(height_cm=functions.round(Employee.height * 100))
+    .where(E("height_cm") >= 200)
+    .sort(Employee.last_name)
+    .limit(10)
+)
+```
+
+In this example we are adding the height converted to centimeters to the query. This value is available to other query clauses, but it is not returned when the query is executed as shown in all previous examples.
+
+Note how in the `WHERE` clause the new field is given as `E("height_cm")`. The `E()` wrapper tells the query builder that the argument is an ES|QL field name and not a string literal. This is done automatically for document fields that are given as class attributes, such as `Employee.first_name`, so `E()` is only needed for fields that are not in the document.
+
+To receive the additional fields that are not part of the document in the query results, the `return_additional=True` argument can be passed to `esql_execute()`. When using this option, the results are given as a tuple with the document as first element, and a dictionary with the additional fields as second element:
+
+```python
+for emp, additional in Employee.esql_execute(query, return_additional=True):
+    print(emp.name, additional)
+```
+
+Example output:
+
+```
+Michael Adkins {'height_cm': 248.0}
+Kimberly Allen {'height_cm': 224.0}
+Crystal Austin {'height_cm': 230.0}
+Rebecca Berger {'height_cm': 240.0}
+Katherine Blake {'height_cm': 214.0}
+Edward Butler {'height_cm': 246.0}
+Steven Carlson {'height_cm': 242.0}
+Mark Carter {'height_cm': 240.0}
+Joseph Castillo {'height_cm': 229.0}
+Alexander Cohen {'height_cm': 245.0}
+```
+
+### Missing fields
+
+The base query returned by the `esql_from()` method includes a `KEEP` command with the complete list of fields that are part of the document. If the query adds a `DROP` command, or a more restrictive `KEEP`, then the `esql_execute()` method will raise an exception, because it will not be able construct the document instances.
+
+To prevent errors, it is recommended that the `KEEP` and `DROP` commands are not used when working with `Document` instances.
+
+If a query has missing fields, it can be forced to execute without errors by passing the `ignore_missing_fields=True` argument to `esql_execute()`. When this option is used, returned documents will have any missing fields set to `None`.
 
 ## Using asyncio with Elasticsearch Python DSL [asyncio]
 

docs/reference/dsl_tutorials.md

@@ -83,15 +83,15 @@ Let’s have a simple Python class representing an article in a blogging system:
 
 ```python
 from datetime import datetime
-from elasticsearch.dsl import Document, Date, Integer, Keyword, Text, connections
+from elasticsearch.dsl import Document, Date, Integer, Keyword, Text, connections, mapped_field
 
 # Define a default Elasticsearch client
 connections.create_connection(hosts="https://localhost:9200")
 
 class Article(Document):
     title: str = mapped_field(Text(analyzer='snowball', fields={'raw': Keyword()}))
     body: str = mapped_field(Text(analyzer='snowball'))
-    tags: str = mapped_field(Keyword())
+    tags: list[str] = mapped_field(Keyword())
     published_from: datetime
     lines: int
 
@@ -216,6 +216,20 @@ response = ubq.execute()
 As you can see, the `Update By Query` object provides many of the savings offered by the `Search` object, and additionally allows one to update the results of the search based on a script assigned in the same manner.
 
 
+## Querying with ES|QL
+
+The DSL module is also integrated with the ES|QL query builder. Using `Article` document from above, we can search for articles that include `"world"` with the following ES|QL query:
+
+```python
+from elasticsearch.esql import functions
+
+query = Article.esql_from().where(functions.match(Article.title, 'world')).limit(10)
+for a in Article.esql_execute(query):
+    print(a.title)
+```
+
+Review the [ES|QL Query Builder section](esql-query-builder.md) to learn more about building ES|QL queries in Python.
+
 ## Migration from the standard client [_migration_from_the_standard_client]
 
 You don’t have to port your entire application to get the benefits of the DSL module, you can start gradually by creating a `Search` object from your existing `dict`, modifying it using the API and serializing it back to a `dict`: