v5.5.0 - Feedback wanted on execute_query 📣

[robsdedude]

execute_query

API In Action

In version 5.5.0, we introduce a new experimental API to the driver. This is a simplified API for running queries against a database without needing to get into the detail of sessions and transactions.

Let us see it in action first, just below:

import neo4j


URI = "neo4j://localhost"
AUTH = ("neo4j", "pass")

with neo4j.GraphDatabase.driver(URI, auth=AUTH) as driver:
    records, summary, keys = driver.execute_query("RETURN 42 AS answer")
    print("keys:", keys)
    for record in records:
        print("answer:", record)

    print(f"result available after {summary.result_available_after}ms")

This runs a the cypher query inside a transaction with retries if necessary and returns an EagerResult object. This object is a named tuple with

• records: list of neo4j.Records returned,
• keys: the list of keys present in the records, and
• summary the neo4j.ResultSummary of the query.

Under the hood

A lot happens under the hood:

• the driver will create a session (and dispose of it when done)
• run and retry a transaction function as often as it needs
• get the result and transform it via the provided transformer.

Moreover, it tracks bookmarks through a default BookmarkManager implementation so that subsequent execute_query reads can read the results of former execute_query writes. You can even pass this bookmark manager instance to sessions via driver.session(..., bookmark_manager=driver.query_bookmark_manager), so the causal consistency is preserved across all the driver calls your application makes.

As a consequence, you do not have to worry about all of this! This is why we believe execute_query is a great starting point for new users. It does many things right by default.

And there is more!

Configuration

For more fine-grained scenarios, execute_query accepts extra parameters:

eager_result = driver.execute_query(
      "RETURN $x AS answerA, $y AS answerB",
      parameters_={"x": "x", "y": "nope"},
      routing_=RoutingControl.READERS,  # or for short routing_="r",
      database_="neo4j",
      # I'm too lazy to set up impersonation privileges
      impersonated_user_="some_user",
      bookmark_manager_=some_bookmark_manager,  # or None to disable bookmark management
      y="y",  # takes precedence over "y" in the dictionary
  )

You can configure impersonation, select a specific database, change from the default WRITERS routing mode and/or configure bookmark management.

Beyond EagerResult

EagerResult is a good starting point, as it includes all you need about the results of your query. However, as its name suggests, it loads all records in memory, which is not a great thing to do when your query returns huge results (in size and/or quantity). Fortunately, we left out one configuration option above as there's much to say about it: result_transformer_. It allows you to process the result cursor in any way. By default it's neo4j.Result.to_eager_result.

driver.execute_query("RETURN 42 AS answer")
# is just the same as
driver.execute_query(
    "RETURN 42 AS answer",
    result_transformer_=neo4j.Result.to_eager_result
)

But you can change it to any function (or any callable, really) that takes a neo4j.Result and returns anything. Whatever it returns, will be returned from the execute_query call.

⚠️ Important to note:

• Don't return the result cursor neo4j.Result from the transformer. This is because underlying transaction of the cursor will be closed when execute_query returns. So the cursor will be out of scope and dysfunctional.
• The transformer needs to be idempotent (yield the same result no matter how often it's called). That is because the driver might retry the underlying transaction.

💡 Good to note:
In particular, any method in Python is a function that takes an object of its class as first argument. This means you can use any method of neo4j.Result that has no non-optional parameters as transformer.

For example (requires pandas to be installed):

dataframe = driver.execute_query(
    "RETURN 42 AS answer",
    result_transformer_=neo4j.Result.to_df,
)
print(dataframe)

Feedback wanted

This new API is currently marked as experimental.
What it means is that we are eagerly (no pun intended) waiting for your feedback.
Does it work well in your scenario? Do you wish there was more?
Could some defaults be improved?

Let us know and we will correct course in the next releases!

[robsdedude]

With

• Mature execute_query API #911

the new API will be stabilized. Some names were slightly adjusted. The feature is scheduled to hit GA with version 5.8.0.

[jalakoo]

I love this new API, much simpler to implement!