Skip to content

HTTP Query API Support - #1319

Draft
robsdedude wants to merge 75 commits into
neo4j:6.xfrom
robsdedude:feat/query-api
Draft

HTTP Query API Support#1319
robsdedude wants to merge 75 commits into
neo4j:6.xfrom
robsdedude:feat/query-api

Conversation

@robsdedude

@robsdedude robsdedude commented Jul 7, 2026

Copy link
Copy Markdown
Member

Neo4j servers recently introduced a new HTTP API for querying the database (called Query API): https://neo4j.com/docs/query-api/current/

This PR add support for configuring the driver with a HTTP URL (e.g. https://neo4j.example.com:7474/), which will make the driver run queries and transactions via the HTTP Query API instead of Bolt.

Notes

Please note that:

  • This feature is in preview (see below).
  • For the HTTP support 3rd party libraries (currently urllib3 for sync and aiohttp for async) are required. Install pip install neo4j[http] instead of pip install neo4j.
  • Some features are not supported and likely won't be in the future:
    • home database resolution (i.e., user code must always provide an explicit database name for each session created)
    • the only supported authentication scheme is basic
  • Some features are not yet supported:
    • Transaction metadata and timeouts are being ignored.
    • Some details in the summary (such as neo4j.ResultSummary.server.protocol_info) might be missing or inaccurate.
    • Vector types, the UnsupportedType type, and UUIDs cannot be exchanged with the DBMS.
    • Notifications and errors have fallback GQL status codes and status information.
    • fetch_size has not effect, there is no backpressure mechanism.
    • Notification filters (notifications_min_severity, notifications_disabled_categories, notifications_disabled_classifications) have no effect.

Preview

This feature is in preview. This means that it does not follow semver. It might be changed (or in rare cases be removed) at any time without following the normal deprecation cycle.


Depends on:

Closes: DRIVERS-118

 * Server agent string cache bound to driver instance, not interpreter state
 * Fix using deprecated config of aiohttp
 * Enable TestKit testing
 * Docs: mentions that server agent string is computed & cacahed
The HTTP Query API will send `null` for notifications without a position. To
make the experience the same between using bolt and HTTP, the driver drops
`null` positions via HTTP to make them absent, just like they are via bolt.
Comment thread testkit/testkit.json
{
"testkit": {
"uri": "https://github.com/neo4j-drivers/testkit.git",
"ref": "6.x"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: revert once TestKit PR has been merged:



class _Http(_Direct):
_default_port = 7687

@StephenCathcart StephenCathcart Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_Http uses _default_port = 7687, the bolt port, so http://localhost with no port goes there rather than on 7474:

driver = neo4j.GraphDatabase.driver("http://localhost", auth=AUTH)
driver.address.port  # -> 7687

is that intended, or does https want its own default (7473) as well?

raise
if protocol_error or not connection_failed:
raise
for request in self._requests:

@StephenCathcart StephenCathcart Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_CommitRequest never looks constructed anywhere (i think), so this isinstance can't match. the loop also scans the queued _requests, and send_all() has already popped the one in flight (the bolt version looks at self.responses for CommitResponse instead i believe).

i pointed the driver at a stub that drops the connection on the commit POST and answers the retry, and the write went in twice:

write query submitted : 2 time(s)
commit attempts       : 2
commits dropped       : 1
execute_write returned ['Alice']

ServiceUnavailable.is_retryable() is True where IncompleteCommit's is False, so the transaction function replays a commit with an outcome we don't know. am i reading that right? happy to hand over the stub i used to test it

@StephenCathcart StephenCathcart Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raised this on testkit#720 too, with a test that catches it, so no need for the stub now

BoundedSemaphore: t.TypeAlias = threading.BoundedSemaphore


def acquire_bounded_semaphore(

@StephenCathcart StephenCathcart Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semaphore.acquire(timeout=...) returns False on timeout rather than raising, and the result isn't kept here, so with max_connection_pool_size=1 i get a second connection out of the sync pool:

conn 0: acquired
conn 1: acquired          <- over the limit
close:  ValueError: Semaphore released too many times

the async twin does block, but comes out as a bare TimeoutError rather than ConnectionAcquisitionTimeoutError like the bolt pool. should this be checking the return value, or is this a none issue?

async def next_id(self) -> int:
async with self._lock:
current_id = self._next_id
self._next_id = min((self._next_id + 1) % IdGenerator._MAX, 1)

@StephenCathcart StephenCathcart Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

min keeps this at 1 forever, so every http connection logged for me as [#0001]. did you mean max, to skip 0 on wrap?

Suggested change
self._next_id = min((self._next_id + 1) % IdGenerator._MAX, 1)
self._next_id = max((self._next_id + 1) % IdGenerator._MAX, 1)

Comment thread src/neo4j/_io.py
@dataclasses.dataclass(slots=True, frozen=True, kw_only=True)
class HTTPServerInfo:
neo4j_version: str
parsed_neo4j_version: tuple[int, int] | None = dataclasses.field(

@StephenCathcart StephenCathcart Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i've got a half-memory of a call about parsing the server version and rejecting the older ones that send values as a flat list rather than a list of records, so users get a clear message instead of a parse error. parsed_neo4j_version is computed here but i couldn't find anything reading it, so i may have the wrong end of this.

a stub advertising 5.19.0 with the flat shape gives:

QueryApiHttpError: protocol error: expected list, got: 1

and that class's docstring points people at filing a driver bug. can't recall if I'm remembering this correctly, and whether it still needs changed?

Comment thread docs/source/async_api.rst
Async Driver Construction
=========================
AsyncDriver Construction
========================

@StephenCathcart StephenCathcart Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

async_api.rst doesn't look like it picked up the http changes that api.rst got: no http/https in the valid-URI list or the scheme table, no AsyncHttpDriver autoclass section, and line 454 still reads "the URI scheme is bolt:// or neo4j://". AsyncHttpDriver is exported and carries the whole "Unavailable Features" docstring, so i think none of it renders for async users. are the async docs to do at a later time?

Comment on lines +1462 to +1475
* :attr:`.ResultSummary.result_available_after` and
:attr:`.ResultSummary.result_consumed_after` will always be
:data:`None` because this information is not provided by the server.
TODO: check whether None or 0!!
* :attr:`.ServerInfo.agent` is being computed from the DBMS's
advertised version. Further, it is being cached to reduce
round-trips and overloading the DBMS's HTTP endpoints.
* :attr:`.ResultSummary.query_type` will always be :data:`None`.

* Transmitting and receiving :class:`Vector` values is currently not
supported.

* The only supported auth scheme (see :ref:`auth-ref`) is ``"basic"`` and
``"bearer"``.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* :attr:`.ResultSummary.result_available_after` and
:attr:`.ResultSummary.result_consumed_after` will always be
:data:`None` because this information is not provided by the server.
TODO: check whether None or 0!!
* :attr:`.ServerInfo.agent` is being computed from the DBMS's
advertised version. Further, it is being cached to reduce
round-trips and overloading the DBMS's HTTP endpoints.
* :attr:`.ResultSummary.query_type` will always be :data:`None`.
* Transmitting and receiving :class:`Vector` values is currently not
supported.
* The only supported auth scheme (see :ref:`auth-ref`) is ``"basic"`` and
``"bearer"``.
* :attr:`.ResultSummary.result_available_after` and
:attr:`.ResultSummary.result_consumed_after` will always be
:data:`None` because this information is not provided by the server.
* :attr:`.ServerInfo.agent` is being computed from the DBMS's
advertised version. Further, it is being cached to reduce
round-trips and overloading the DBMS's HTTP endpoints.
* :attr:`.ResultSummary.query_type` will always be :data:`None`.
* Transmitting and receiving :class:`Vector` values is currently not
supported.
* The only supported auth schemes (see :ref:`auth-ref`) are ``"basic"`` and
``"bearer"``.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is -> are, and removed the TODO as I "think" it's None - worth checking

@property
@abc.abstractmethod
def supports_notification_filtering(self) -> bool:
"""Whether the connection version supports re-authentication."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""Whether the connection version supports re-authentication."""
"""Whether the connection version supports notification filtering."""

Comment thread docs/source/api.rst
Comment on lines +1165 to +1166
A string value must be provided connected via
``http://`` or ``https://`` scheme.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A string value must be provided connected via
``http://`` or ``https://`` scheme.
A string value must be provided when connected via
``http://`` or ``https://`` scheme.

return await self._fetch_unguarded(query_api)
except Exception as e:
log.warning(
"[#%04X] _: Failed to fetch server info: %r",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"[#%04X] _: Failed to fetch server info: %r",
"[#%04X] _: Failed to fetch server info: %r",

Comment on lines +554 to +560
def _compute_repr(self) -> str:
fields_repr = (
f"{key!r}: {value!r}" for key, value in self._iter_sanintized()
)
return f"{{{', '.join(fields_repr)}}}"

def _iter_sanintized(self) -> t.Generator[tuple[str, str]]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _compute_repr(self) -> str:
fields_repr = (
f"{key!r}: {value!r}" for key, value in self._iter_sanintized()
)
return f"{{{', '.join(fields_repr)}}}"
def _iter_sanintized(self) -> t.Generator[tuple[str, str]]:
def _compute_repr(self) -> str:
fields_repr = (
f"{key!r}: {value!r}" for key, value in self._iter_sanitized()
)
return f"{{{', '.join(fields_repr)}}}"
def _iter_sanitized(self) -> t.Generator[tuple[str, str]]:

Comment thread docs/source/api.rst
.. _http-driver-ref:

HttpDriver
===========

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
===========
==========

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ocd

@StephenCathcart StephenCathcart left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than open questions looks good to me 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants