Download full collection via collection.query

[canek13]

I want to download full collection from milvus base via query method:
https://milvus.io/api-reference/pymilvus/v2.4.x/MilvusClient/Vector/query.md
But it is said that we have limitations for offset and lmiit.

But if the collection is about 3.2M samples. It is much larger than 16384...
Trying to download with this code:

from pymilvus import connections, Collection

# Connect to Milvus
c = connections.connect(
    alias="default",  # Connection name
    host="some-host",  # Replace with your Milvus host
    port="some-port",       # Replace with your Milvus port
    user="some-user",     # Replace with your username if needed
    password="pass", # Replace with your password if needed,
    db_name="db_name"
)

collection_name = "collection_name"

collection = Collection(name=collection_name)
collection.load()

expr = ""
query_collection = collection.query(
  expr, 
  output_fields=milvus_collection_field_names, 
  offset=1,
  limit=16385,
)

Drops me an error
MilvusException: (code=65535, message=invalid max query result window, (offset+limit) should be in range [1, 16384], but got 16386)

So how can i manage to download full collection ?

[yhmo]

The max value of limit is 16384 in milvus. And if the size of the result exceeds RPC transfer limit, you will get the error "query results exceed the limit size". So, a single call of query() cannot fetch all data of a large collection.

The correct way is using the QueryIterator to fetch data batch by batch.
https://milvus.io/docs/get-and-scalar-query.md#Overview