fix(NODE-6878): ensure documents are cleared correctly to handle emptyGe…

[italojs]

Jira issue: https://jira.mongodb.org/browse/NODE-6878

MongoDB Driver: documents?.clear() is not a function error in AbstractCursor.rewind()

Issue Description

When using the MongoDB Node.js driver, there's an issue in the rewind() method of AbstractCursor. The method attempts to call this.documents?.clear() without checking if the clear() method exists on the documents object.

In certain scenarios, particularly when the MongoDB driver uses an optimization called emptyGetMore to avoid unnecessary server calls, this.documents is set to a simplified object literal that lacks the clear() method:

CursorResponse.emptyGetMore = {
    id: new bson_1.Long(0),
    length: 0,
    shift: () => null
    // Note: no clear() method defined
};

This causes an error when rewind() is called on cursors that have this optimization applied:

rewind() {
    if (!this.initialized) {
        return;
    }
    this.cursorId = null;
    this.documents?.clear(); // <-- This line causes the error
    this.isClosed = false;
    this.isKilled = false;
    this.initialized = false;
    // ...
}

Impact

This issue can cause applications to crash when:

1. Using cursor operations that internally call rewind()
2. Working with cursors that have been optimized using emptyGetMore
3. Attempting to reset or reuse cursors after they've been exhausted

Reproduction Details

In this repository, the reproduction.js script demonstrates this issue by:

1. Creating a MongoDB cursor with a small batch size
2. Consuming all documents to exhaust the cursor
3. Simulating the emptyGetMore optimization by replacing the documents object with one that lacks a clear() method
4. Attempting to rewind the cursor, which triggers the error
5. Trying to reuse the cursor, which would normally work if rewind() succeeded

Running the Reproduction

# Install dependencies
npm i

# Run the reproduction script
node reproduction.js

Expected output will show the error:

Rewind error: this.documents?.clear is not a function
Error name: TypeError

Proposed Solution

The issue can be fixed by modifying the rewind() method to safely handle cases where documents doesn't have a clear() method. Here's the proposed fix:

rewind() {
    if (!this.initialized) {
        return;
    }
    this.cursorId = null;
    
    // Check if documents exists and has a clear method before calling it
    if (this.documents && typeof this.documents.clear === 'function') {
        this.documents.clear();
    } else {
        // If documents is the emptyGetMore object or doesn't have the clear() method
        // Simply set it to null to force reinitialization
        this.documents = null;
    }
    
    this.isClosed = false;
    this.isKilled = false;
    this.initialized = false;
    // ... rest of the method
}

Why This Solution Works

1. Safety: The solution checks if clear() exists before calling it
2. Functionality: Setting documents to null achieves the same goal as clear() - it resets the cursor's document buffer
3. Compatibility: The solution maintains compatibility with both regular cursors and optimized emptyGetMore cursors
4. Performance: The solution doesn't introduce any significant overhead

References

• Meteor's AsynchronousCursor._rewind() implementation
• Meteor's issue

[baileympearson]

Hey @italojs ,

Thanks for the submission and good catch! We can reproduce this issue as well. The culprit behind this issue seems to be a Typescript cast as unknown as CursorResponse:

node-mongodb-native/src/cmap/wire_protocol/responses.ts

Line 237 in 1f8e01f

} as unknown as CursorResponse;

. Your solution seems to make the error go away but we're concerned that other errors might arise in the future and be squashed by the same Typescript cast.

We think that a more future-proof approach might be to directly instantiate a CursorResponse:

static get emptyGetMore(): CursorResponse {
  return new CursorResponse(
    serialize({ ok: 1, cursor: { id: 0, nextBatch: [] } })
  )
}

Would you be willing to make this change? The static getter is important here to avoid serialization at the module-level and only serialize when needed.

Also, we'll need a test for this scenario to confirm that the behavior has been fixed. Would you be willing to add a regression test? I believe find.test.js is a good place for a test, and we can trigger the error by:

1. testing on sharded clusters
2. insert a few documents into a collection
3. create a cursor with a limit and batch size, where limit is a multiple of the batch size and also less than the number of documents inserted
4. iterate the cursor limit + 1 times
5. rewind the cursor

Without the changes in this PR, the above throws an error. I'm also happy to make these changes if you'd prefer, just let me know.

Thanks again!

[W-A-James]

@italojs Seems like you're failing these tests

[italojs]

@W-A-James, How can I see the CI output tests? It's passing on my machine, but I would like to know if it's an environmental issue or a flaky error.

      #count()
        ✔ returns the number of documents
      #forEach()
        15) iterates all the documents
      #toArray()
        16) returns an array with all documents
      #next()
        17) is callable without blocking
      #stream()
        18) creates a node stream that emits data events
      #explain()
        ✔ returns an explain document
  
   

In the GitHub CI, it says I must have to a mongodb.com account. I'm a little bit lot on it to be honest
screenshot in Portuguese

[W-A-James]

@italojs Apologies for the delay, here are the relevant logs: https://gist.github.com/W-A-James/080802d7f36ef774f06641a1b13ffc75

It doesn't seem like we'd be able to grant you access to our CI platform

[italojs]

@W-A-James @baileympearson Ok, can you trigger the CI again, please?
I ran a few local tests for ./test/tools/cluster_setup.sh sharded_cluster', ./test/tools/cluster_setup.sh replica_set, and ./test/tools/cluster_setup.sh server, but a few tests were skipped; I'm not sure why.
hope all tests pass in the CI.