[2.x] Add incremental mailbox synchronization

[stevebauman]

This adds the IMAP primitives needed to keep a local mailbox in sync: fetching changes since a saved checkpoint, discovering deleted messages, and updating flags without overwriting changes made by another client.

Incremental synchronization

• Add CONDSTORE and QRESYNC selection options, with QRESYNC enabled before selection.
• Return a SelectionResult containing UIDVALIDITY, UIDNEXT, HIGHESTMODSEQ, permanent flags, and any changes returned during selection. NOMODSEQ is exposed when persistent modification sequences are unavailable.
• Add MessageData::modSequence() and changesSince() for fetching changed messages directly by UID, including optional VANISHED responses.
• Support CHANGEDSINCE and VANISHED directly through fetch() with typed modifiers. Ordinary and modified fetches share the same FetchResult, containing parsed message data, vanished UIDs, and raw responses.
• Support conditional updates directly through store() with an UnchangedSince modifier. Ordinary and conditional updates return StoreResult, preserving returned message data and exposing conflicting identifiers through MODIFIED.
• Preserve the selected folder when starting another message query, and clear connection-specific state when disconnecting or cloning a mailbox.
• Route folder examination through Mailbox::examine(), clearing the cached selection before EXAMINE so the next query selects the correct folder again.
• Return an empty FetchResult immediately when changesSince() receives an empty UID array.
• Require UID FETCH results to include the requested attributes, retaining unsolicited updates in the raw responses.
• Expand compact VANISHED and MODIFIED ranges without allocating a second full-sized array.

Applications remain responsible for storing checkpoints, handling UIDVALIDITY changes, and falling back when the server does not support these extensions.

Extensible fetched data

FetchedMessageData now retains the returned attributes instead of copying a fixed list into constructor properties. It provides has(), get(), and immutable merge() alongside the existing typed accessors.

This preserves arbitrary body sections, partial offsets, extension attributes, nested lists, and explicit NIL values. It also distinguishes an attribute that was not fetched from one that was returned empty.

Message retains the complete data container, including through serialization. Lazy fetches merge into the existing data, and previously fetched complete body parts can be reused when peeking.

Command parameter fixes

• Preserve ID field names and NIL values, framing multiline strings as literals.
• Always send APPEND messages as literals, including messages without line breaks.
• Expand ALL, FAST, and FULL fetch macros, and recognize PEEK and partial-body response attributes when fetching by message number.
• Support SASL challenge/response authentication through an Authenticator contract and Authentication coordinator, with XOAuth2 available through the mailbox's xoauth2 configuration. Initial responses are opt-in for servers supporting SASL-IR, and outgoing credentials are redacted.
• Support LIST selection options, multiple patterns, and additional return responses such as STATUS.
• Support QRESYNC sequence-match pairs alongside known UIDs.

V2 API changes

• Connection SELECT and EXAMINE return SelectionResult; raw responses remain available through responses(). SELECT, EXAMINE, and STATUS consistently default to INBOX.
• FETCH, STORE, COPY, and MOVE take the message set first. A set can be an ID, an array of IDs, or a string such as '1:3,7:*'; the separate from/to arguments are removed. UID EXPUNGE also accepts a string set.
• STORE is explicitly flags-only; the generic item argument is removed.
• SEARCH and SORT accept criteria and an explicit charset option. SEARCH omits CHARSET by default; SORT defaults to UTF-8.
• LIST accepts pattern, selection, and return options, preserving all untagged responses. The folder repository filters folder entries itself.
• STATUS calls its requested attributes items and omits RECENT from the default request. IMAP4rev1 callers can still request RECENT explicitly.
• Connection AUTHENTICATE accepts a SASL mechanism and optional initial response instead of a username/token pair. The Authentication coordinator accepts an Authenticator and manages the challenge/response exchange.
• Connection fetch() returns FetchResult instead of ResponseCollection. The separate fetchChanges() method is removed; changesSince() is a query convenience method built on fetch().
• Connection store() returns StoreResult instead of ResponseCollection and accepts typed modifiers instead of a separate storeConditionally() method. Adding flags remains the default; mode: null replaces flags and mode: '-' removes them. StoreResult::modified() returns identifiers matching the command's addressing mode, replacing the UID-specific modifiedUids() alias.
• Rename ImapFetchIdentifier to ImapIdentifier and support it consistently on FETCH, STORE, SEARCH, SORT, COPY, and MOVE. UIDs remain the default; EXPUNGE still accepts an optional UID set.
• Rename connection quota() and quotaRoot() to getQuota() and getQuotaRoot(). Preserve both QUOTAROOT and QUOTA responses without changing the folder-level quota API.
• LOGOUT waits for completion and closes the local connection, including on failure. ID, DONE, and EXPUNGE documentation now reflects their protocol roles.
• Remove the connection-level uid(), bodyText(), bodyHeader(), bodyStructure(), bodyPart(), flags(), and size() shortcuts in favor of fetch(). Message-level convenience methods remain unchanged, including lazy loading, caching, and PEEK behavior.
• Folder and mailbox selection return SelectionResult and accept selection options.
• FetchedMessageData accepts an attribute array.
• Message is constructed with new Message($folder, $data), and data() exposes the fetched attributes.
• Message array and JSON output use the complete IMAP attribute map instead of the previous fixed lowercase fields.

For example:

use DirectoryTree\ImapEngine\Selection\QuickResync;

$selection = $folder->select(
    options: new QuickResync($uidValidity, $highestModSequence, $knownUids),
);

$changes = $selection->changes();

foreach ($changes->messages() as $data) {
    $uid = $data->uid();
    $flags = $data->flags();
    $modSequence = $data->modSequence();
}

$vanishedUids = $changes->vanishedUids();

Before applying these changes, the application should compare the returned UIDVALIDITY with its saved value. It should only save the new checkpoint after successfully applying the synchronization results.

To fetch changes directly on the connection:

use DirectoryTree\ImapEngine\Fetch\ChangedSince;

$connection->enable('QRESYNC');
$connection->select('INBOX');

$result = $connection->fetch(
    $knownUids,
    ['FLAGS', 'MODSEQ'],
    modifiers: new ChangedSince($highestModSequence, vanished: true),
);

$messages = $result->messages();
$vanishedUids = $result->vanishedUids();
$responses = $result->responses();

CHANGEDSINCE requires CONDSTORE support (also provided by QRESYNC). Requesting VANISHED additionally requires UID FETCH and QRESYNC to be enabled on the connection. New modifiers can implement FetchModifier without adding another fetch method.

Conditional flag updates follow the same pattern:

use DirectoryTree\ImapEngine\Store\UnchangedSince;

$result = $connection->store(
    $knownUids,
    ['\Seen'],
    modifiers: new UnchangedSince($highestModSequence),
);

$conflictingUids = $result->modified();

The set-first connection API follows the same ordering across message operations:

$connection->fetch('1:*', ['UID', 'FLAGS']);
$connection->store([7, 8], ['\\Seen']);
$connection->copy([7, 8], 'Archive');
$connection->move([7, 8], 'Trash');

Higher-level message and query APIs keep their existing argument order. Extension options still require the corresponding server capabilities.

[Copilot]

🟡 Changes recommended

FETCH filtering and mailbox selection tracking can return incorrect data, while empty synchronization sets generate invalid commands.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds incremental IMAP synchronization and modernizes the v2 connection API.

Changes:

• Adds CONDSTORE/QRESYNC synchronization and conditional STORE support.
• Preserves complete fetched attributes and introduces typed results.
• Updates authentication, command parameters, selection state, and tests.

File summaries

File	Description
tests/Unit/Support/StrTest.php	Tests sequence parsing and literals.
tests/Unit/MessageTest.php	Updates message construction tests.
tests/Unit/MessageQueryTest.php	Tests lookup and APPEND behavior.
tests/Unit/MessageDataTest.php	Tests MODSEQ fetch items.
tests/Unit/IncrementalSyncTest.php	Covers incremental synchronization.
tests/Unit/FolderTest.php	Tests folder examination.
tests/Unit/FetchedMessageDataTest.php	Covers attribute preservation and caching.
tests/Unit/Connection/ImapConnectionTest.php	Updates connection API tests.
tests/Unit/Connection/ImapConnectionParametersTest.php	Tests command parameter handling.
tests/Unit/Connection/ImapConnectionOperationsTest.php	Tests message operations and results.
tests/Unit/Connection/ImapConnectionAuthenticationTest.php	Tests SASL authentication.
tests/Integration/FoldersTest.php	Updates STATUS expectations.
src/Vanished.php	Models VANISHED responses.
src/Testing/FakeMessageQuery.php	Adds fake incremental fetching.
src/Testing/FakeMessage.php	Adds fake modification sequences.
src/Testing/FakeMailbox.php	Supports selection results and ENABLE.
src/Testing/FakeFolder.php	Updates fake folder selection.
src/Support/Str.php	Adds literal-list and sequence parsing.
src/StoreResult.php	Models STORE results.
src/StoreModifier.php	Defines STORE modifiers.
src/Store/UnchangedSince.php	Adds UNCHANGEDSINCE.
src/SelectionResult.php	Models SELECT/EXAMINE metadata.
src/SelectionOption.php	Defines selection options.
src/Selection/QuickResync.php	Adds QRESYNC parameters.
src/Selection/CondStore.php	Adds CONDSTORE selection.
src/MessageQueryInterface.php	Extends query synchronization API.
src/MessageQuery.php	Implements changed-message fetching.
src/MessageInterface.php	Exposes modification sequences.
src/MessageData/Attribute.php	Adds MODSEQ attribute.
src/MessageData.php	Adds MODSEQ factory.
src/Message.php	Stores and merges fetched attributes.
src/MailboxInterface.php	Extends mailbox selection API.
src/Mailbox.php	Tracks selection and enabled capabilities.
src/FolderRepository.php	Filters extended LIST responses.
src/FolderInterface.php	Returns typed selection results.
src/Folder.php	Updates selection and quota handling.
src/FileMessage.php	Implements modification sequence accessor.
src/FetchResult.php	Models FETCH results.
src/FetchModifier.php	Defines FETCH modifiers.
src/FetchedMessageData.php	Preserves arbitrary fetched attributes.
src/Fetch/ChangedSince.php	Adds CHANGEDSINCE/VANISHED.
src/Enums/ImapIdentifier.php	Generalizes message identifiers.
src/Connection/ImapTokenizer.php	Parses binary literals.
src/Connection/ImapConnection.php	Implements revised IMAP operations.
src/Connection/ConnectionInterface.php	Defines the v2 connection API.
src/Authenticator.php	Defines SASL authenticators.
src/Authentication/XOAuth2.php	Implements XOAUTH2 authentication.

Review details

• Files reviewed: 47/47 changed files
• Comments generated: 4
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Synchronization state, unsolicited response filtering, capability caching, and 64-bit checkpoint handling remain incorrect.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (4)

Previously missed (3) — in code that hasn't changed since the last review.

src/Mailbox.php:83

• The clone still inherits the cached server capabilities even though it opens a new connection. Capability sets can differ after reconnecting (for example across failover nodes or authentication state), so the clone may skip required checks or attempt unsupported QRESYNC/CONDSTORE operations. Clear $capabilities with the other connection-specific fields.

This issue also appears on line 194 of the same file.
src/SelectionResult.php:62

• IMAP modification sequences are unsigned 64-bit values, but casting HIGHESTMODSEQ to PHP's signed int corrupts valid checkpoints above 9223372036854775807 (for example, 18446744073709551615 saturates to PHP_INT_MAX). Such checkpoints cannot round-trip into CHANGEDSINCE/UNCHANGEDSINCE. Preserve them as decimal strings consistently across selection results, fetched data, query methods, and modifiers.
  src/StoreResult.php:31
• Every untagged FETCH response is exposed as a successfully changed message, including unsolicited updates for messages outside the STORE set. This makes messages() contradict its contract and can cause callers to apply unrelated state. Pass the command's set/addressing mode into result parsing and filter these entries, leaving unsolicited responses available through responses().

src/Mailbox.php:197

• Disconnecting clears the enabled and selection caches but leaves $capabilities cached for the next connection. A subsequent connection can therefore make capability decisions from the previous session, despite capabilities being connection/server-state dependent. Reset the capability cache here as well.

            $this->connection = null;
            $this->selected = null;
            $this->selection = null;
            $this->enabled = [];

• Files reviewed: 49/49 changed files
• Comments generated: 2
• Review effort level: Balanced

[stevebauman]

Addressed the latest review in 0844b7b, including the pending password-aware reconnect and explicit login/xoauth2 configuration changes from our discussion.

The additional findings in the review summary were checked as well:

• Capability caches now reset on both disconnect and clone.
• STORE uses the same requested-set filtering as FETCH, while retaining unrelated updates in responses().
• The unsigned 64-bit checkpoint recommendation is based on the superseded specification. RFC 7162 section 3.1 explicitly changed modification sequences to unsigned 63-bit values. The maximum, 9223372036854775807, round-trips through the existing integer API on 64-bit PHP, so I kept that API unchanged: https://datatracker.ietf.org/doc/html/rfc7162#section-3.1

QRESYNC is now checked as an enabled session capability, not merely an advertised one. ENABLE must happen before folder selection; issuing it inside an already-selected query would violate the protocol.

Verified: 625 tests pass, including the live-server suite, and formatting is clean.

The separate Laravel adapter remains unchanged; its plain default will need to become login when adopting this v2 configuration.

[Copilot]

🟡 Changes recommended

MODSEQ precision, QRESYNC validation, and failed-selection state handling can currently produce incorrect synchronization behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 70/70 changed files
• Comments generated: 6
• Review effort level: Balanced

[Copilot]

Copilot review overview

🟡 Changes recommended

Raw sequence sets and SORT charsets permit IMAP command injection, and large sparse synchronization sets are filtered quadratically.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 6

Open (6)

• Validate COPY sequence sets before command compilation · New
• Validate MOVE sequence sets before protocol serialization · New
• Validate STORE sequence sets to prevent command injection · New
• Quote SORT charset to prevent command injection · New
• Validate FETCH sequence sets to prevent IMAP command injection · New
• Validate QRESYNC known UIDs before serialization · New

Resolved since last review (6)

• A normal SELECT that returns neither HIGHESTMODSEQ nor NOMODSEQ currently reports…
• RFC 7162 modification sequences are unsigned 64-bit values, but this cast truncates any value above…
• sequence-match-data is only valid after the optional known-uids field in the QRESYNC grammar.…
• Only $selection is invalidated before EXAMINE. If EXAMINE fails, $folder still references the…
• A SELECT attempt automatically deselects the previous mailbox, including when the new SELECT fails.…
• enable() only records capabilities present in the server's ENABLED response, but this path…

Previously missed (1)

In code that hasn't changed since last review

Avoid quadratic message-set scanning in FETCH and STORE

src/​Connection/​ImapConnection.php:764

This reparses and linearly scans the complete message set for every FETCH/STORE response. Incremental sync with N sparse requested UIDs and N returned messages becomes O(N²); parse the set once per command into a lookup/range matcher and reuse it in the callback.