Skip to content

Fix Sentinel connection leak and AbortOnConnectFail=false handling - #3187

Open
HarnageaGabriel wants to merge 3 commits into
StackExchange:mainfrom
HarnageaGabriel:fix-sentinel-abortonconnectfail-leak-2980
Open

Fix Sentinel connection leak and AbortOnConnectFail=false handling#3187
HarnageaGabriel wants to merge 3 commits into
StackExchange:mainfrom
HarnageaGabriel:fix-sentinel-abortonconnectfail-leak-2980

Conversation

@HarnageaGabriel

Copy link
Copy Markdown
Contributor

Summary

Fixes #2980.

When connecting via Sentinel (ConnectionMultiplexer.Connect/ConnectAsync/SentinelConnect) with AbortOnConnectFail=false, a failed initial connect — sentinels unreachable, or no primary discovered within ConnectTimeout — threw a RedisConnectionException instead of returning a multiplexer to the caller, and leaked the internally-created ConnectionMultiplexer instances (the sentinel monitor connection, and per-retry primary candidates in GetSentinelMasterConnection's loop), since nothing disposed them on the failure path.

This didn't match the non-Sentinel Connect/ConnectImpl behavior, which honors AbortOnConnectFail=false by returning the muxer (with LastException set) instead of throwing.

Changes in ConnectionMultiplexer.Sentinel.cs:

  • AbortOnConnectFail=false: no longer throws. Returns a usable multiplexer with LastException set, the same event wiring (ConnectionRestored/ConnectionFailed) and background retry (SwitchPrimary/reconnect timer) as the success path, so callers can attach handlers and let it reconnect once sentinels/primary become reachable — mirroring the non-Sentinel path.
  • AbortOnConnectFail=true: preserves today's throwing behavior and exception messages (no breaking change), but now disposes the sentinel connection and any superseded/failed candidate connections before the exception propagates, instead of leaking them.
  • Per-iteration candidate connections in the primary-discovery loop are now disposed before being superseded, regardless of AbortOnConnectFail.

No public API changes.

Test plan

  • dotnet build (Release) — 0 errors, 0 warnings
  • New tests in SentinelConfigTests.cs covering AbortOnConnectFail=false (sync Connect, async ConnectAsync, and SentinelConnect) returning a non-null, non-throwing, disconnected multiplexer with LastException set against unreachable sentinel endpoints, plus a regression guard that AbortOnConnectFail=true still throws — all against closed loopback ports, no live topology required
  • Full StackExchange.Redis.Tests suite (net10.0, Release) against the docker-compose Redis/Sentinel/Cluster topology: 5880 passed, 0 failed (the live Sentinel integration tests are gated [SkipOnWindows], unaffected by this change)

🤖 Generated with Claude Code

On OSS cluster, the direct (NoRedirect) probe messages used during
connection setup and keep-alive could target a hash slot the
connected node doesn't own, so the server replies MOVED and the
probe is dropped instead of following it.

- Skip the replica_read_only SET fallback in AutoConfigureAsync once
  cluster topology already reports our role, since it's both
  redundant and slot-unsafe there.
- Skip the tie-breaker GET fallback in AutoConfigureAsync on cluster,
  where a tie-breaker key isn't meaningful.
- When the ECHO/PING/TIME tracer is unavailable, build the EXISTS
  fallback key with a hash-tag targeting a slot this endpoint
  actually owns, reusing the existing hash-tag cache.

Fixes StackExchange#2970.
@HarnageaGabriel
HarnageaGabriel force-pushed the fix-sentinel-abortonconnectfail-leak-2980 branch from 6e6dd2b to 6f167d7 Compare August 18, 2026 20:18
When connecting via Sentinel with AbortOnConnectFail=false, a failed
initial connect (unreachable sentinels, or no primary discovered
within ConnectTimeout) threw instead of returning the multiplexer,
and leaked the internally-created ConnectionMultiplexer instances
(sentinel monitor connection and per-retry primary candidates) since
nothing disposed them. This now mirrors the non-Sentinel Connect path:
AbortOnConnectFail=false returns a usable, disposed-free multiplexer
with LastException set and background retry wired up, while
AbortOnConnectFail=true still throws but disposes everything first.

Fixes StackExchange#2980
@HarnageaGabriel
HarnageaGabriel force-pushed the fix-sentinel-abortonconnectfail-leak-2980 branch from 6f167d7 to 19103ee Compare August 19, 2026 19:33
@mgravell

Copy link
Copy Markdown
Collaborator

I think the sentinel side of this is reasonable and captures the key scenarios (not quite full test coverage, but probably enough), however the slot stuff shares the same feedback from #3185 (and for good reason: they share a commit)

@mgravell mgravell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(hash-tag / slot feedback common with #3185)

…nch's shared code

This branch carries commit 1c2a472 ("Make fallback discovery and
keep-alive probes cluster-slot aware") in its history, so mgravell's
review of StackExchange#3185 pointing out bugs in ServerSelectionStrategy's hash-tag
handling and ClusterNode.Parent applies here too. Porting the same fix:

- Unify InventKey's O(16384) map-scan and the tracer key's slot lookup
  into one primitive, ServerEndPoint.GetServableSlot(), which also now
  falls back to a replica's primary's slots (a replica's own Slots is
  always empty).
- Fix ClusterNode.Parent, which always returned null due to a backwards
  null-check on its own backing field.
- Replace the cached-byte[]-prefix API (a 16384-entry static array
  publishing shared mutable state as a RedisKey) with
  ServerSelectionStrategy.CreateKeyForSlot(slot, suffix), composed per
  call.
- Memoize the composed tracer key on ServerEndPoint since GetTracerKey
  runs on the heartbeat path.
- GetClusterNode uses ClusterConfiguration's O(1) endpoint indexer.
- Comment AutoConfigureAsync's cluster guards (first-handshake window,
  why the SET probe can't work on cluster) and drop a redundant clause.

See StackExchange#3185 for the full review and the
matching commits on that PR's branch.
@HarnageaGabriel

Copy link
Copy Markdown
Contributor Author

Re: "hash-tag / slot feedback common with #3185" - confirmed, this branch carries #3185's commit (1c2a4725) in its history, so it inherited the same bugs in ServerSelectionStrategy's hash-tag handling, InventKey, GetClusterNode, and ClusterNode.Parent. This PR's own commit doesn't touch that code directly.

Pushed the same structural fix as a separate commit (a98b42a0) on top of this branch - see #3185's review thread for the point-by-point breakdown of what changed. Sentinel-specific code (ConnectionMultiplexer.Sentinel.cs, SentinelConfigTests.cs) is untouched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Sentinel connection with AbortOnConnectFail=false fails to return multiplexer and leaks objects on startup failure

2 participants