Skip to content

Backport native quota enforcement (ZOOKEEPER-3301, 4219, 4377) - #166

Open
arkmish wants to merge 3 commits into
branch-3.6from
armishra/zookeeper-3301-quota-enforcement
Open

arkmish wants to merge 3 commits into
branch-3.6from
armishra/zookeeper-3301-quota-enforcement

Conversation

@arkmish

@arkmish arkmish commented Oct 19, 2026 •

Copy link
Copy Markdown

Description

Backports native znode quota enforcement from upstream ZooKeeper (released in 3.7/3.8) to branch-3.6. The PR has one commit per upstream change, in upstream merge order:

Commit Upstream Change
1 ZOOKEEPER-3301 (apache/zookeeper@190a227aa) Hard quota limits and enforcement, plus our fork-specific error-code renumbering
2 ZOOKEEPER-4219 (apache/zookeeper@eb1569e4f) setData quota checks use the previous data from the prepared ChangeRecord, so a multi() that creates a node and then sets its data works correctly
3 ZOOKEEPER-4377 (apache/zookeeper@9f355f5a5) An unknown error code produces an explicit IllegalArgumentException instead of an NPE in KeeperException.create. This matters for older clients that receive the new -128 code.

Today branch-3.6 has only soft quotas, which log a WARN and never block a write. This PR adds:

  • Hard limits: setquota -N <count> / -B <bytes>, with matching delquota -N/-B and extended listquota output.
  • An extensible StatsTrack format that the legacy parser can still read.
  • A pre-transaction check in PrepRequestProcessor for create and setData; deletes are never rejected.
  • QuotaExceededException (-125).
  • The startup flag -Dzookeeper.enforceQuota, default false. As in upstream 3.8, the soft-limit warning also moves behind this flag.

Fork-specific: QUOTAEXCEEDED takes upstream's -125, so TotalEphemeralLimitExceeded moves to -128. This is the same mapping as our 3.8 branch (#138). Upgrade clients before servers, because -125 changes meaning.

Per-namespace quota metrics (ZOOKEEPER-4211) are not included.

Supersedes #165. #167 and #168 were folded into this PR.

Tests

  • New: StatsTrackTest, QuotasTest, KeeperExceptionQuotaTest, ZooKeeperQuotaEnforcementTest (including the multi cases), QuorumQuotaEnforcementTest, EnforceQuotaTest, and QuotaClientCompatibilityTest (old and new client JVMs).
  • Updated: ZooKeeperQuotaTest, QuorumQuotaTest, DataTreeTest, StringUtilTest.
  • Local run of 18 quota, client, multi and ephemeral-limit suites: 195 tests passed, 0 failures, 0 skips. This includes a mixed-version case with old and new clients against the new server.
  • Checkstyle and the full reactor test-compile pass. Each commit also builds and passes the suites on its own.

Changes that Break Backward Compatibility (Optional)

  • TotalEphemeralLimitExceeded changes from -125 to -128.
  • StatsTrack count is now long.
  • Quotas.quotaPath() returns the quota directory; use limitPath() for the limits node.
  • KeeperException.Code.get throws IllegalArgumentException for unknown codes instead of returning null.

Documentation (Optional)

Updated zookeeperQuotas.md, zookeeperCLI.md and zookeeperAdmin.md. Overview: ZooKeeper Znode Quotas

  • Local code review completed

Created with GitHub Copilot CLI.

arkmish and others added 3 commits October 19, 2026 20:13
Backport of apache/zookeeper 190a227 (released in 3.7.0/3.8.0), adapted to branch-3.6.

Adds hard count/byte quotas (setquota -N/-B, delquota -N/-B, listquota), the
extensible StatsTrack format, QuotaExceededException, and pre-transaction
quota checks in PrepRequestProcessor gated by -Dzookeeper.enforceQuota
(default false).

Fork-specific: native QUOTAEXCEEDED takes -125 (upstream value), so the fork's
TotalEphemeralLimitExceeded moves from -125 to -128, matching the existing
LinkedIn 3.8 mapping (#138).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Backport of apache/zookeeper eb1569e (released in 3.7.0/3.8.0).

checkQuota now receives the previous data from the prepared ChangeRecord
instead of looking the node up in DataTree, which NPE'd for a create followed
by setData in the same multi().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…low version client requests the high version server

Backport of apache/zookeeper 9f355f5 (released in 3.5.10/3.7.1/3.8.0).

Code.get throws IllegalArgumentException for unknown codes and
KeeperException.create reports the code, instead of an NPE.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@arkmish arkmish changed the title ZOOKEEPER-3301: Enforce the quota limit (backport) Backport native quota enforcement (ZOOKEEPER-3301, 4219, 4377) Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant