Closed-form RF calculation for read-only-kv

[homatthew]

What am I trying to do?

Improve the partition-aware capacity planning algorithm in read_only_kv.py:

1. Replace iterative while loop with closed-form RF calculation
2. Extract algorithm to standalone partition_capacity.py module for clarity
3. Add comprehensive test suite showing algorithm equivalence and differences

Why did I do it this way?

Standalone partition_capacity.py module (new!)

The algorithm logic was buried in private methods inside read_only_kv.py. This made it hard to:

• Understand what the algorithm does
• See where different variants are equivalent vs. different
• Test the algorithm in isolation

Solution: Extract to partition_capacity.py with three clear algorithm variants:

from partition_capacity import CapacityProblem, search_algorithm

result = search_algorithm(CapacityProblem(
    n_partitions=100,
    partition_size_gib=50.0,
    disk_per_node_gib=1000.0,
    cpu_needed=50,
    cpu_per_node=2,
    min_rf=2,
    max_nodes=1000,
))
# Returns: CapacityResult(node_count=25, rf=5, partitions_per_node=20, base_nodes=5)

Three algorithm variants with clear relationships

Algorithm	Description	Complexity
original_algorithm	While-loop from legacy code (greedy, max PPn only)	O(rf)
closed_form_algorithm	Mathematical equivalent to original	O(1)
search_algorithm	Searches PPn from max to 1, finds solutions original misses	O(ppn)

Key findings documented in tests:

• CLOSED_FORM == ORIGINAL (always, by mathematical proof)
• SEARCH ⊇ ORIGINAL (search finds everything original finds)
• SEARCH ≠ ORIGINAL when max_nodes is tight (search finds solutions original misses)

Closed-form RF calculation (commit 1)

The original while True loop incremented RF until CPU was satisfied. This obscures the underlying math.

The insight: For a given PPn, we can directly compute the minimum RF:

# Instead of incrementing rf in a loop...
rf = max(min_rf, math.ceil(cpu_needed / (base * cpu_per_node)))

Extract _PartitionSearchInputs dataclass (commit 2)

Clear boundary between "model domain" (Instance, CapacityRequirement) and "algorithm domain" (pure numbers).

Are there any tests?

Yes - 79 tests total including:

• Equivalence tests: Verify closed_form == original for 500 random problems (Hypothesis)
• Subsumption tests: Verify search finds everything original finds, plus more
• Difference tests: Concrete examples where search succeeds but original fails
• Constraint tests: All algorithms satisfy CPU, disk, RF, and size constraints

How would I use the new code?

The public interface (nflx_read_only_kv_capacity_model.capacity_plan()) is unchanged.

For direct algorithm testing:

from service_capacity_modeling.models.org.netflix.partition_capacity import (
    CapacityProblem,
    original_algorithm,
    closed_form_algorithm,
    search_algorithm,
)

problem = CapacityProblem(
    n_partitions=21,
    partition_size_gib=100.0,
    disk_per_node_gib=1000.0,  # max_ppn = 10
    cpu_needed=10,
    cpu_per_node=1,
    min_rf=2,
    max_nodes=10,
)

# Original/closed_form: returns None (greedy exceeds max_nodes)
# Search: returns CapacityResult(node_count=10, rf=2, ppn=5, base=5)

Architecture

graph TB
    subgraph partition_capacity.py
        A[CapacityProblem] --> B[original_algorithm]
        A --> C[closed_form_algorithm]
        A --> D[search_algorithm]
        B --> E[CapacityResult]
        C --> E
        D --> E
    end

    subgraph read_only_kv.py
        F[Model Objects] --> G[_extract_planning_inputs]
        G --> H[_PartitionSearchInputs]
        H --> I[CapacityProblem]
        I --> D
        E --> J[RegionClusterCapacity]
    end

Loading

🤖 Generated with Claude Code