perf(rpc): per-source-file eviction to bound polyglot RewriteRpc server memory#8297
Merged
Conversation
…ack visits it
Polyglot RewriteRpc peers (Java/JS/Python/Go/C#) kept every visited SourceFile in
their localObjects/remoteObjects + ref caches for the process lifetime; only an
explicit Reset cleared them, so within a repo every file stayed resident across
every recipe cycle.
Because RecipeRunCycle iterates source-outer / recipe-inner (editSource applies the
whole recipe stack to one file before the next), the end of editSource (and the scan
lambda) is a clean per-file boundary. Add a targeted Evict primitive:
- New Evict{id} request + RewriteRpc.evict / inbound handler / refCheckpoint, driven
from a RecipeRunCycle hook that, after the stack finishes a file, drops that file's
tree and rolls each touched peer's ref maps back to the pre-file checkpoint.
- Evict is a fire-and-forget JSON-RPC notification (NON_NULL serialization omits the
null id => a true notification everywhere); handlers are total, and the Go/Python
read loops skip replying to a null-id request so an errored notification can never
fail in-flight requests.
- Ref rollback is symmetric/lockstep: dropping the send-side ref forces the next file
to re-ADD (interned JavaType is shared across files), so the rolled-back receiver
never sees a dangling REF_USE. Send-side rollback = protocol correctness; receive-
side = the memory bound.
- Handlers added to all five peers (Java, JS onNotification, Python, Go with
ReferenceMap.RollbackTo, C#), preserving recipe/accumulator/execution-context state.
Recipe/accumulator/prepared-recipe state is keyed separately and left intact, so a
scanning recipe's accumulator survives across files within a cycle.
Profiling (Phase 0): Go and JS metrics CSVs gain local_objects/remote_objects/refs
residency columns, and RewriteRpcProcess gains an opt-in subprocess RSS sampler
(REWRITE_RPC_RSS_MS -> rpc-rss-<pid>.csv) that uniformly covers all four languages,
including native/off-heap growth the in-process counters miss.
Full RewriteRpcTest (19 tests, loopback recipe runs exercise the hook + rollback +
inbound Evict) passes, including a new evictDropsTreeFromBothPeers.
Phase-0 profiling parity. Go and JS already write local_objects/remote_objects/refs residency columns per RPC call, so a run shows cache-size ramp (no evict) vs sawtooth (per-file Evict). Python's --metrics-csv was a no-op stub and C# had no metrics CSV at all. Both now emit the Go 9-column schema: timestamp, method, duration_ms, error, memory_used_bytes, memory_max_bytes, local_objects, remote_objects, refs. - Python server.py: metrics writer + a per-call row from the dispatch loop. refs counts remote_refs only; Python's send-side refs live on a per-call RpcSendQueue, so remote_refs is the only cross-call ref cache handle_evict rolls back. - C# has no central dispatch loop (StreamJsonRpc [JsonRpcMethod] dispatch), so a MetricsMessageHandler wraps the IJsonRpcMessageHandler and records a row when each inbound request's response is written back: post-dispatch, so counts reflect the handler's mutations. Notifications (Evict) expect no response and are not recorded; the drop shows on the next recorded call. Adds RpcMetricsWriter, RewriteRpcServer.RunAsync(metricsCsv:), Program.cs parsing of --metrics-csv=, and CSharpRewriteRpc.Builder.metricsCsv(Path) threaded into all three command builders. Memory columns are RSS best-effort; the RewriteRpcProcess RSS sampler remains the source of truth for process memory. The object-count columns are the point. Verified to compile: py_compile, dotnet build, :rewrite-csharp:compileJava.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bounds the memory of the polyglot
RewriteRpcservers (Java/JS/Python/Go/C#), whose object + ref caches previously grew insert-only for the whole process lifetime, and adds the Phase-0 profiling instrumentation to measure it. Two commits:perf(rpc): evict each source file from RPC caches after the recipe stack visits it— the eviction primitive.perf(rpc): emit per-call metrics CSV for the Python and C# RPC servers— profiling parity across all four servers.Problem
Every RewriteRpc peer keyed visited
SourceFiles intolocalObjects/remoteObjects+ ref maps and never self-evicted; only an explicitResetcleared them. Within a repo, every file stayed resident across every recipe cycle, so peak memory scaled with file count and was only reclaimed when the subprocess was killed per repo.Fix — per-source-file
EvictRecipeRunCycleiterates source-outer / recipe-inner (editSourceruns the whole recipe stack over one file before the next), so the end ofeditSource(and the scan lambda) is a clean per-file boundary. A newEvict{id}primitive drops that file's tree and rolls each touched peer's ref maps back to a pre-file checkpoint:JavaType(shared across files), so the receiver never sees a danglingREF_USE. Send-side rollback = protocol correctness; receive-side = the memory bound.onNotification, Python, Go viaReferenceMap.RollbackTo, C#). Recipe/accumulator/execution-context state is keyed separately and preserved, so a scanning recipe's accumulator survives across files within a cycle.Eviction is unconditional (no config flag).
Profiling (Phase 0)
local_objects,remote_objects,refsresidency columns for all four language servers (Go/JS in commit 1; Python + C# in commit 2), so a run shows the cache-size ramp (evict off) vs sawtooth (evict on). Python's--metrics-csvwas previously a no-op stub; C# had no metrics CSV at all. C# records via a message-handler wrapper since StreamJsonRpc dispatches by attribute and has no central loop.RewriteRpcProcessgains an opt-in subprocess RSS sampler (REWRITE_RPC_RSS_MS->rpc-rss-<pid>.csv) that uniformly covers all four languages, including native/off-heap growth the in-process counters miss.Measured
Same-binary A/B on the Python server (per-repo peak subprocess RSS, evict off -> on): flask 133->94 MB (-30%), poetry 285->86 MB (-70%), scrapy 374->68 MB (-82%). The "before" curve climbs monotonically with file count; "after" is bounded and does not scale with repo size. Cost: ~30% longer wall-clock from the per-file evict notifications.
Testing
Full
RewriteRpcTest(19 tests; loopback recipe runs exercise the hook + client rollback + inboundEvict) passes, including a newevictDropsTreeFromBothPeers. The metrics commit is additive and does not touch the Java loopback test path; it is verified to compile viapy_compile,dotnet build, and:rewrite-csharp:compileJava.Follow-up (separate repo)
The Moderne CLI already wires each server's
metricsCsv; C# (csharp-rpc.csv) needs a one-line CLI change to pass the path, which lands after this releases.