Conversation
Dispatches each invocation to whichever runtime owns it, decided by whether its bundle carries an `r` block. An R operation is a script rather than a binary, so it is the script source that is executed, with the same argument vector the recipe's io and parameters already produce. The two runtimes cannot share a filesystem. aioli's tools live in one worker and webR's R session in another, so an R operation cannot see a file an aioli tool wrote and vice versa. Every output is already kept as a DataValue, so chaining copies the bytes across rather than reading a filesystem the other runtime has no access to: a C tool's output is materialised into webR before an R operation consumes it, and an R operation's results are copied back into aioli afterwards, where a later C tool expects to find them under the same name. A run made entirely of R operations no longer builds an aioli worker, which it would otherwise do only to satisfy Aioli's refusal of an empty tool list. webR is started on first use rather than up front, since it is a far larger download than any single tool's wasm and most runs never touch R. A second R operation mounts its library into the running session instead of starting another one. Both workers are released in a finally, so neither accumulates for the lifetime of the page when a run throws part way. Verified with stubbed runtimes: an all-R run creates no aioli worker, chaining copies bytes in both directions, a second R tool reuses the session, and both workers close on the success and the error path.
Assigning the invocation's error list after exec discarded anything already recorded against it. Preparing the inputs for an R operation can record one: when a chained input names an output that was never produced, there is no file to copy across and the operation runs without it. That message was overwritten before anyone could see it, leaving a run that failed for a knowable reason looking like it failed for none.
Three defects found by review, all of them silent. loadToolIndex replaced whatever was already known about a tool with the index entry, and the index carries six fields, not `runtime`. It runs on every mount of the tools and workflow pages, so returning to a page stripped `runtime` from a tool loadTool had already resolved. isROperation then answered "not R" for an R operation and routed it to aioli, which asked for a wasm binary that does not exist. The index is now merged over what is known rather than replacing it, and runTools refuses outright to run a tool whose bundle was never loaded instead of guessing at its runtime. Every operation in a recipe ships the same package library, and each operation lives in its own registry repository, so nothing deduplicated it: a three-operation workflow fetched the library three times and held three decompressed copies in the worker. It is now keyed by library digest, fetched once and mounted once. A library is only loadable by the webR release it was built against, and the recipe records which. The runtime now refuses a mismatch with a message naming both versions, rather than letting a routine dependency bump surface as an unattributable load error inside a worker. Verified: fifteen checks, including that three operations of one recipe share a single fetch, mount and session.
A second review pass found that two of the previous round's fixes were themselves wrong, and turned up several defects around them. The webR worker is spawned by the constructor, not by init(), so anything that threw between the two leaked one. The version check added last time sat exactly there, making a mismatch leak a worker deterministically and the natural response -- pressing Run again -- leak another. Startup is now wrapped so the worker is closed on any failure. The cleanup uses try/catch rather than .catch(), because close() returns undefined rather than a promise when the worker never initialised, and calling .catch on that throws a TypeError that replaces the error being reported. The guard against running an unloaded tool was placed after the loop it was supposed to protect. An unloaded tool was treated as an aioli tool and died dereferencing a wasm digest it had no bundle for, several steps before the guard could speak. It now runs before anything reads a runtime. Also fixed in the runtime: stdin was bound on every invocation but only ever set, so one operation's input was delivered again to the next; webR reports output a line at a time with the terminator stripped and the lines were rejoined by appending one to each, giving every stdout a newline it never had; ls() read the whole file to report its length, copying every output across the worker boundary twice; and the argument vector was allocated outside the shelter, so purge() could not reclaim it. Two in the dispatch: an output declared with mode "files" never assigns a result, and the copy-across block dereferenced it; and an input whose upstream output was missing recorded the problem and then passed the filename on anyway, so the operation failed again inside R on a file that was not there, burying the error that explained it. Finally, runWorkflow awaited runTools with no handler, and isRunning is cleared per node as each invocation finishes. Loading an R operation can fail outright where the wasm path merely returned null, so a failure left every node spinning with no error and no way back but a reload. Verified: ten runtime checks including the version mismatch and the stdin, newline and empty-argv cases, and fifteen dispatch checks.
webR loads around 20 MB of R.js, R.wasm and its support files at run time, and by default takes them from https://webr.r-wasm.org. That made the R interpreter the one component the app executes that arrives from a third party: outside the registry, unpinned by any digest, absent from any SBOM the recipes pipeline produces, and available only for as long as that host is. Every wasm binary for every other tool is pulled from our own registry and checked against a digest recorded in its recipe. Those files ship inside the npm package already, so the build copies them alongside the app and the runtime is pointed at the copy. Only what webR actually requests is copied; the REPL, the tests and the source maps that share the package are left out. baseUrl is derived from webpack's public path, so it resolves under the /Biochef/ prefix the deployed site uses as well as at the root in development, and remains overridable for tests. The deployed site grows by about 46 MB, but a run does not download that: most of it is lazily mounted images -- help, docs, translations, the geospatial data -- that batch execution never asks for. Starting R fetches roughly 20 MB, once per page. Verified by building and booting R 4.6.0 from the build output rather than from node_modules or the CDN: it starts in 2.4 seconds, takes its argument vector and writes its output file.
The newline change in the last commit was wrong. It assumed webR
distinguishes output that ended in a newline from output that did not.
It does not: cat("x\n") and cat("x") both arrive as a single event
carrying "x", so the terminator cannot be recovered either way. Joining
the lines therefore dropped the terminator from every newline-terminated
output, which is the overwhelmingly common case, to avoid inventing one
in the rare case. Appending per line is restored, with the reasoning
recorded so it is not "fixed" again.
The error protocol overloaded "" as "no error", but stop() with no
arguments gives an empty conditionMessage, so a failed operation with
truncated output was reported as a success. Success is now a zero-length
vector, distinguished by length rather than content.
The stdin clear and the shelter purge sat on the happy path, so a
captureR rejection -- a wasm trap is not an R condition -- skipped both.
They are in a finally now, with the result read before the purge that
would invalidate it.
The finally added to runWorkflow could itself throw. `component` is a
snapshot taken before the run, a node can be deleted while it is in
flight, and getNode then returns undefined: the sweep threw from inside
the finally, masking the original error and leaving the rest of the
nodes spinning, which is exactly what the block was added to prevent.
Both that and the callback's node lookup are guarded.
Two more from review: a second R library mounted into a running session
was never version-checked, so only the first recipe's build was
verified; and the aioli stdin path dereferenced a read that returns
undefined when the upstream output never materialised.
The skip added for a missing chained input is reverted. It avoided
compounding one error with another, but for a positional input it
dropped an argument and shifted every later one, so the operation read
the wrong file instead of failing -- a silent misread is worse, and the
recorded message survives to explain the failure either way.
loadTool returned undefined where callers test for false, so a tool with
no bundle read as loaded.
Finally, the copied webR runtime was being re-minified by Terser, so it
could not be compared against the package it came from -- half the point
of serving it ourselves. It is excluded from minification and now
matches byte for byte, and the licence covering those binaries ships
with them.
Verified: R.js, webr-worker.js, libRblas.so, libRlapack.so and R.wasm
all byte-identical to the package; R 4.6.0 boots from the build output;
eleven runtime and recipe checks; fifteen dispatch checks.
The purge runs in a finally reached when captureR rejects, and a rejection there is exactly the case where the worker may be in no state to purge. An exception thrown from a finally replaces the one being propagated, so a wasm trap would have surfaced as a purge failure instead of as itself. This is the same shape as the close().catch() fault fixed earlier in the same file. With this, every finally on the R path is safe against its own cleanup failing: the shelter purge here, each worker close in runTools, and the node sweep in runWorkflow.
The skip added for an aioli stdin input whose upstream output was missing also skipped the assignment below it, leaving stdin holding whatever the previous invocation had set and feeding this operation input it never asked for. It falls through with an empty value instead. Two comments claimed the app checks artifacts against a digest recorded in a recipe. It does not: the digest addresses the blob in the registry, and nothing here hashes what it receives. Since those comments are the stated justification for shipping the webR runtime ourselves, they say what is true now, and note explicitly that the bytes are not verified.
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it does
Dispatches each invocation to whichever runtime owns it, decided by whether its bundle carries an
rblock. An R operation is a script rather than a binary, so it is the script source that is executed — with the same argument vector the recipe'sioandparametersalready produce, so nothing about how a recipe describes an operation changes.The bridging, which is the substance of this change
The two runtimes cannot share a filesystem. aioli's tools live in one worker, webR's R session in another. An R operation cannot see a file an aioli tool wrote, and a C tool cannot see what R produced. That is not a limitation worth fighting: webR owns its worker.
Every output is already kept as a
DataValue, so chaining copies bytes across rather than reading a filesystem the other runtime has no access to:${uniqueId}-${name}.txtname a later C tool expectsThe result is that mixed pipelines work in both directions without either runtime knowing the other exists.
Two smaller things
Both workers are released in a
finally, so neither accumulates for the lifetime of the page when a run throws part way.Verification
Eleven checks against stubbed runtimes, exercising the real
runTools:C → Rchaining materialises the C output into webR's filesystemR → Cchaining copies the R output back into aioli'sSupply chain: the R runtime is served from this origin
webR loads ~20 MB of
R.js/R.wasmat run time, and by default fromhttps://webr.r-wasm.org. That would make the R interpreter the one component the app executes that comes from a third party — outside the registry, unpinned by any digest, absent from any SBOM, and available only for as long as that host is. Every other tool's wasm is pulled from our registry and checked against a digest recorded in its recipe.Those files ship in the npm package, so the build copies them alongside the app and
baseUrlpoints at the copy.baseUrlis derived from webpack's public path, so it resolves under the/Biochef/prefix as well as at the root in development.Cost: the deployed site grows by ~46 MB. A run does not download that — most of it is lazily mounted images (help, docs, translations, geospatial data) that batch execution never requests. Starting R fetches roughly 20 MB, once per page. If that size is unacceptable on Pages, the alternative is reverting to the CDN and accepting the dependency explicitly.
Verified by building and booting R 4.6.0 from the build output rather than node_modules or the CDN: starts in 2.4 s, takes its argument vector, writes its output file.
Review notes
close-aioli-worker), which adds afinallyclosing the aioli worker in this same function. The conflict is real, not adjacency: thefinallyhere supersedes it by closing both workers. If Close the Aioli worker when a workflow run finishes #93 lands first I will rebase and drop the duplicate; if this lands first, Close the Aioli worker when a workflow run finishes #93 becomes unnecessary.finallyhere is the same reasoning as Close the Aioli worker when a workflow run finishes #93's, extended to cover webR — worth reading them together.Merge order
Stacked on #94 — base it on
add-r-runtime, notmain.src/utils/toolUtils.jsis where five open branches meet: #91, #92, #93, this one, and WildBunnie'saioli-vendored(#90). They all conflict there pairwise, so the order matters more than usual.Suggested order:
aioli-vendored(#90) first, since it is not mine and it vendors the aioli module this depends on; then #91 → #92; then #93; then #94 → this. Rebasing this last is deliberate — it is the largest of them and the cheapest to move.Note #93 becomes redundant once this lands: the
finallyhere closes both the aioli and the webR worker. If #93 lands first, which it should, drop the duplicate block when rebasing this.