Skip to content

Commit 0bdcb5c

Browse files
committed
Add fe build --from-metadata to rebuild from a metadata.json input
`fe build --emit metadata` produces a complete, deterministic recompilation input, but the reverse step existed only as a test helper. External toolchains (foundry-compilers, Sourcify-style verifiers) need it as a CLI feature: one invocation, JSON in, artifacts out. `fe build --from-metadata <PATH|->` (`-` reads stdin) materializes the recorded project (regenerated fe.toml per settings.ingots[] entry, sources routed by namespace) into a temporary directory and runs the normal ingot build on it: - default target is the settings.compilationTarget contract; --contract overrides - optimizer level comes from settings.optimizer.level; an explicit -O wins with a stderr warning that the bytecode deviates - compiler.version mismatch warns on stderr without aborting - artifacts follow --emit into --out-dir, defaulting to ./out - invalid input (bad JSON, language != "Fe", missing sources/ingots) exits 1; combining with [path], --ingot, --standalone, or --report is a clap conflict (exit 2) The reconstruction lives in crates/fe/src/metadata_input.rs and validates untrusted input (no path traversal out of the temp dir). The round-trip tests now exercise the CLI feature directly; the test helper is removed. New tests cover stdin, target selection, the -O and version warnings, and the error cases.
1 parent ce53c6e commit 0bdcb5c

6 files changed

Lines changed: 997 additions & 96 deletions

File tree

CLI.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ Builds use the Sonatina codegen pipeline and generate EVM bytecode directly.
9494

9595
```
9696
fe build [--standalone] [--contract <name>] [--optimize <level>] [--out-dir <dir>] [--report [--report-out <out>] [--report-failed-only]] [path]
97+
fe build --from-metadata <PATH|-> [--contract <name>] [--optimize <level>] [--out-dir <dir>] [--emit <artifacts>]
9798
```
9899

99100
If `path` is omitted, it defaults to `.`.
@@ -234,6 +235,35 @@ Filenames are “sanitized” from contract names:
234235

235236
This sanitization is also what the workspace collision check uses.
236237

238+
### Rebuilding from metadata: `--from-metadata`
239+
240+
`fe build --from-metadata <PATH|->` rebuilds a contract from a `<Contract>.metadata.json`
241+
recompilation input (as produced by `--emit metadata`) instead of a source tree. `-` reads the
242+
JSON from stdin, so external toolchains can pipe a synthesized metadata document into a single
243+
`fe build` invocation. All diagnostics go to stderr.
244+
245+
Behavior:
246+
247+
- The recorded project (all `sources`, one regenerated `fe.toml` per `settings.ingots[]` entry) is
248+
materialized into a temporary directory and built through the normal ingot build path; the
249+
bundled `std`/`core` are provided by the compiler.
250+
- **Contract selection**: defaults to the contract recorded in `settings.compilationTarget`;
251+
`--contract <name>` overrides it.
252+
- **Optimizer level**: defaults to `settings.optimizer.level`; an explicit `-O`/`--optimize` wins,
253+
with a warning on stderr that the rebuilt bytecode will not match the verified artifact.
254+
- **Arithmetic**: the per-ingot effective `arithmetic` values from the metadata are applied via
255+
the regenerated `fe.toml` files (`dependency-arithmetic` is deliberately not re-applied; the
256+
metadata records post-forcing values).
257+
- **Version check**: if `compiler.version` differs from the running compiler, a warning is printed
258+
to stderr (no abort); exact bytecode reproduction is only guaranteed with the same version.
259+
- **Output**: artifacts are selected with `--emit` as usual and written to `--out-dir`, which
260+
defaults to `./out` (relative to the current working directory, since there is no project
261+
directory to anchor to).
262+
263+
Errors (exit code 1): unreadable input, invalid JSON, `language != "Fe"`, missing `sources`, or a
264+
missing/rootless `settings.ingots`. Combining `--from-metadata` with the `[path]` argument,
265+
`--ingot`, `--standalone`, or `--report` is a CLI usage error (exit code 2).
266+
237267
### Reports: `--report`
238268

239269
`fe build` can optionally write a `.tar.gz` debugging report (useful for sharing failures):

crates/fe/src/build.rs

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -306,6 +306,137 @@ pub fn build(
306306
}
307307
}
308308

309+
/// `fe build --from-metadata`: rebuild from a `metadata.json` recompilation
310+
/// input instead of a source tree. Reads the metadata (from a file, or stdin
311+
/// for `-`), materializes the recorded project into a temporary directory,
312+
/// and runs the normal ingot build on it. Exits the process on failure.
313+
pub fn build_from_metadata(
314+
input: &Utf8Path,
315+
contract: Option<&str>,
316+
optimize: Option<&str>,
317+
emit: &[BuildEmit],
318+
out_dir: Option<&Utf8PathBuf>,
319+
profile: &str,
320+
use_recovery_mode: bool,
321+
) {
322+
let emit = EmitSelection::from_requested(emit);
323+
324+
let metadata = match crate::metadata_input::read_metadata(input) {
325+
Ok(metadata) => metadata,
326+
Err(err) => {
327+
eprintln!("Error: {err}");
328+
std::process::exit(1);
329+
}
330+
};
331+
if let Err(err) = crate::metadata_input::validate_metadata(&metadata) {
332+
eprintln!("Error: {err}");
333+
std::process::exit(1);
334+
}
335+
336+
// Exact reproduction is only guaranteed with the same compiler version.
337+
if let Some(version) = metadata["compiler"]["version"].as_str() {
338+
let running = env!("CARGO_PKG_VERSION");
339+
if version != running {
340+
eprintln!(
341+
"Warning: metadata was produced by fe {version}, but this is fe {running}; \
342+
the rebuilt bytecode may differ from the original"
343+
);
344+
}
345+
}
346+
// Non-release builds share a package version; the recorded commit pins the
347+
// exact source revision.
348+
if let (Some(recorded), Some(running)) = (
349+
metadata["compiler"]["commit"].as_str(),
350+
option_env!("FE_GIT_HASH").filter(|hash| !hash.is_empty()),
351+
) && recorded != running
352+
{
353+
eprintln!(
354+
"Warning: metadata was produced by fe commit {recorded}, but this is fe commit \
355+
{running}; the rebuilt bytecode may differ from the original"
356+
);
357+
}
358+
359+
// The metadata's optimizer level is the default; an explicit `-O` wins.
360+
let recorded_level = metadata["settings"]["optimizer"]["level"].as_str();
361+
let level = match (optimize, recorded_level) {
362+
(Some(flag), Some(recorded)) if flag != recorded => {
363+
eprintln!(
364+
"Warning: -O {flag} overrides optimizer level {recorded} recorded in the \
365+
metadata; the rebuilt bytecode will not match the verified artifact"
366+
);
367+
flag
368+
}
369+
(Some(flag), _) => flag,
370+
(None, Some(recorded)) => recorded,
371+
(None, None) => "1",
372+
};
373+
let opt_level: OptLevel = match level.parse() {
374+
Ok(level) => level,
375+
Err(err) => {
376+
eprintln!("Error: {err}");
377+
std::process::exit(1);
378+
}
379+
};
380+
381+
// Default target: the contract recorded in `compilationTarget`; an
382+
// explicit `--contract` overrides it.
383+
let contract = contract.map(str::to_string).or_else(|| {
384+
metadata["settings"]["compilationTarget"]
385+
.as_object()
386+
.and_then(|target| target.values().next())
387+
.and_then(|name| name.as_str())
388+
.map(str::to_string)
389+
});
390+
391+
let temp = match tempfile::Builder::new()
392+
.prefix("fe-from-metadata")
393+
.tempdir()
394+
{
395+
Ok(temp) => temp,
396+
Err(err) => {
397+
eprintln!("Error: Failed to create temporary project directory: {err}");
398+
std::process::exit(1);
399+
}
400+
};
401+
let Some(temp_root) = Utf8Path::from_path(temp.path()) else {
402+
eprintln!("Error: temporary project directory path is not valid UTF-8");
403+
std::process::exit(1);
404+
};
405+
let root_dir = match crate::metadata_input::reconstruct_project(&metadata, temp_root) {
406+
Ok(dir) => dir,
407+
Err(err) => {
408+
eprintln!("Error: {err}");
409+
std::process::exit(1);
410+
}
411+
};
412+
413+
let out_dir = out_dir.cloned().unwrap_or_else(|| Utf8PathBuf::from("out"));
414+
415+
let mut db = DriverDataBase::default();
416+
db.compiler_options()
417+
.set_recovery_mode(&mut db)
418+
.to(use_recovery_mode);
419+
db.compilation_settings()
420+
.set_profile(&mut db)
421+
.to(profile.into());
422+
423+
let had_errors = build_directory(
424+
&mut db,
425+
&root_dir,
426+
None,
427+
contract.as_deref(),
428+
opt_level,
429+
emit,
430+
Some(&out_dir),
431+
None,
432+
);
433+
434+
drop(temp);
435+
if had_errors {
436+
std::process::exit(1);
437+
}
438+
}
439+
309440
#[allow(clippy::too_many_arguments)]
310441
fn build_file(
311442
db: &mut DriverDataBase,

crates/fe/src/main.rs

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ mod dependency_diagnostics;
77
mod doc;
88
#[cfg(feature = "doc-server")]
99
mod doc_serve;
10+
mod metadata_input;
1011
mod report;
1112
mod test;
1213
#[cfg(not(target_arch = "wasm32"))]
@@ -111,6 +112,17 @@ pub enum Command {
111112
/// Treat a `.fe` file target as standalone, even if it is inside an ingot.
112113
#[arg(long)]
113114
standalone: bool,
115+
/// Rebuild from a `<Contract>.metadata.json` recompilation input produced by
116+
/// `--emit metadata` (`-` reads the JSON from stdin).
117+
///
118+
/// The recorded project is materialized into a temporary directory and built with
119+
/// the settings captured in the metadata. Artifacts default to `./out`.
120+
#[arg(
121+
long,
122+
value_name = "PATH",
123+
conflicts_with_all = ["path", "ingot", "standalone", "report"]
124+
)]
125+
from_metadata: Option<Utf8PathBuf>,
114126
/// Build a specific contract by name (defaults to all contracts in the target).
115127
#[arg(long)]
116128
contract: Option<String>,
@@ -456,6 +468,7 @@ pub fn run(opts: &Options) {
456468
path,
457469
ingot,
458470
standalone,
471+
from_metadata,
459472
contract,
460473
optimize,
461474
out_dir,
@@ -466,6 +479,18 @@ pub fn run(opts: &Options) {
466479
report_failed_only,
467480
recovery_mode,
468481
} => {
482+
if let Some(metadata_path) = from_metadata {
483+
build::build_from_metadata(
484+
metadata_path,
485+
contract.as_deref(),
486+
optimize.as_deref(),
487+
emit,
488+
out_dir.as_ref(),
489+
profile,
490+
*recovery_mode,
491+
);
492+
return;
493+
}
469494
let opt_level = match effective_opt_level(optimize.as_deref()) {
470495
Ok(level) => level,
471496
Err(err) => {

0 commit comments

Comments
 (0)