Skip to content

Commit d6f4840

Browse files
committed
Add fe build --from-metadata to rebuild from a metadata.json input
`fe build --emit metadata` produces a complete, deterministic recompilation input, but the reverse step existed only as a test helper. External toolchains (foundry-compilers, Sourcify-style verifiers) need it as a CLI feature: one invocation, JSON in, artifacts out. `fe build --from-metadata <PATH|->` (`-` reads stdin) materializes the recorded project (regenerated fe.toml per settings.ingots[] entry, sources routed by namespace) into a temporary directory and runs the normal ingot build on it: - default target is the settings.compilationTarget contract; --contract overrides - optimizer level comes from settings.optimizer.level; an explicit -O wins with a stderr warning that the bytecode deviates - compiler.version mismatch warns on stderr without aborting - artifacts follow --emit into --out-dir, defaulting to ./out - invalid input (bad JSON, language != "Fe", missing sources/ingots) exits 1; combining with [path], --ingot, --standalone, or --report is a clap conflict (exit 2) The reconstruction lives in crates/fe/src/metadata_input.rs and validates untrusted input (no path traversal out of the temp dir). The round-trip tests now exercise the CLI feature directly; the test helper is removed. New tests cover stdin, target selection, the -O and version warnings, and the error cases.
1 parent ce53c6e commit d6f4840

6 files changed

Lines changed: 1162 additions & 106 deletions

File tree

CLI.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ Builds use the Sonatina codegen pipeline and generate EVM bytecode directly.
9494

9595
```
9696
fe build [--standalone] [--contract <name>] [--optimize <level>] [--out-dir <dir>] [--report [--report-out <out>] [--report-failed-only]] [path]
97+
fe build --from-metadata <PATH|-> [--contract <name>] [--optimize <level>] [--out-dir <dir>] [--emit <artifacts>]
9798
```
9899

99100
If `path` is omitted, it defaults to `.`.
@@ -234,6 +235,35 @@ Filenames are “sanitized” from contract names:
234235

235236
This sanitization is also what the workspace collision check uses.
236237

238+
### Rebuilding from metadata: `--from-metadata`
239+
240+
`fe build --from-metadata <PATH|->` rebuilds a contract from a `<Contract>.metadata.json`
241+
recompilation input (as produced by `--emit metadata`) instead of a source tree. `-` reads the
242+
JSON from stdin, so external toolchains can pipe a synthesized metadata document into a single
243+
`fe build` invocation. All diagnostics go to stderr.
244+
245+
Behavior:
246+
247+
- The recorded project (all `sources`, one regenerated `fe.toml` per `settings.ingots[]` entry) is
248+
materialized into a temporary directory and built through the normal ingot build path; the
249+
bundled `std`/`core` are provided by the compiler.
250+
- **Contract selection**: defaults to the contract recorded in `settings.compilationTarget`;
251+
`--contract <name>` overrides it.
252+
- **Optimizer level**: defaults to `settings.optimizer.level`; an explicit `-O`/`--optimize` wins,
253+
with a warning on stderr that the rebuilt bytecode will not match the verified artifact.
254+
- **Arithmetic**: the per-ingot effective `arithmetic` values from the metadata are applied via
255+
the regenerated `fe.toml` files (`dependency-arithmetic` is deliberately not re-applied; the
256+
metadata records post-forcing values).
257+
- **Version check**: if `compiler.version` differs from the running compiler, a warning is printed
258+
to stderr (no abort); exact bytecode reproduction is only guaranteed with the same version.
259+
- **Output**: artifacts are selected with `--emit` as usual and written to `--out-dir`, which
260+
defaults to `./out` (relative to the current working directory, since there is no project
261+
directory to anchor to).
262+
263+
Errors (exit code 1): unreadable input, invalid JSON, `language != "Fe"`, missing `sources`, or a
264+
missing/rootless `settings.ingots`. Combining `--from-metadata` with the `[path]` argument,
265+
`--ingot`, `--standalone`, or `--report` is a CLI usage error (exit code 2).
266+
237267
### Reports: `--report`
238268

239269
`fe build` can optionally write a `.tar.gz` debugging report (useful for sharing failures):

crates/fe/src/build.rs

Lines changed: 206 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -267,6 +267,7 @@ pub fn build(
267267
&dir_path,
268268
ingot,
269269
contract,
270+
None,
270271
opt_level,
271272
emit,
272273
out_dir,
@@ -306,6 +307,144 @@ pub fn build(
306307
}
307308
}
308309

310+
/// `fe build --from-metadata`: rebuild from a `metadata.json` recompilation
311+
/// input instead of a source tree. Reads the metadata (from a file, or stdin
312+
/// for `-`), materializes the recorded project into a temporary directory,
313+
/// and runs the normal ingot build on it. Exits the process on failure.
314+
pub fn build_from_metadata(
315+
input: &Utf8Path,
316+
contract: Option<&str>,
317+
optimize: Option<&str>,
318+
emit: &[BuildEmit],
319+
out_dir: Option<&Utf8PathBuf>,
320+
profile: &str,
321+
use_recovery_mode: bool,
322+
) {
323+
let emit = EmitSelection::from_requested(emit);
324+
325+
let metadata = match crate::metadata_input::read_metadata(input) {
326+
Ok(metadata) => metadata,
327+
Err(err) => {
328+
eprintln!("Error: {err}");
329+
std::process::exit(1);
330+
}
331+
};
332+
if let Err(err) = crate::metadata_input::validate_metadata(&metadata) {
333+
eprintln!("Error: {err}");
334+
std::process::exit(1);
335+
}
336+
337+
// Exact reproduction is only guaranteed with the same compiler version.
338+
if let Some(version) = metadata["compiler"]["version"].as_str() {
339+
let running = env!("CARGO_PKG_VERSION");
340+
if version != running {
341+
eprintln!(
342+
"Warning: metadata was produced by fe {version}, but this is fe {running}; \
343+
the rebuilt bytecode may differ from the original"
344+
);
345+
}
346+
}
347+
// Non-release builds share a package version; the recorded commit pins the
348+
// exact source revision.
349+
if let (Some(recorded), Some(running)) = (
350+
metadata["compiler"]["commit"].as_str(),
351+
option_env!("FE_GIT_HASH").filter(|hash| !hash.is_empty()),
352+
) && recorded != running
353+
{
354+
eprintln!(
355+
"Warning: metadata was produced by fe commit {recorded}, but this is fe commit \
356+
{running}; the rebuilt bytecode may differ from the original"
357+
);
358+
}
359+
360+
// The metadata's optimizer level is the default; an explicit `-O` wins.
361+
let recorded_level = metadata["settings"]["optimizer"]["level"].as_str();
362+
let level = match (optimize, recorded_level) {
363+
(Some(flag), Some(recorded)) if flag != recorded => {
364+
eprintln!(
365+
"Warning: -O {flag} overrides optimizer level {recorded} recorded in the \
366+
metadata; the rebuilt bytecode will not match the verified artifact"
367+
);
368+
flag
369+
}
370+
(Some(flag), _) => flag,
371+
(None, Some(recorded)) => recorded,
372+
(None, None) => "1",
373+
};
374+
let opt_level: OptLevel = match level.parse() {
375+
Ok(level) => level,
376+
Err(err) => {
377+
eprintln!("Error: {err}");
378+
std::process::exit(1);
379+
}
380+
};
381+
382+
// Default target: the contract recorded in `compilationTarget`; an
383+
// explicit `--contract` overrides it.
384+
let recorded_target = metadata["settings"]["compilationTarget"]
385+
.as_object()
386+
.and_then(|target| target.iter().next())
387+
.and_then(|(path, name)| Some((path.clone(), name.as_str()?.to_string())));
388+
let contract = contract
389+
.map(str::to_string)
390+
.or_else(|| recorded_target.as_ref().map(|(_, name)| name.clone()));
391+
// The recorded source path pins which module defines the target, in case
392+
// another module defines a same-named contract; it only applies while
393+
// building the recorded contract.
394+
let target_source = recorded_target
395+
.filter(|(_, name)| contract.as_deref() == Some(name))
396+
.map(|(path, _)| path);
397+
398+
let temp = match tempfile::Builder::new()
399+
.prefix("fe-from-metadata")
400+
.tempdir()
401+
{
402+
Ok(temp) => temp,
403+
Err(err) => {
404+
eprintln!("Error: Failed to create temporary project directory: {err}");
405+
std::process::exit(1);
406+
}
407+
};
408+
let Some(temp_root) = Utf8Path::from_path(temp.path()) else {
409+
eprintln!("Error: temporary project directory path is not valid UTF-8");
410+
std::process::exit(1);
411+
};
412+
let root_dir = match crate::metadata_input::reconstruct_project(&metadata, temp_root) {
413+
Ok(dir) => dir,
414+
Err(err) => {
415+
eprintln!("Error: {err}");
416+
std::process::exit(1);
417+
}
418+
};
419+
420+
let out_dir = out_dir.cloned().unwrap_or_else(|| Utf8PathBuf::from("out"));
421+
422+
let mut db = DriverDataBase::default();
423+
db.compiler_options()
424+
.set_recovery_mode(&mut db)
425+
.to(use_recovery_mode);
426+
db.compilation_settings()
427+
.set_profile(&mut db)
428+
.to(profile.into());
429+
430+
let had_errors = build_directory(
431+
&mut db,
432+
&root_dir,
433+
None,
434+
contract.as_deref(),
435+
target_source.as_deref(),
436+
opt_level,
437+
emit,
438+
Some(&out_dir),
439+
None,
440+
);
441+
442+
drop(temp);
443+
if had_errors {
444+
std::process::exit(1);
445+
}
446+
}
447+
309448
#[allow(clippy::too_many_arguments)]
310449
fn build_file(
311450
db: &mut DriverDataBase,
@@ -414,6 +553,7 @@ fn build_directory(
414553
dir_path: &Utf8PathBuf,
415554
ingot: Option<&str>,
416555
contract: Option<&str>,
556+
target_source: Option<&str>,
417557
opt_level: OptLevel,
418558
emit: EmitSelection,
419559
out_dir: Option<&Utf8PathBuf>,
@@ -487,6 +627,7 @@ fn build_directory(
487627
db,
488628
&url,
489629
contract,
630+
target_source,
490631
opt_level,
491632
emit,
492633
&out_dir,
@@ -602,6 +743,7 @@ fn build_workspace(
602743
db,
603744
&matches[0].url,
604745
Some(contract),
746+
None,
605747
opt_level,
606748
emit,
607749
&out_dir,
@@ -654,6 +796,7 @@ fn build_workspace(
654796
db,
655797
&member.url,
656798
None,
799+
None,
657800
opt_level,
658801
emit,
659802
&out_dir,
@@ -853,6 +996,7 @@ fn build_ingot_url(
853996
db: &mut DriverDataBase,
854997
ingot_url: &Url,
855998
contract: Option<&str>,
999+
target_source: Option<&str>,
8561000
opt_level: OptLevel,
8571001
emit: EmitSelection,
8581002
out_dir: &Utf8Path,
@@ -899,6 +1043,7 @@ fn build_ingot_url(
8991043
db,
9001044
ingot,
9011045
contract,
1046+
target_source,
9021047
opt_level,
9031048
emit,
9041049
out_dir,
@@ -914,6 +1059,7 @@ fn build_ingot(
9141059
db: &DriverDataBase,
9151060
ingot: hir::Ingot<'_>,
9161061
contract: Option<&str>,
1062+
target_source: Option<&str>,
9171063
opt_level: OptLevel,
9181064
emit: EmitSelection,
9191065
out_dir: &Utf8Path,
@@ -930,6 +1076,21 @@ fn build_ingot(
9301076
}
9311077
};
9321078

1079+
// `--from-metadata` records the contract's defining source file in
1080+
// `settings.compilationTarget`; scope emission to that module so a
1081+
// same-named contract in another module cannot make the target ambiguous.
1082+
let target_mod = match (target_source, contract) {
1083+
(Some(key), Some(name)) => {
1084+
let mut sources = BTreeMap::new();
1085+
collect_ingot_sources_prefixed(db, ingot, None, &mut sources);
1086+
sources
1087+
.get(key)
1088+
.map(|(top_mod, _)| *top_mod)
1089+
.filter(|top_mod| module_defines_contract(db, *top_mod, name))
1090+
}
1091+
_ => None,
1092+
};
1093+
9331094
if contract_names.is_empty() {
9341095
eprintln!("Error: No contracts found to build");
9351096
return BuildSummary { had_errors: true };
@@ -949,8 +1110,13 @@ fn build_ingot(
9491110
let mut had_errors = false;
9501111
if emit.ir || emit.writes_any_bytecode() {
9511112
if emit.ir {
952-
let ir = match codegen::emit_ingot_sonatina_ir_optimized(db, ingot, opt_level, contract)
953-
{
1113+
let ir = match target_mod {
1114+
Some(top_mod) => {
1115+
codegen::emit_module_sonatina_ir_optimized(db, top_mod, opt_level, contract)
1116+
}
1117+
None => codegen::emit_ingot_sonatina_ir_optimized(db, ingot, opt_level, contract),
1118+
};
1119+
let ir = match ir {
9541120
Ok(ir) => ir,
9551121
Err(err) => {
9561122
eprintln!("Error: Failed to compile Sonatina IR: {err}");
@@ -965,14 +1131,19 @@ fn build_ingot(
9651131
}
9661132
}
9671133
if emit.writes_any_bytecode() {
968-
let bytecode =
969-
match codegen::emit_ingot_sonatina_bytecode(db, ingot, opt_level, contract) {
970-
Ok(bytecode) => bytecode,
971-
Err(err) => {
972-
eprintln!("Error: Failed to compile Sonatina bytecode: {err}");
973-
return BuildSummary { had_errors: true };
974-
}
975-
};
1134+
let bytecode = match target_mod {
1135+
Some(top_mod) => {
1136+
codegen::emit_module_sonatina_bytecode(db, top_mod, opt_level, contract)
1137+
}
1138+
None => codegen::emit_ingot_sonatina_bytecode(db, ingot, opt_level, contract),
1139+
};
1140+
let bytecode = match bytecode {
1141+
Ok(bytecode) => bytecode,
1142+
Err(err) => {
1143+
eprintln!("Error: Failed to compile Sonatina bytecode: {err}");
1144+
return BuildSummary { had_errors: true };
1145+
}
1146+
};
9761147
had_errors |= write_sonatina_bytecode_artifacts(
9771148
&names_to_build,
9781149
&bytecode,
@@ -987,6 +1158,9 @@ fn build_ingot(
9871158
// For ingot builds, generate ABI from each top-level module
9881159
use hir::hir_def::HirIngot;
9891160
for top_mod in ingot.all_modules(db) {
1161+
if target_mod.is_some_and(|target| *top_mod != target) {
1162+
continue;
1163+
}
9901164
had_errors |= write_abi_artifacts(db, *top_mod, &names_to_build, out_dir, report_dir);
9911165
}
9921166
}
@@ -1124,6 +1298,28 @@ fn collect_ingot_contract_names(
11241298
Ok(names)
11251299
}
11261300

1301+
/// Whether `top_mod` defines `name` as a contract or manual contract root — the
1302+
/// same definitions `write_metadata_artifacts` records in `compilationTarget`.
1303+
fn module_defines_contract(db: &DriverDataBase, top_mod: TopLevelMod<'_>, name: &str) -> bool {
1304+
if top_mod
1305+
.all_contracts(db)
1306+
.iter()
1307+
.any(|contract| contract.name(db).to_opt().is_some_and(|n| n.data(db) == name))
1308+
{
1309+
return true;
1310+
}
1311+
top_mod.all_funcs(db).iter().any(|&func| {
1312+
func.top_mod(db) == top_mod
1313+
&& match func.manual_contract_root_attr(db) {
1314+
Some(ManualContractRootAttr::Init { contract_name })
1315+
| Some(ManualContractRootAttr::Runtime { contract_name }) => {
1316+
contract_name.data(db) == name
1317+
}
1318+
Some(ManualContractRootAttr::Error(_)) | None => false,
1319+
}
1320+
})
1321+
}
1322+
11271323
fn collect_workspace_contract_names(db: &DriverDataBase, ingot: hir::Ingot<'_>) -> Vec<String> {
11281324
let mut names = BTreeSet::new();
11291325
for &top_mod in ingot.all_modules(db) {

0 commit comments

Comments
 (0)