Skip to content

optionally collect compiler metrics #2812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
+36 −1
Draft

optionally collect compiler metrics #2812

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,11 @@ services:
- "/var/run/docker.sock:/var/run/docker.sock"
- ".rustwide-docker:/opt/docsrs/rustwide"
- "cratesio-index:/opt/docsrs/prefix/crates.io-index"
- "./ignored/cratesfyi-prefix/metrics:/opt/docsrs/prefix/metrics"
- "./static:/opt/docsrs/static:ro"
environment:
DOCSRS_RUSTWIDE_WORKSPACE: /opt/docsrs/rustwide
DOCSRS_COMPILER_METRICS_PATH: /opt/docsrs/prefix/metrics
DOCSRS_DATABASE_URL: postgresql://cratesfyi:password@db
DOCSRS_STORAGE_BACKEND: s3
S3_ENDPOINT: http://s3:9000
Expand Down
6 changes: 6 additions & 0 deletions src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ pub struct Config {
// for the remote archives?
pub(crate) local_archive_cache_path: PathBuf,

// Where to collect metrics for the metrics initiative.
// When empty, we won't collect metrics.
pub(crate) compiler_metrics_collection_path: Option<PathBuf>,

// Content Security Policy
pub(crate) csp_report_only: bool,

Expand Down Expand Up @@ -226,6 +230,8 @@ impl Config {
prefix.join("archive_cache"),
)?,

compiler_metrics_collection_path: maybe_env("DOCSRS_COMPILER_METRICS_PATH")?,

temp_dir,

rustwide_workspace: env("DOCSRS_RUSTWIDE_WORKSPACE", PathBuf::from(".workspace"))?,
Expand Down
29 changes: 28 additions & 1 deletion src/docbuilder/rustwide_builder.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -862,6 +862,18 @@ impl RustwideBuilder {
})
};

if let Some(compiler_metric_target_dir) = &self.config.compiler_metrics_collection_path {
let metric_output = build.host_target_dir().join("metrics/");
info!(
"found {} files in metric dir, copy over to {} (exists: {})",
fs::read_dir(&metric_output)?.count(),
&compiler_metric_target_dir.to_string_lossy(),
&compiler_metric_target_dir.exists(),
);
copy_dir_all(&metric_output, compiler_metric_target_dir)?;
fs::remove_dir_all(&metric_output)?;
}

// For proc-macros, cargo will put the output in `target/doc`.
// Move it to the target-specific directory for consistency with other builds.
// NOTE: don't rename this if the build failed, because `target/doc` won't exist.
Expand Down Expand Up @@ -945,7 +957,7 @@ impl RustwideBuilder {
];

rustdoc_flags_extras.extend(UNCONDITIONAL_ARGS.iter().map(|&s| s.to_owned()));
let cargo_args = metadata.cargo_args(&cargo_args, &rustdoc_flags_extras);
let mut cargo_args = metadata.cargo_args(&cargo_args, &rustdoc_flags_extras);

// If the explicit target is not a tier one target, we need to install it.
let has_build_std = cargo_args.windows(2).any(|args| {
Expand All @@ -966,6 +978,21 @@ impl RustwideBuilder {
command = command.env(key, val);
}

if self.config.compiler_metrics_collection_path.is_some() {
// set the `./target/metrics/` directory inside the build container
// as a target directory for the metric files.
let flag = "-Zmetrics-dir=/opt/rustwide/target/metrics";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to make it work?

Copy link
Member Author

@syphar syphar May 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, it works like this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

( there are output files generated)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for:

#![feature(doc_cfg)]

//! ```
//! macro_rules! bla {
//!     ($($x:tt)*) => {}
//! }
//! ```

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a crate / release on crates.io I can just run it with?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's just a test I wrote locally. You can do so as well:

cargo new --lib foo
cd foo
# copy content into it
# run cargo doc


// this is how we can reach it from outside the container.
fs::create_dir_all(build.host_target_dir().join("metrics/"))?;

let rustflags = toml::Value::try_from(vec![flag])
.expect("serializing a string should never fail")
.to_string();
cargo_args.push("--config".into());
cargo_args.push(format!("build.rustflags={rustflags}"));
}

Ok(command.args(&cargo_args))
}

Expand Down
Loading