pyo3 crate recompiled on (every?) build

[cswinter]

I don't have a minimal repro yet, but the pyo3 crate seems to be recompiled on most builds which significantly increases compilation times. I'm on pyo3 version 0.13.2, rustc version 1.53.0.

[davidhewitt]

See also #1551

This might be fixed on main / upcoming 0.14 release depending on your build environment.

I think that rust-analyzer runs things like cargo check which can invalidate pyo3's build config unless the environments carefully match.

This is worthy of a topic in the guide, and probably there could be more discussion whether we need to make further changes.

[cswinter]

Ah good to know! I tried setting PYO3_PYTHON both when building manually and in rust-analyzer.server.extraEnv but that doesn't seem to have done the trick yet. I think I'll just remove PyO3 completely from Rust builds for now which I have mostly set up that way anyway already because of that linker issue. Happy to do some more testing/debugging on this once 0.14 lands.

[davidhewitt]

Thanks. I've also been considering reaching out to the rust-analyzer folks who might be more familiar with all the cases when pyo3's build.rs is invoked, however haven't gotten around to that yet.

[mejrs]

I have pyo3 as an optional dependency for this reason.

[dbr]

I have this happening semi-randomly (happens "almost all the time", but, not always). I am running rust-analyzer (in emacs via lsp-mode), and mostly notice this when switching to terminal to cargo run. It was mysterious how rust-analyzer was causing the command-line builds to change, so I looked into it a bit:

Running build with

CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build

I get the following output (can probably change 'trace' to 'info')

INFO  cargo::core::compiler::fingerprint]     err: env var `PATH` changed: previously Some("/home/dbr/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"), now Some("/home/dbr/.cargo/bin:/home/dbr/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games")

Basically ~/.cargo/bin is in $PATH twice in rust-analyzer but only once in regular build (I assume, might be other way around), which then triggers a re-build of PyO3 as intended (as per the #1551 issue David mentioned above)

rust-analyzer uses the same target/ directory, which is how it can interfere with running cargo build. Specifically, the files like target/debug/.fingerprint/pyo3-a2b72c2ee6b7ccd3 stores the previous value of $PATH like so:

[..snip..] {"RerunIfEnvChanged":{"var":"PATH","val":"/home/dbr/.cargo/bin:/home/dbr/.cargo/bin:[..snip..]"}}],"rustflags":[],"metadata":0,"config":0,"compile_kind":0} [..snip..]

So I guess this issue basically becomes: why is ~/.cargo/bin getting added to $PATH twice (which is most certainly an issue for either rust-analyzer, or possibly cargo?)

[dbr]

After much poking around, I think the problem lies in rustup (not rust-analyzer nor cargo), so have filed a bug report in that project:

rust-lang/rustup#2848

For now I think configuring CARGO_TARGET_DIR for rust-analyzer would seem to be the most reliable workaround (at the cost of having two separate builds)

[davidhewitt]

Thanks for the investigation! If the rustup bug getting fixed does indeed remove this frequently-encountered papercut, I'll be very happy 😄