feat(log): emit build-started JSON message with run_id - #16632

r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

cc @Kobzol @soqb

Until a better solution is out, I think this is sufficient for your use case (with some extra probe to ~/.cargo/log/ directory). Let me know if this is suffcient, or you have some other wild idea. I should create a separate issue discussing the interaction between logging and JSON message.

So, for my use-case with rustwide, it doesn't help :( Because rustwide (used e.g. by crater) runs Rust crates in a Docker container where CARGO_HOME is mounted as read-only:

Error: Experiment run failed

Caused by:
    command failed: exit status: 101
    
    error: failed to create directory `/opt/rustwide/cargo-home/log`
    
    Caused by:
      Read-only file system (os error 30)

If the writing into CARGO_HOME ever became performed default, I guess this would also currently break crater 😆

I think we have a few options here:

• Redirect log to stdout
• A configurable log directory path

The second one is less ambiguous and straightforward to implement, though I am not sure if that meets your use case

It would work for the Docker/container use-case, and making the log dir be configurable sounds like a good idea regardless. (streaming to stdout would also be appreciated, but I guess that can wait until there's more design work done on -Zbuild-analysis)

i am in roughly the same boat as @Kobzol here. a configurable log dir is necessary for my use case.

Useful for rust-lang/crater#825
Zulip

Created #16696 for specifying log file paths.

Just noting that I wanted to do a benchmark for #8545 and measure only the duration of building the build script itself, for which --timings=json, or this PR, would be quite useful. Otherwise I don't really know how to extract the durations of individual crates out of Cargo directly at the moment.

@Kobzol you not only need this but also #16696, right?

For the large-scale rustwide experiment, yeah. But now I don't even know how to run a one-off measurement to get the build script duration, other than eyeballing the HTML timings page or patching Cargo.

Just noting that I wanted to do a benchmark for #8545 and measure only the duration of building the build script itself, for which --timings=json, or this PR, would be quite useful. Otherwise I don't really know how to extract the durations of individual crates out of Cargo directly at the moment.

Noting that this has also been happening to the handful of people doing performance evaluations to enable the new solver and polonius alpha on nightly.

This PR would still be useful for people to analyze these experiments' results and have per-dependency stats instead of the coarse grained full crate graph results. An alternative is ofc to parse the HTML from --timings... With this PR they should be able to more easily see the slowdowns/improvements from e.g. the next solver, on each crate.

For my own experiments, strictly speaking I'm in the same boat as Jakub, doing crater-like rustwide experiments, and thus would also need #16696, so it's fine.

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@epage

Would you mind taking a look at this? I think even without #16696, this is useful standalone and help do analysis locally. At least better than parsing HTML. The message is behind -Zbuild-analysis of course. And we should have a holistic design for the potential v2 JSON message.

(See #16632 (comment))

The re-review of this will definitely be a blocker before stabilization.

Can you call out an item for this in the tracking issue?

Can you call out an item for this in the tracking issue?

Done. Thanks!