RFC: Crate changelog field

[kxxt]

Rendered

This is the second attempt to introduce the changelog field into crate manifest.

Previous attempt: #2129

Comparing with previous attempt,
this RFC does not impose any format constraint over the contents of the changelog and instead only requires an external URL.

This RFC also reserves the possibility of using a local file included in crate as changelog while leaving it to future RFCs.

[epage]

This doesn't discuss the other half of the reason the other RFC was closed

If the proposal then reduces to just a link that will be displayed, then it seems to not bring much value over placing a link within a README.

See #2129 (comment)

[kxxt]

Addressed in f36581d

[epage]

Not seeing this add anything new compared to when this was previously rejected.

[kxxt]

Not seeing this add anything new compared to when this was previously rejected.

The same argument about reducing to just a link could also be made about HomePage and Documentation fields.
We could just link to the crate home page in README and live without the homepage field.
But I don't expect anyone will be willing to see the homepage field removed.

The value those fields bring really are just ease of discoverability, which makes them easier to discover on crates.io, docs.rs and by other external tools.

And there's nothing wrong about doing it step by step. Possibility of strictly formatted machine readable changelog could be introduced as a future RFC without breaking existing usage of this field.

[epage]

I want to confirm a framing for this conversation: the most important thing is for the RFC to acknowledge that this was previously rejected and talk to that. Justifying it solely by homepage, documentation, and repository is not a very strong case because that was known at the time.

The same argument about reducing to just a link could also be made about HomePage and Documentation fields.
We could just link to the crate home page in README and live without the homepage field.

For most crates, documentation has become irrelevant because crates.io automatically links to docs.rs (and to the right version).

That leaves homepage and repository. I see these as important options for finding the origin of a crate, important enough that we require one of them to be set when publishing. I do not see us requiring a changelog to publish though.

Something else to keep in mind is that we previously had badges which could contain things like links to CI, etc but were later dropped (rust-lang/crates.io#2436). These extra links were helpful before the README was rendered but became redundant with it. Some also included mutable metadata that required a new publish to update (maintenance status). Where people publish a changelog is also generally independent of the specific .crate file and seems like it would also be in the category of mutable metadata (e.g. cargo just moved its changelog). We do not yet have a solution for mutable metadata (a reference to it is made in the 1.78 update post).

[kornelski]

For https://lib.rs, I've tried to guess changelog URLs, and found these problems:

• Crates may have a changelog file in the repo, but repos are annoying to deal with, especially at scale:

  • cloning of the repos just to check the files is a massive operational headache. That's a lot of data, lots of files. Some repos are massive and take an hour to clone. Shallow clones are slow to update, and flaky (e.g. updates cause bloat that triggers expensive git gc, which may break the local copy if interrupted).
  • Plenty of crates have missing or broken repo URLs. Some point to wrong repos (not updated after a fork). Sometimes crates are in a branch, and cloning the main branch gets a wrong thing. Some crates use pijul or hg, or point to a web front-end, not cloneable URL.
  • In monorepos, it's hard to find the relevant file. The /Changelog.md at the root may apply to some primary crate only, or no crates at all. Sometimes it may contain "For the crate X, please see crates/X/Changelog.md". Conversely, there may be a crates/X/Changelog.md that says "We track all changes in the /Changelog.md", or may be a symlink to such file, that really contains all changes for all crates in the repo.
  • People use CHANGELOG.md, Changelog.md, changelog, changes.txt, History.md, HISTORY, etc.

• GitHub-based projects sometimes use GitHub's Releases as a changelog substitute. The GitHub API allows checking if Releases exist, but they may be outdated (project stopped updating them, only does cargo publish), or may be garbage (e.g. release notes are always "please see Changelog.md instead!")

A simple changelog URL won't solve all of these issues, but it can help:

• Crates have a way to be explicit about the preferred location (disambiguate whether they use GH Releases, or a file in the repo, and where it is in a monorepo). changelog = false could be supported too, as a way to opt-out of any default guesswork.
• A single HTTP URL is easier to fetch than a repo (GitHub allows fetching individual repo files via HTTP, but that's GitHub-specific, and requires guessing the file path).

[kornelski]

A changelog URL is useful outside of a README. It can be displayed in a context of a "Just Updated" section of a crate index, where users may wonder what has been updated.

It could be displayed in a summary of cargo update, especially when Cargo can't update a dependency because of semver-major change. In such case the next action for the user is to check what has been broken to see if it's safe to upgrade.

The changelog doesn't have to be parseable, but when it is, it could be neatly integrated into crate's detailed /versions page.

Getting such URL out of a README requires ability to parse the markup, and heuristics to find a relevant link. Something like <a rel=changelog …> could work to make it unambiguous, but the solution is non-obvious, especially in Markdown. HTML parsing is harder than TOML parsing.

The content of the README is user-visible, and not all projects may think it's the right place to advertise a changelog. Metadata gives ability to display the link in the right context, without the rest of the readme.

[mathstuf]

Of particular interest to me is showing the difference between the current version (either in Cargo.toml or Cargo.lock) and the current version. Beyond the "bug fixes and performance improvements" non-statements that are so rife today, it is also the changelog between the previous update and the current one that I really care about, not the one that I have now and the current one.

I understand that changelog formats are all over the place, so getting agreement there is unlikely to get any useful consensus, but at least pointing me at the document itself can get me the information I'm looking for.

[mathstuf]

If anyone could teach Dependabot about that stuff as well, I think we might get more useful update PRs and incentives to actually make (useful) changelogs.

[epage]

Mechanism vs policy is tricky.

From a mechanism perspective, this seems ok (these two pieces fit together, let people create whatever solution they want).

From a policy perspective, I worry this will lead to a lot of incorrect links. A repository link is rightfully shared in a workspace. clap, clap_derive, and clap_builer can rightfully share a changelog. But once its in workspace.package, it is unclear what all its relevant for. For example, cargo new will automatically inherit everything in workspace.package which can lead to inheriting irrelevant changelogs. For example, if I run cargo new clap_complete_fish, I'll get claps changelog. That could be easy to overlook and only find out if you test our your own links once published.

The example is given of documentation.workspace. We didn't think too much on what field to include (there are other, more clear cases of regrets for what we allowed to inherit) and have often overlooked that because it rarely needs to be set these days because crates.io infers it.

[epage]

Another drawback: this requires treating packages in a workspace as a pet (individual care), rather than cattle (only group care) which we've been moving away from with features like workspace inheritance, automatic docs.rs links etc.

If I were to use this in a workspace, I have to manually set up the link when creating or moving a workspace member without any way to verify it until I push to the upstream repo. By the time the new package is merged, I'm unlikely to remember to go back and test the link. I'm also unlikely to remember to test when publishing. Most likely, I'll only discover there is a problem when someone reports it. Today, the amount of per-package metadata I have to enter is minimal and is almost all verified locally or at least before publish.

[michalfita]

These last few comments let me to conclusion that cargo publish shall require publication from tagged commit (tricky as git is golden standard, but not only VCS used), then commits list starting from previous tag shall be included as formal changelog - this would eliminate crates without references to repositories (plenty) and provide guarantee the content published on the crates.io is consisted with code changes on that repository (today rouge party could publish something not in the repo if managed to steal the key).

But I feel I'm running too far with this idea in the context of this RFC.

Whatever angle I look at this, I see (as with docs.rs) there should be sensible defaults for the changelog (peekable from crates.io for each new release/publication), for me this looks like CHANGELOG.md in directory of each crate with fallback to CHANGLOG.md of a workspace with some flag of this being the case.

The whole idea only makes sense of the changelog shows changes for particular release, not some generic journal of changes.

[epage]

This doesn't just require deeper git integration (and lack of support for other vcs') but Cargo to know your tag format and to handle changes in tag formats. As mentioned elsewhere, commits are also a fairly low quality changelog. That can be improved through conventions but then we'd need to enforce those conventions. Those conventions also do not help when a commit crosses multiple packages.

The whole idea only makes sense of the changelog shows changes for particular release, not some generic journal of changes.

The challenge is doing so without running into the problems #2129 ran into.

At least with this RFC, if I were to feel comfortable setting this up for all workspace members, I would tweak my changlogs so the anchors are more predictable and setup a cargo-release regex to update the url anchor used in package.changlog every release so it at least points to the right location.

[weihanglo]

--no-metadata is a better fit.

Suggested change

	unless `--no-verify` is also specified.
	unless `--no-metadata` is also specified.

[epage]

If its a url, shouldn't that be part of the schema and always error (on any command), if present?

[kxxt]

My initial idea was that when non-url path become supported for changelog in the future, someone publishing a crate with it using an older cargo might want to skip this check.

But I agree with @weihanglo that we should use a different flag because --no-verify is skipping too many checks.

[epage]

If parsers are supposed to get any meaning from the field, it likely needs one format. Alternative formats could be handled with alternate fields (like license and license-file) or dotted keys (e.g. this could be changelog.url)

[weihanglo]

If parsers are supposed to get any meaning from the field…

Agree with this. It depends on how much functionality we expect from this field.

[Aloso]

While markdown is currently the most popular choice, plain text and reStructuredText are also in use. Instead of handling the rendering of various different formats of changelog in crates.io, it is much easier to just have an external link

crates.io is already able to render READMEs written in Markdown. Other formats like reStructuredText are not supported, but that doesn't seem to be a big problem.

Cargo doesn't enforce READMEs to be written in Markdown, even .txt is allowed. But since Markdown looks nicer on crates.io, people are encouraged to use it. The same could apply to changelog files.

crates.io could add support for other formats like RST and AsciiDoc, but it's not a requirement to support changelogs as local files.