Merge master into dependabot/github_actions/actions-4a3a8eedec

Author: pront

.github/ISSUE_TEMPLATE/minor-release.md

@@ -35,44 +35,22 @@ closes the window automatically once both are done.
 
 # Publish the release
 
-- [ ] On release day, squash-merge the approved preparation PR directly into `master` using the
-      release-freeze bypass. Do not use the merge queue.
+- [ ] On release day, squash-merge the approved preparation PR directly into `master` using admin
+      permissions (`Merge without waiting for requirements to be met (bypass rules)`). Do not use the merge queue.
 
-The merge approves the release. [Tag approved release](https://github.com/vectordotdev/vector/actions/workflows/release_autotag.yml)
-validates the squash-merge commit and creates the version tag and release branch at that exact commit.
+The merge kicks off the [Tag approved release](https://github.com/vectordotdev/vector/actions/workflows/release_autotag.yml)
+workflow, which creates the version tag and release branch at that exact commit.
 The tag starts the release workflow; do not create the tag or release branch manually.
 
-- [ ] Wait for release workflow to complete.
-  - Discoverable via [release.yml](https://github.com/vectordotdev/vector/actions/workflows/release.yml)
-- [ ] Wait for the release workflow to reset the `website` branch to the release commit,
-      publishing the release notes to https://vector.dev
-  - [ ] Confirm that the release changelog was published to https://vector.dev/releases/
-    - Refer to the internal releasing doc to monitor the deployment.
+- [ ] Wait for [release workflow](https://github.com/vectordotdev/vector/actions/workflows/release.yml) to complete.
+  - If it fails, use **Re-run failed jobs** to continue the release.
+- [ ] Confirm that the release changelog was published to https://vector.dev/releases/
+  - Refer to the internal releasing doc to monitor the deployment.
+- [ ] Confirm that [Homebrew](https://github.com/vectordotdev/homebrew-brew) was released ([workflow](https://github.com/vectordotdev/homebrew-brew/actions/workflows/release.yml))
 - [ ] Release Linux packages. Refer to the internal releasing doc.
-- [ ] Wait for the [Homebrew release](https://github.com/vectordotdev/homebrew-brew/actions/workflows/release.yml) to complete.
-  - The Vector release workflow starts it automatically after publishing a stable release.
-    It updates the ARM64 formula and commits directly to the tap's default branch, without a PR.
-- [ ] Update the latest [release tag](https://github.com/vectordotdev/vector/releases) description with the release announcement.
 
-# Post-release housekeeping
+- [ ] Wait for the Helm chart [Post Release](https://github.com/vectordotdev/helm-charts/actions/workflows/release-post.yml) to complete.
+  - See [releasing Helm chart](https://github.com/vectordotdev/helm-charts/blob/develop/RELEASING.md).
 
-- [ ] Wait for the release workflow to push its post-release housekeeping directly to `master`.
-      It begins the next minor `-dev` version, restores VRL `main`, and refreshes licenses and documentation.
-  - If it fails, use **Re-run failed jobs**; this also starts the Helm release.
-- [ ] Wait for the [Helm chart release](https://github.com/vectordotdev/helm-charts/actions/workflows/release-prepare.yml).
-  - It starts after housekeeping and pushes directly, without a PR.
-    See [releasing Helm chart](https://github.com/vectordotdev/helm-charts/blob/develop/RELEASING.md) if it fails.
-- [ ] Wait for the Helm chart release to push the Kubernetes manifests directly to `master`.
-  - The chart release triggers [Refresh Kubernetes manifests](https://github.com/vectordotdev/vector/actions/workflows/release_manifests.yml),
-    which runs `cargo vdev build manifests` and, when the generated manifests differ,
-    commits and pushes them to `master` itself as the `vectordotdev-bot` — no PR, no review,
-    no merge queue. If the run reports no changes, the manifests already match the chart.
-- [ ] Wait for the [Unfreeze master](https://github.com/vectordotdev/vector/actions/workflows/release_unfreeze.yml)
-      workflow to close the direct-push window after the manifests run succeeds.
-  - It removes the temporary `vectordotdev-bot` **Always** bypass from every ruleset in
-    `RELEASE_FREEZE_BOT_BYPASS`, then sets the `RELEASE_FREEZE_RULESET_ID` ruleset back to **Disabled**.
-    It waits for the "Release Suite" run and any pending release workflow first,
-    and gives up after ten minutes, leaving the freeze active.
-  - Run it manually with `workflow_dispatch` if the release never starts the manifests workflow, if that
-    run fails, or to retry a failed closeout.
-    A manual run makes the same checks unless you set `force`, which closes the window anyway.
+- [ ] Wait for the Helm chart release to push the Kubernetes manifests directly to `master` ([workflow](https://github.com/vectordotdev/vector/actions/workflows/release_manifests.yml)).
+- [ ] Wait for the [Unfreeze master](https://github.com/vectordotdev/vector/actions/workflows/release_unfreeze.yml) workflow to finalize the release.

vdev/src/commands/check/mod.rs

@@ -11,26 +11,49 @@ mod markdown;
 mod rust;
 mod scripts;
 
-crate::cli_subcommands! {
-    "Check parts of the Vector code base..."
-    changelog_fragments,
-    generated_docs,
-    component_features,
-    component_examples,
-    deny,
-    docs,
-    events,
-    examples,
-    fmt,
-    licenses,
-    markdown,
-    rust,
-    scripts,
+use crate::utils::command::ScriptArgs;
+
+/// Check parts of the Vector code base...
+#[derive(clap::Args, Debug)]
+pub(super) struct Cli {
+    #[command(subcommand)]
+    command: Commands,
 }
 
-// These should eventually be migrated to Rust code
+#[derive(clap::Subcommand, Debug)]
+enum Commands {
+    ChangelogFragments(changelog_fragments::Cli),
+    GeneratedDocs(generated_docs::Cli),
+    ComponentFeatures(component_features::Cli),
+    ComponentExamples(component_examples::Cli),
+    Deny(deny::Cli),
+    /// Check that all /docs files are valid
+    Docs(ScriptArgs),
+    Events(events::Cli),
+    Examples(examples::Cli),
+    Fmt(fmt::Cli),
+    Licenses(licenses::Cli),
+    Markdown(markdown::Cli),
+    Rust(rust::Cli),
+    Scripts(scripts::Cli),
+}
 
-crate::script_wrapper! {
-    docs = "Check that all /docs files are valid"
-        => "check-docs.sh"
+impl Cli {
+    pub fn exec(self) -> anyhow::Result<()> {
+        match self.command {
+            Commands::ChangelogFragments(cli) => cli.exec(),
+            Commands::GeneratedDocs(cli) => cli.exec(),
+            Commands::ComponentFeatures(cli) => cli.exec(),
+            Commands::ComponentExamples(cli) => cli.exec(),
+            Commands::Deny(cli) => cli.exec(),
+            Commands::Docs(args) => args.exec("check-docs.sh"),
+            Commands::Events(cli) => cli.exec(),
+            Commands::Examples(cli) => cli.exec(),
+            Commands::Fmt(cli) => cli.exec(),
+            Commands::Licenses(cli) => cli.exec(),
+            Commands::Markdown(cli) => cli.exec(),
+            Commands::Rust(cli) => cli.exec(),
+            Commands::Scripts(cli) => cli.exec(),
+        }
+    }
 }

vdev/src/commands/mod.rs

@@ -113,25 +113,76 @@ cli_commands! {
     version,
 }
 
-/// This macro creates a wrapper for an existing script.
-#[macro_export]
-macro_rules! script_wrapper {
-    ( $mod:ident = $doc:literal => $script:literal ) => {
-        pastey::paste! {
-            mod $mod {
-                #[doc = $doc]
-                #[derive(clap::Args, Debug)]
-                #[command()]
-                pub(super) struct Cli {
-                    args: Vec<String>,
-                }
+#[cfg(test)]
+mod tests {
+    use clap::{CommandFactory as _, error::ErrorKind};
 
-                impl Cli {
-                    pub(super) fn exec(self) -> anyhow::Result<()> {
-                        $crate::app::exec(concat!("scripts/", $script), self.args, true)
-                    }
-                }
+    use super::Cli;
+
+    const SCRIPT_COMMANDS: &[(&str, &str)] = &[
+        ("check", "docs"),
+        ("package", "archive"),
+        ("package", "deb"),
+        ("package", "msi"),
+        ("package", "rpm"),
+        ("release", "docker"),
+        ("release", "s3"),
+    ];
+
+    #[test]
+    fn script_commands_forward_arguments() {
+        let cases: &[(&[&str], &[&str])] = &[
+            (&[], &[]),
+            (&["0.58.0"], &["0.58.0"]),
+            (
+                &["--chart-version", "0.46.0"],
+                &["--chart-version", "0.46.0"],
+            ),
+            (&["--chart-version=0.46.0"], &["--chart-version=0.46.0"]),
+            (
+                &["-x", "path with spaces", "-1"],
+                &["-x", "path with spaces", "-1"],
+            ),
+            (&["value", "--help", "-v"], &["value", "--help", "-v"]),
+            (&["--", "--help"], &["--help"]),
+            (
+                &["--", "--chart-version", "0.46.0"],
+                &["--chart-version", "0.46.0"],
+            ),
+        ];
+
+        for &(group, command) in SCRIPT_COMMANDS {
+            for &(args, expected) in cases {
+                let matches = Cli::command()
+                    .try_get_matches_from(
+                        ["vdev", group, command]
+                            .into_iter()
+                            .chain(args.iter().copied()),
+                    )
+                    .unwrap_or_else(|error| panic!("{group} {command} {args:?}: {error}"));
+                let script = matches
+                    .subcommand_matches(group)
+                    .unwrap()
+                    .subcommand_matches(command)
+                    .unwrap();
+                let forwarded: Vec<_> = script
+                    .get_many::<String>("args")
+                    .into_iter()
+                    .flatten()
+                    .map(String::as_str)
+                    .collect();
+                assert_eq!(forwarded, expected, "{group} {command} {args:?}");
             }
         }
-    };
+    }
+
+    #[test]
+    fn script_commands_keep_vdev_help() {
+        for &(group, command) in SCRIPT_COMMANDS {
+            let error = Cli::command()
+                .try_get_matches_from(["vdev", group, command, "--help"])
+                .unwrap_err();
+            assert_eq!(error.kind(), ErrorKind::DisplayHelp, "{group} {command}");
+        }
+    }
 }

vdev/src/commands/package.rs

@@ -1,21 +1,31 @@
-crate::cli_subcommands! {
-    "Package Vector in various formats..."
-        archive, deb, msi, rpm,
-}
+use crate::utils::command::ScriptArgs;
 
-crate::script_wrapper! {
-    archive = "Create a .tar.gz package for the specified $TARGET"
-        => "package-archive.sh"
-}
-crate::script_wrapper! {
-    deb = "Create a .deb package to be distributed in the APT package manager"
-        => "package-deb.sh"
+/// Package Vector in various formats...
+#[derive(clap::Args, Debug)]
+pub(super) struct Cli {
+    #[command(subcommand)]
+    command: Commands,
 }
-crate::script_wrapper! {
-    msi = "Create a .msi package for Windows"
-        => "package-msi.sh"
+
+#[derive(clap::Subcommand, Debug)]
+enum Commands {
+    /// Create a .tar.gz package for the specified $TARGET
+    Archive(ScriptArgs),
+    /// Create a .deb package to be distributed in the APT package manager
+    Deb(ScriptArgs),
+    /// Create a .msi package for Windows
+    Msi(ScriptArgs),
+    /// Create a .rpm package to be distributed in the YUM package manager
+    Rpm(ScriptArgs),
 }
-crate::script_wrapper! {
-    rpm = "Create a .rpm package to be distributed in the YUM package manager"
-        => "package-rpm.sh"
+
+impl Cli {
+    pub fn exec(self) -> anyhow::Result<()> {
+        match self.command {
+            Commands::Archive(args) => args.exec("package-archive.sh"),
+            Commands::Deb(args) => args.exec("package-deb.sh"),
+            Commands::Msi(args) => args.exec("package-msi.sh"),
+            Commands::Rpm(args) => args.exec("package-rpm.sh"),
+        }
+    }
 }

vdev/src/commands/release/mod.rs

@@ -7,6 +7,8 @@ mod workflow;
 use anyhow::{Result, ensure};
 use semver::Version;
 
+use crate::utils::command::ScriptArgs;
+
 fn ensure_stable(version: &Version, label: &str) -> Result<()> {
     ensure!(
         version.pre.is_empty() && version.build.is_empty(),
@@ -23,22 +25,36 @@ fn preparation_branch(version: &Version) -> String {
     )
 }
 
-crate::cli_subcommands! {
-    "Manage the release process..."
-    channel,
-    docker,
-    generate_cue,
-    github,
-    prepare,
-    workflow,
-    s3,
+/// Manage the release process...
+#[derive(clap::Args, Debug)]
+pub(super) struct Cli {
+    #[command(subcommand)]
+    command: Commands,
 }
 
-crate::script_wrapper! {
-    docker = "Build the Vector docker images and optionally push it to the registry"
-        => "build-docker.sh"
+#[derive(clap::Subcommand, Debug)]
+enum Commands {
+    Channel(channel::Cli),
+    /// Build the Vector docker images and optionally push it to the registry
+    Docker(ScriptArgs),
+    GenerateCue(generate_cue::Cli),
+    Github(github::Cli),
+    Prepare(prepare::Cli),
+    Workflow(workflow::Cli),
+    /// Uploads archives and packages to AWS S3
+    S3(ScriptArgs),
 }
-crate::script_wrapper! {
-    s3 = "Uploads archives and packages to AWS S3"
-        => "release-s3.sh"
+
+impl Cli {
+    pub fn exec(self) -> Result<()> {
+        match self.command {
+            Commands::Channel(cli) => cli.exec(),
+            Commands::Docker(args) => args.exec("build-docker.sh"),
+            Commands::GenerateCue(cli) => cli.exec(),
+            Commands::Github(cli) => cli.exec(),
+            Commands::Prepare(cli) => cli.exec(),
+            Commands::Workflow(cli) => cli.exec(),
+            Commands::S3(args) => args.exec("release-s3.sh"),
+        }
+    }
 }

vdev/src/utils/command.rs

@@ -5,6 +5,26 @@ use std::{
     process::{self, Command},
 };
 
+use crate::app::CommandExt as _;
+
+/// Arguments forwarded to a repository script.
+#[derive(clap::Args, Debug)]
+pub struct ScriptArgs {
+    /// Arguments passed to the script (use `-- --help` for the script's help).
+    #[arg(allow_hyphen_values = true, trailing_var_arg = true)]
+    args: Vec<String>,
+}
+
+impl ScriptArgs {
+    /// Run a script from the repository's scripts directory with the forwarded arguments.
+    pub fn exec(self, script: &str) -> anyhow::Result<()> {
+        Command::script(script)
+            .args(self.args)
+            .in_repo()
+            .check_run()
+    }
+}
+
 /// Trait for chaining command arguments
 pub trait ChainArgs {
     fn chain_args<I: Into<OsString>>(&self, args: impl IntoIterator<Item = I>) -> Vec<OsString>;