Add fly:bats rake task to run BATs from local branch via Concourse

Introduces `bundle exec rake fly:bats` (from src/) so developers can run
the full BOSH Acceptance Test suite against their local branch without
waiting for the shared CI pipeline.

How it works
------------
The rake task sets a standalone Concourse pipeline (ci/fly-bats.yml),
then triggers and streams the single `bats` job.  The pipeline:

  1. Checks out the current branch from GitHub (branch is pushed
     automatically before set-pipeline).
  2. Runs make-candidate.yml to build a dev release tarball.
  3. Compiles the tarball via bosh-deployment/ci/tasks/shared/bosh-agent-compile.yml.
  4. Runs ci/tasks/run-bats-pipeline.sh which provisions a GCP environment
     with terraform, deploys the BOSH director, runs BATs, and tears down.

All GCP credentials are resolved from the shared Concourse credential
store (((gcp_json_key)) / ((gcp_project_id))).

Files added
-----------
- ci/fly-bats.yml            — standalone Concourse pipeline (replaces the
                               two separate fly-execute task files)
- ci/tasks/run-bats-pipeline.sh — orchestrates the full BATs lifecycle in GCP

Env-var overrides (all optional)
---------------------------------
  BATS_ENV_NAME     terraform env name (default: bats-local)
  STEMCELL_NAME     override GCP stemcell
  DEPLOY_ARGS       extra ops-files for bosh create-env
  BAT_RSPEC_FLAGS   extra RSpec flags (e.g. "--tag wip")
  CONCOURSE_TARGET  target alias (default: bosh)

Cherry-picked from fly-bats-pipeline branch (PR #2753).

Author: colins

ci/fly-bats.yml

@@ -0,0 +1,116 @@
+---
+# Standalone pipeline for running the BOSH Acceptance Test suite (BATs)
+# against a specific branch.
+#
+# Intended to be driven by `bundle exec rake fly:bats` from src/, which
+# pushes the current branch and sets this pipeline automatically.
+#
+# Manual usage:
+#   fly -t bosh set-pipeline -p bats-local \
+#       -c ci/fly-bats.yml \
+#       --var bosh_repo=https://github.com/cloudfoundry/bosh.git \
+#       --var bosh_branch=my-feature-branch \
+#       --var env_name=bats-local \
+#       --var stemcell_name=bosh-google-kvm-ubuntu-noble \
+#       --var deploy_args="-o bosh-deployment/external-ip-not-recommended.yml" \
+#       --var bat_rspec_flags=""
+#   fly -t bosh unpause-pipeline -p bats-local
+#   fly -t bosh trigger-job -j bats-local/bats -w
+
+resources:
+  - name: bosh
+    type: git
+    source:
+      uri: ((bosh_repo))
+      branch: ((bosh_branch))
+
+  # bosh-ci is the same repo as bosh but filtered to ci/ paths so that
+  # scripts under ci/bats/ are available at bosh-ci/ in the task workspace.
+  - name: bosh-ci
+    type: git
+    source:
+      uri: ((bosh_repo))
+      branch: ((bosh_branch))
+      paths: [ci]
+
+  - name: bosh-cli
+    type: github-release
+    source:
+      owner: cloudfoundry
+      repository: bosh-cli
+      access_token: ((github_public_repo_token))
+
+  - name: stemcell
+    type: bosh-io-stemcell
+    source:
+      name: bosh-google-kvm-ubuntu-noble
+
+  - name: bats
+    type: git
+    source:
+      uri: https://github.com/cloudfoundry/bosh-acceptance-tests.git
+      branch: master
+
+  - name: bosh-deployment
+    type: git
+    source:
+      uri: https://github.com/cloudfoundry/bosh-deployment
+      branch: master
+
+  - name: integration-image
+    type: registry-image
+    source:
+      repository: ghcr.io/cloudfoundry/bosh/integration
+      tag: main
+      username: ((github_read_write_packages.username))
+      password: ((github_read_write_packages.password))
+
+jobs:
+  - name: bats
+    serial: true
+    plan:
+      - do:
+          - in_parallel:
+              - get: bosh
+              - get: bosh-ci
+              - get: bosh-cli
+                params:
+                  globs: [bosh-cli-*-linux-amd64]
+              - get: stemcell
+              - get: bats
+              - get: bosh-deployment
+              - get: integration-image
+
+          - task: make-candidate
+            image: integration-image
+            file: bosh-ci/ci/tasks/make-candidate.yml
+
+          - task: compile-bosh-release
+            file: bosh-deployment/ci/tasks/shared/bosh-agent-compile.yml
+
+          - task: run-bats
+            image: integration-image
+            input_mapping:
+              bosh-release: compiled-release
+            config:
+              platform: linux
+              inputs:
+                - name: bosh
+                - name: bosh-ci
+                - name: bosh-cli
+                - name: bosh-deployment
+                - name: stemcell
+                - name: bats
+                - name: bosh-release
+              caches:
+                - path: cache-dot-bosh-dir
+              params:
+                BAT_INFRASTRUCTURE: gcp
+                GCP_JSON_KEY: ((gcp_json_key))
+                GCP_PROJECT_ID: ((gcp_project_id))
+                STEMCELL_NAME: ((stemcell_name))
+                ENV_NAME: ((env_name))
+                DEPLOY_ARGS: ((deploy_args))
+                BAT_RSPEC_FLAGS: ((bat_rspec_flags))
+              run:
+                path: bosh/ci/tasks/run-bats-pipeline.sh

ci/tasks/run-bats-pipeline.sh

@@ -114,9 +114,9 @@ function teardown {
   local exit_code=$?
   set +e
 
-  if [[ "${exit_code}" -ne 0 ]]; then
-    collect_director_diagnostics
-  fi
+  # Always collect diagnostics – on success this helps correlate logs with
+  # passing runs; on failure it captures the state at the point of failure.
+  collect_director_diagnostics
 
   echo "--- Tearing down BOSH director ---"
   if [[ -f director-state/director-state.json ]]; then

src/tasks/fly.rake

@@ -1,3 +1,5 @@
+require 'shellwords'
+
 namespace :fly do
   desc 'Fly unit specs'
   task :unit do
@@ -9,6 +11,53 @@ namespace :fly do
             COVERAGE: ENV.fetch('COVERAGE', false))
   end
 
+  desc 'Run BATs (BOSH Acceptance Tests) against the current branch via Concourse'
+  #
+  # Sets the ci/fly-bats.yml pipeline on Concourse, then triggers and watches
+  # the bats job.  The current branch is pushed to origin automatically so
+  # Concourse can fetch it.
+  #
+  # GCP credentials are resolved from the Concourse credential store
+  # (((gcp_json_key)) and ((gcp_project_id))).
+  #
+  # Useful env vars:
+  #   BATS_ENV_NAME    – terraform env name, must be unique per concurrent run
+  #                      (default: "bats-local")
+  #   STEMCELL_NAME    – GCP stemcell name override
+  #   DEPLOY_ARGS      – extra ops-files passed to bosh create-env
+  #   BAT_RSPEC_FLAGS  – extra flags appended to the RSpec BATs run
+  task :bats do
+    env_name = ENV.fetch('BATS_ENV_NAME', 'bats-local')
+
+    branch = `git -C .. rev-parse --abbrev-ref HEAD`.strip
+    repo   = `git -C .. remote get-url origin`.strip
+                .sub(/\Agit@github\.com:/, 'https://github.com/')
+                .sub(/\.git\z/, '.git')
+
+    # Push the current branch so Concourse can check it out.
+    sh "git -C .. push origin HEAD"
+
+    # ── Set the pipeline ─────────────────────────────────────────────────────
+    sh [
+      "fly #{concourse_target}",
+      'set-pipeline',
+      '--non-interactive',
+      '--pipeline bats-local',
+      '--config ../ci/fly-bats.yml',
+      "--var bosh_repo=#{Shellwords.escape(repo)}",
+      "--var bosh_branch=#{Shellwords.escape(branch)}",
+      "--var env_name=#{Shellwords.escape(env_name)}",
+      "--var stemcell_name=#{Shellwords.escape(ENV.fetch('STEMCELL_NAME', 'bosh-google-kvm-ubuntu-noble'))}",
+      "--var deploy_args=#{Shellwords.escape(ENV.fetch('DEPLOY_ARGS', '-o bosh-deployment/external-ip-not-recommended.yml'))}",
+      "--var bat_rspec_flags=#{Shellwords.escape(ENV.fetch('BAT_RSPEC_FLAGS', ''))}",
+    ].compact.join(' ')
+
+    sh "fly #{concourse_target} unpause-pipeline --pipeline bats-local"
+
+    # ── Trigger and stream the job output ────────────────────────────────────
+    sh "fly #{concourse_target} trigger-job --job bats-local/bats --watch"
+  end
+
   desc 'Fly integration specs'
   task :integration, [:cli_dir] do |_, args|
     db, db_version = fetch_db_and_version('postgresql')