Skip to content

Commit 90d3644

Browse files
jst-cyrjst-cyr
committed
TEMPORARY: inline full pipeline in puppet8 caller to isolate sidebar bug
Experiment per user request: does dropping the workflow_call boundary to _compatibility-runner-reusable.yml restore correct job names in the Actions run page's left sidebar? If yes, confirms the reusable-workflow split itself (not naming, not depth) is the cause, and Step C's architecture needs to change to per-major duplicated job definitions instead of one shared reusable workflow. _compatibility-runner-reusable.yml is left in place, unused for now.
1 parent 59f4c47 commit 90d3644

1 file changed

Lines changed: 356 additions & 27 deletions

File tree

‎.github/workflows/compatibility-runner-puppet8.yml‎

Lines changed: 356 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,20 @@
11
name: Puppet Module Compatibility Runner (Puppet 8)
22

3-
# Thin trigger workflow for the Puppet 8 major. All pipeline logic lives in
4-
# _compatibility-runner-reusable.yml (see docs/puppet-core-9-dual-major-support.md
5-
# §4) — this file exists only to get its own scheduled/manual triggers,
6-
# Actions-tab history, status badge, and concurrency group, independent of the
7-
# Puppet 9 caller.
3+
# TEMPORARY EXPERIMENT (see docs/puppet-core-9-dual-major-support.md §12,
4+
# Step C): this file normally calls _compatibility-runner-reusable.yml via
5+
# workflow_call. That produced a real regression - the GitHub Actions run
6+
# page's left sidebar collapsed every unit-test job's displayed name down to
7+
# just "unit", even though the job's actual name (confirmed via the Jobs and
8+
# Checks APIs, and the per-job detail header) was correct and unique
9+
# ("test / <module-id> / unit"). A byte-for-byte diff against the old
10+
# compatibility-runner.yml showed the job/matrix/name definitions are
11+
# otherwise identical - the only structural difference is the workflow_call
12+
# boundary itself.
813
#
9-
# NOTE: the `schedule:` trigger is intentionally NOT enabled yet. The old
10-
# compatibility-runner.yml still owns the nightly cron until this caller's
11-
# gate dispatch passes; enabling both on the same cron would double-fire and
12-
# race on the ledger (different concurrency group names, so they would not
13-
# serialize against each other). The cron entry moves here in the same
14-
# commit that retires compatibility-runner.yml.
14+
# This version inlines the full pipeline directly (no workflow_call) to test
15+
# that isolation: if the sidebar renders correctly here, the workflow_call
16+
# boundary is confirmed as the cause, and the reusable-workflow architecture
17+
# needs to be dropped in favor of per-major duplicated job definitions.
1518

1619
on:
1720
workflow_dispatch:
@@ -54,22 +57,348 @@ concurrency:
5457
group: compat-8-${{ github.ref }}
5558
cancel-in-progress: false
5659

60+
env:
61+
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
62+
5763
jobs:
58-
# Named "test" (not "run") because GitHub always prepends the calling job's
59-
# own name to every job defined in the called reusable workflow — "test" is
60-
# what restores the pre-Step-C job-name strings ("test / <module> / unit")
61-
# instead of adding a redundant extra nesting level (see the reusable
62-
# workflow's job-name comments).
63-
test:
64-
uses: ./.github/workflows/_compatibility-runner-reusable.yml
64+
prepare:
65+
runs-on: ubuntu-latest
66+
outputs:
67+
unit_matrix: ${{ steps.matrix.outputs.unit_matrix }}
68+
acceptance_matrix: ${{ steps.matrix.outputs.acceptance_matrix }}
69+
has_unit: ${{ steps.matrix.outputs.has_unit }}
70+
has_acceptance: ${{ steps.matrix.outputs.has_acceptance }}
71+
steps:
72+
- name: Checkout
73+
uses: actions/checkout@v6
74+
with:
75+
fetch-depth: 0
76+
77+
- name: Set up Python
78+
uses: actions/setup-python@v6
79+
with:
80+
python-version: '3.11'
81+
82+
- name: Validate modules config schema
83+
shell: bash
84+
run: |
85+
set -euo pipefail
86+
python -m pip install --disable-pip-version-check jsonschema
87+
python scripts/validate_modules_config.py --config config/modules.json --schema config/modules.schema.json
88+
89+
- name: Detect changes (lean matrix)
90+
id: detect
91+
if: github.event.inputs.modules_json == ''
92+
shell: bash
93+
run: |
94+
set -euo pipefail
95+
mkdir -p .tmp
96+
python scripts/detect_changes.py
97+
env:
98+
MODULES_FILE: config/modules.json
99+
LEDGER_FILE: status/ledger.json
100+
OUTPUT_FILE: .tmp/change-decisions.json
101+
WINDOW_HOURS: ${{ vars.PUPPET_CHANGE_WINDOW_HOURS || '48' }}
102+
STALE_DAYS: ${{ vars.PUPPET_STALE_DAYS || '30' }}
103+
EVENT_NAME: ${{ github.event_name }}
104+
LEAN: ${{ github.event.inputs.lean || 'true' }}
105+
GITHUB_TOKEN: ${{ github.token }}
106+
CALLER_WORKFLOW_FILE: .github/workflows/compatibility-runner-puppet8.yml
107+
108+
- name: Build module matrix
109+
id: matrix
110+
shell: bash
111+
run: |
112+
set -euo pipefail
113+
OVERRIDE='${{ github.event.inputs.modules_json }}'
114+
RUN_ALL='${{ steps.detect.outputs.run_all }}'
115+
INCLUDE_IDS='${{ steps.detect.outputs.include_ids }}'
116+
# No detection ran (modules_json override) -> include everything.
117+
if [ -z "$RUN_ALL" ]; then RUN_ALL='true'; fi
118+
RUN_ALL="$RUN_ALL" INCLUDE_IDS="$INCLUDE_IDS" ruby scripts/build_matrix.rb "$OVERRIDE" > .tmp-matrix.json
119+
120+
UNIT_MATRIX=$(python -c "import json; print(json.dumps(json.load(open('.tmp-matrix.json', encoding='utf-8'))['unit_matrix']))")
121+
ACCEPTANCE_MATRIX=$(python -c "import json; print(json.dumps(json.load(open('.tmp-matrix.json', encoding='utf-8'))['acceptance_matrix']))")
122+
HAS_UNIT=$(python -c "import json; print(json.load(open('.tmp-matrix.json', encoding='utf-8'))['has_unit'])")
123+
HAS_ACCEPTANCE=$(python -c "import json; print(json.load(open('.tmp-matrix.json', encoding='utf-8'))['has_acceptance'])")
124+
echo "unit_matrix=$UNIT_MATRIX" >> "$GITHUB_OUTPUT"
125+
echo "acceptance_matrix=$ACCEPTANCE_MATRIX" >> "$GITHUB_OUTPUT"
126+
echo "has_unit=$HAS_UNIT" >> "$GITHUB_OUTPUT"
127+
echo "has_acceptance=$HAS_ACCEPTANCE" >> "$GITHUB_OUTPUT"
128+
129+
if [ "$HAS_UNIT" != 'true' ] && [ "$HAS_ACCEPTANCE" != 'true' ]; then
130+
echo "::notice title=Run result::no-op: no module required testing this run"
131+
echo "No modules selected for testing — test jobs will be skipped (no-op run)."
132+
fi
133+
134+
- name: Upload change decisions
135+
if: always()
136+
uses: actions/upload-artifact@v7
137+
with:
138+
name: change-decisions
139+
path: .tmp/change-decisions.json
140+
if-no-files-found: ignore
141+
142+
test_unit:
143+
name: test / ${{ matrix.module.id }} / unit
144+
runs-on: ${{ matrix.module.os }}
145+
timeout-minutes: ${{ fromJSON(vars.PUPPET_JOB_TIMEOUT_MINUTES || '360') }}
146+
needs: prepare
147+
# GitHub Actions errors on an empty matrix vector instead of producing zero
148+
# combinations, so gate the job rather than relying on an empty unit_matrix.
149+
if: needs.prepare.outputs.has_unit == 'true'
150+
strategy:
151+
fail-fast: false
152+
matrix:
153+
module: ${{ fromJson(needs.prepare.outputs.unit_matrix) }}
154+
155+
env:
156+
PUPPET_CORE_API_KEY: ${{ secrets.PUPPET_CORE_API_KEY }}
157+
PUPPET_CORE_SOURCE_URL: https://rubygems-puppetcore.puppet.com
158+
PUPPET_COMPAT_METADATA_MODE: ${{ github.event.inputs.metadata_mode || 'warn' }}
159+
PUPPET_COMPAT_BUNDLE_PATH: .b
160+
PUPPET_ENFORCE_PRIVATE_SOURCE: "true"
161+
PUPPET_ENFORCE_NO_OPENVOX: "false"
162+
PUPPET_ENFORCE_EXACT_PUPPET_VERSION: "true"
163+
PUPPET_SPLIT_SOURCES: "true"
164+
PUPPET_STAGE_TIMEOUT_SECONDS: ${{ vars.PUPPET_STAGE_TIMEOUT_SECONDS || '1800' }}
165+
PUPPET_ACCEPTANCE_DEBUG: ${{ github.event.inputs.enable_debug || 'false' }}
166+
167+
steps:
168+
- name: Checkout
169+
uses: actions/checkout@v6
170+
171+
- name: Run module compatibility test
172+
uses: ./.github/actions/run-module-test
173+
with:
174+
module-json: ${{ toJson(matrix.module) }}
175+
module-id: ${{ matrix.module.id }}
176+
profile: ${{ github.event.inputs.profile || '8-latest-maintained' }}
177+
metadata-mode: ${{ github.event.inputs.metadata_mode || 'warn' }}
178+
enable-debug: ${{ github.event.inputs.enable_debug || 'false' }}
179+
test-mode: unit
180+
output-dir: o/${{ matrix.module.id }}
181+
artifact-name: compatibility-${{ matrix.module.id }}-unit
182+
test-lane: unit
183+
acceptance-target: unit
184+
prereqs-json: ${{ toJson(matrix.module.prereqs) }}
185+
186+
- name: Upload module artifacts (unit)
187+
if: always()
188+
uses: actions/upload-artifact@v7
189+
with:
190+
name: compatibility-${{ matrix.module.id }}-unit
191+
path: o/${{ matrix.module.id }}
192+
if-no-files-found: warn
193+
194+
test_acceptance:
195+
name: test / ${{ matrix.module.id }} / acceptance / ${{ matrix.module.target_id }}
196+
runs-on: ${{ matrix.module.os }}
197+
timeout-minutes: ${{ fromJSON(vars.PUPPET_JOB_TIMEOUT_MINUTES || '360') }}
198+
needs: prepare
199+
if: needs.prepare.outputs.has_acceptance == 'true'
200+
strategy:
201+
fail-fast: false
202+
matrix:
203+
module: ${{ fromJson(needs.prepare.outputs.acceptance_matrix) }}
204+
205+
env:
206+
PUPPET_CORE_API_KEY: ${{ secrets.PUPPET_CORE_API_KEY }}
207+
PUPPET_CORE_SOURCE_URL: https://rubygems-puppetcore.puppet.com
208+
PUPPET_COMPAT_METADATA_MODE: ${{ github.event.inputs.metadata_mode || 'warn' }}
209+
PUPPET_COMPAT_BUNDLE_PATH: .b
210+
PUPPET_ENFORCE_PRIVATE_SOURCE: "true"
211+
PUPPET_ENFORCE_NO_OPENVOX: "false"
212+
PUPPET_ENFORCE_EXACT_PUPPET_VERSION: "true"
213+
PUPPET_SPLIT_SOURCES: "true"
214+
PUPPET_STAGE_TIMEOUT_SECONDS: ${{ vars.PUPPET_STAGE_TIMEOUT_SECONDS || '1800' }}
215+
PUPPET_ACCEPTANCE_DEBUG: ${{ github.event.inputs.enable_debug || 'false' }}
216+
217+
steps:
218+
- name: Checkout (for Beaker host seeding)
219+
uses: actions/checkout@v6
220+
221+
- name: Seed Beaker host mappings
222+
if: runner.os == 'Linux'
223+
shell: bash
224+
run: |
225+
set -euo pipefail
226+
mkdir -p .tmp
227+
setfile="config/beaker/setfiles/${{ matrix.module.setfile }}.yml"
228+
echo "Reading host entries from $setfile"
229+
230+
ruby -ryaml -e "
231+
data = YAML.safe_load(File.read(ARGV[0]), permitted_classes: [Symbol])
232+
hosts = data.fetch('HOSTS', {})
233+
hosts.each do |name, cfg|
234+
next unless cfg.is_a?(Hash)
235+
ip = cfg['ip'].to_s.strip
236+
next if ip.empty?
237+
puts \"#{ip} #{name}\"
238+
end
239+
" "$setfile" > .tmp/beaker-host-mappings.txt
240+
241+
if [ ! -s .tmp/beaker-host-mappings.txt ]; then
242+
echo "No explicit host mappings found in setfile"
243+
exit 0
244+
fi
245+
246+
while read -r ip host; do
247+
[ -n "$ip" ] || continue
248+
[ -n "$host" ] || continue
249+
250+
if getent hosts "$host" > /dev/null; then
251+
echo "Host already resolvable: $host"
252+
else
253+
echo "Adding host mapping: $ip $host"
254+
echo "$ip $host" | sudo tee -a /etc/hosts > /dev/null
255+
fi
256+
done < .tmp/beaker-host-mappings.txt
257+
258+
echo "---- Relevant /etc/hosts entries ----"
259+
while read -r _ host; do
260+
[ -n "$host" ] || continue
261+
getent hosts "$host" || true
262+
done < .tmp/beaker-host-mappings.txt
263+
264+
- name: Run module compatibility test (acceptance)
265+
uses: ./.github/actions/run-module-test
266+
with:
267+
module-json: ${{ toJson(matrix.module) }}
268+
module-id: ${{ matrix.module.id }}
269+
profile: ${{ github.event.inputs.profile || '8-latest-maintained' }}
270+
metadata-mode: ${{ github.event.inputs.metadata_mode || 'warn' }}
271+
enable-debug: ${{ github.event.inputs.enable_debug || 'false' }}
272+
test-mode: acceptance
273+
allow-acceptance: 'true'
274+
beaker-setfile: config/beaker/setfiles/${{ matrix.module.setfile }}.yml
275+
output-dir: o/${{ matrix.module.id }}-${{ matrix.module.target_id }}
276+
artifact-name: compatibility-${{ matrix.module.id }}-acceptance-${{ matrix.module.target_id }}
277+
test-lane: acceptance
278+
acceptance-target: ${{ matrix.module.target }}
279+
prereqs-json: ${{ toJson(matrix.module.prereqs) }}
280+
docker-mode: ${{ matrix.module.docker_mode || 'sshd' }}
281+
install-puppetserver: ${{ matrix.module.install_puppetserver || 'false' }}
282+
setup-commands: ${{ toJson(matrix.module.setup_commands) }}
283+
pre-acceptance-commands: ${{ toJson(matrix.module.pre_acceptance_commands) }}
284+
285+
- name: Upload module artifacts (acceptance)
286+
if: always()
287+
uses: actions/upload-artifact@v7
288+
with:
289+
name: compatibility-${{ matrix.module.id }}-acceptance-${{ matrix.module.target_id }}
290+
path: o/${{ matrix.module.id }}-${{ matrix.module.target_id }}
291+
if-no-files-found: warn
292+
293+
- name: Upload acceptance fallback diagnostics
294+
if: ${{ always() && (failure() || cancelled()) }}
295+
uses: actions/upload-artifact@v7
296+
with:
297+
name: compatibility-${{ matrix.module.id }}-acceptance-${{ matrix.module.target_id }}-diagnostics
298+
path: |
299+
o/${{ matrix.module.id }}-${{ matrix.module.target_id }}
300+
.tmp/beaker-host-mappings.txt
301+
if-no-files-found: warn
302+
303+
publish:
304+
runs-on: ubuntu-latest
305+
needs:
306+
- prepare
307+
- test_unit
308+
- test_acceptance
309+
# Runs even when both test jobs were skipped (no-op run): the ledger still
310+
# needs reconciling against modules.json / KNOWN_* and the summary still
311+
# needs to say why nothing ran.
312+
if: always() && needs.prepare.result == 'success'
65313
permissions:
66314
contents: write
67-
secrets: inherit
68-
with:
69-
profile: ${{ github.event.inputs.profile || '8-latest-maintained' }}
70-
metadata_mode: ${{ github.event.inputs.metadata_mode || 'warn' }}
71-
enable_debug: ${{ github.event.inputs.enable_debug || 'false' }}
72-
lean: ${{ github.event.inputs.lean || 'true' }}
73-
modules_json: ${{ github.event.inputs.modules_json || '' }}
74-
event_name: ${{ github.event_name }}
75-
caller_workflow_file: .github/workflows/compatibility-runner-puppet8.yml
315+
steps:
316+
- name: Checkout
317+
uses: actions/checkout@v6
318+
with:
319+
ref: ${{ github.ref_name }}
320+
fetch-depth: 0
321+
322+
- name: Set up Python
323+
uses: actions/setup-python@v6
324+
with:
325+
python-version: '3.11'
326+
327+
- name: Download module artifacts
328+
if: needs.prepare.outputs.has_unit == 'true' || needs.prepare.outputs.has_acceptance == 'true'
329+
uses: actions/download-artifact@v8
330+
with:
331+
path: all-artifacts
332+
pattern: compatibility-*
333+
merge-multiple: false
334+
335+
- name: Download change decisions
336+
continue-on-error: true
337+
uses: actions/download-artifact@v8
338+
with:
339+
name: change-decisions
340+
path: .tmp-decisions
341+
342+
- name: Summarize results
343+
shell: bash
344+
run: |
345+
set -euo pipefail
346+
python scripts/summarize_module_statuses.py
347+
env:
348+
STATUS_ROOT: all-artifacts
349+
SKIP_MANIFEST: .tmp-decisions/change-decisions.json
350+
351+
# Persist to the ledger/dashboard ONLY for real runs (schedule, or a
352+
# manual dispatch of the configured fleet). An ad-hoc modules_json
353+
# override is a throwaway experiment and must not write committed state.
354+
- name: Update status ledger
355+
if: github.event.inputs.modules_json == ''
356+
shell: bash
357+
run: |
358+
set -euo pipefail
359+
python scripts/update_ledger.py
360+
env:
361+
STATUS_ROOT: all-artifacts
362+
LEDGER_FILE: status/ledger.json
363+
MODULES_FILE: config/modules.json
364+
KNOWN_INCOMPATIBLE_FILE: KNOWN_INCOMPATIBLE.md
365+
KNOWN_DEPRECATED_FILE: KNOWN_DEPRECATED.md
366+
367+
- name: Render status dashboard
368+
if: github.event.inputs.modules_json == ''
369+
shell: bash
370+
run: |
371+
set -euo pipefail
372+
python scripts/render_status_dashboard.py
373+
env:
374+
LEDGER_FILE: status/ledger.json
375+
MODULES_FILE: config/modules.json
376+
STATUS_FILE: STATUS.md
377+
KNOWN_COMPATIBLE_FILE: KNOWN_COMPATIBLE.md
378+
KNOWN_INCOMPATIBLE_FILE: KNOWN_INCOMPATIBLE.md
379+
STALE_DAYS: ${{ vars.PUPPET_STALE_DAYS || '30' }}
380+
381+
- name: Render acceptance audit
382+
if: github.event.inputs.modules_json == ''
383+
shell: bash
384+
run: |
385+
set -euo pipefail
386+
python scripts/render_acceptance_audit.py
387+
env:
388+
MODULES_FILE: config/modules.json
389+
AUDIT_FILE: docs/available-acceptance-tests.md
390+
391+
- name: Commit status ledger and dashboard
392+
if: github.event.inputs.modules_json == ''
393+
shell: bash
394+
run: |
395+
set -euo pipefail
396+
git config user.name "github-actions[bot]"
397+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
398+
git add status/ledger.json STATUS.md KNOWN_COMPATIBLE.md docs/available-acceptance-tests.md
399+
if git diff --cached --quiet; then
400+
echo "No status changes to commit."
401+
exit 0
402+
fi
403+
git commit -m "chore(status): update compatibility ledger and dashboard [skip ci]"
404+
git push origin "HEAD:${GITHUB_REF_NAME}"

0 commit comments

Comments
 (0)