Support default convention for dynamic pipeline generation

[davidarcher]

Is your feature request related to a problem? Please describe.
I want to generate a pipeline dynamically but don't want to change the default pipeline defined in the Buildkite UI and don't want extra build steps for every build just to generate/upload dynamic pipelines.

Describe the solution you'd like
We show an example in the docs of:

$ ./script/dynamic_step_generator | buildkite-agent pipeline upload

My request is to have a conventional default for that behavior rather than requiring a custom pipeline to invoke it.

Add a default "dynamic pipeline generator script" to the list of pipeline files that the buildkite-agent automatically detects.

Specific name doesn't matter but buildkite.sh / .buildkite/pipeline.sh seems reasonable.

When buildkite-agent pipeline upload is run and a buildkite.sh or .buildkite/pipeline.sh file exists and is executable, execute it and take the output as a pipeline.

Describe alternatives you've considered

• Living with having one CI job step to "upload pipeline" and then another immediately after to "generate and upload dynamic pipeline"
• Editing each project in the Buildkite UI where we use dynamic pipelines to have a different default pipeline. This doesn't feel great for us as we really want the pipeline logic to fully live in the git repo.

Additional context
N/A

[DrJosh9000]

Thanks @davidarcher, I think this is a pretty good idea!

I think it will need an agent config option / flag / env var to disable it - maybe it's similar enough to a "hook" that disabling it with no-local-hooks makes sense.