Features.md

Build system features using Bash

Following text provides suggestions for using bash to implement build system features.

Build flows

Now focus on creating building flows. For parallel tasks, you can use the following pattern¹:

#!/usr/bin/env bash
set -eo pipefail # this can be harmful, see https://www.youtube.com/watch?v=4Jo3Ml53kvc
(
	trap 'kill 0' SIGINT ;
	bs/taskA &
	bs/taskB &
	bs/taskC &
	wait
)

For serial tasks²:

#!/usr/bin/env bash
set -eo pipefail # this can be harmful, see https://www.youtube.com/watch?v=4Jo3Ml53kvc
bs/taskA &&
bs/taskB &&
bs/taskC

For serial tasks with limited concurrency³:

max=4
running=0

for t in bs/taskA bs/taskB bs/taskC bs/taskD; do
	"$t" &
	running=$((running+1))
	if [ "$running" -ge "$max" ]; then
		wait -n
		running=$((running-1))
	fi
done
wait

…pipe tasks:

#!/usr/bin/env bash
set -eo pipefail # this can be harmful, see https://www.youtube.com/watch?v=4Jo3Ml53kvc
cat src/*.js | manipulate > index.js

Build dependencies

The simplified version (just “announcing” dependencies) is declaring variables on top of the script file:

# depends on
declare -r rollup='node_modules/.bin/rollup'
declare -r analyze='bs/analyze'
declare -r config='rollup.config.js'
# …

Alternatively, you can use a dedicated file to explicitly define dependencies between tasks, see for example:

bs/
├───.build.deps
├───build.sh
├───test.sh
├───lint.sh
…

…where .build.deps looks like:

bs/test.sh
bs/lint.sh

…and build.sh looks like:

#!/usr/bin/env bash
set -eo pipefail # this can be harmful, see https://www.youtube.com/watch?v=4Jo3Ml53kvc
bash bs/.build.deps
# build logic

Incremental builds

You can use timestamps⁴ or hashes⁵ to determine whether a build is needed:

#!/usr/bin/env bash
set -eo pipefail # this can be harmful, see https://www.youtube.com/watch?v=4Jo3Ml53kvc
src='src/index.js'
out='build/app.js'
if [[ -f "$out" && "$out" -nt "$src" ]]; then
    exit 0
fi
# build logic

…or

#!/usr/bin/env bash
set -eo pipefail # this can be harmful, see https://www.youtube.com/watch?v=4Jo3Ml53kvc
src='src/index.js'
out='build/app.js'
hash_file='bs/.cache/build.hash'
src_hash=$(md5sum 'src/*' | md5sum)
if [[ -f "$hash_file" && $(cat "$hash_file") == "$src_hash" ]]; then
    exit 0
fi
# build logic

Footnotes

1. you can also use templates: bs/templates.sh bash/parallel >> script.sh ↩

2. you can also use templates: bs/templates.sh bash/sequence >> script.sh ↩

3. you can also use templates: bs/templates.sh bash/concurrency >> script.sh ↩

4. You can use function needs_rebuild from template bs/templates.sh bash/needs_rebuild_timestamp > bs/.libs/needs_rebuild.sh ↩

5. You can use function needs_rebuild from template bs/templates.sh bash/needs_rebuild_timestamp > bs/.libs/needs_rebuild.sh ↩