CI: the same tests are run five times

[felixfontein]

While looking a bit closer at CI I noticed that make test is run in all five Build and test ${{ matrix.os }} ${{ matrix.arch }} GHA runs. Since every of these runs runs on ubuntu-latest, the exact same tests are run in all five runs. This seems a bit wasteful, especially since running these tests takes around 2/3 of the total run time for each of these runs.

I would propose to run the unit tests (make test) only once in a second stage, similar to the functional tests - or maybe as an intermediate stage between build and functional tests?

@getsops/maintainers what do you think?

[hiddeco]

Likely a bug I introduced, and probably should look more like: https://github.com/hiddeco/sshsig/blob/main/.github/workflows/test.yaml#L18

[felixfontein]

It's nothing you introduced, it has been like that for a longer time: a115ce4
You only increased the amount of parallel runs from 3 to 5 ;-)

I checked back the travis config,

sops/.travis.yml

Line 28 in 886c1ee

- 'if [ "$TRAVIS_PULL_REQUEST" != "false" ]; then make; fi'

, it simply ran them once and all on the same platform (probably some form of Linux).

[hiddeco]

Looking into this, the better way would be to leverage QEMU in combination with binfmt_misc (1, 2) while running e.g. ARM64 tests with GOARCH set (3).

In combination with the available Ubuntu, MacOS and Windows GitHub runners, this would allow us to run the Go tests for almost every platform (except for Darwin/ARM64). While running integration tests for at least every platform (but not every CPU architecture).