Skip to content

Try caching ghcup in CI #2104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 22 additions & 22 deletions .github/workflows/applications.yml
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ jobs:
MATRIX="$(jq -c '.' <<EOF
{
# The first version in this list is the version to be used for release
"ghc": ["9.8.2", "9.10.1", "9.6.6"],
"ghc": ["9.10.1", "9.6.6"],
"cabal": ["3.12"],
"os": ["ubuntu-22.04"],
"use-freeze-file": ["false"],
Expand All @@ -153,7 +153,7 @@ jobs:
else
MATRIX="$(jq -c '.' <<EOF
{
"ghc": ["9.8.2", "9.10.1", "9.6.6"],
"ghc": ["9.10.1", "9.6.6"],
"cabal": ["3.12"],
"os": ["ubuntu-22.04"],
"use-freeze-file": ["false"],
Expand Down Expand Up @@ -250,6 +250,18 @@ jobs:
- name: Set permissions for .ghcup
if: startsWith(matrix.os, 'ubuntu-')
run: sudo chown -R $USER /usr/local/.ghcup
- uses: actions/cache@v4
name: cache
with:
path: |
~/.ghcup
~/.cabal/packages
~/.cabal/store
dist-newstyle
key: ${{ env.key }}-${{ matrix.os }}-${{ matrix.ghc }}-${{ hashFiles('cabal.*', '*.cabal', 'src/**', 'test/**') }}
restore-keys: |
${{ env.key }}-${{ matrix.os }}-${{ matrix.ghc }}-

- name: Install GHC and Cabal
id: setup
uses: haskell-actions/setup@v2
Expand All @@ -261,8 +273,11 @@ jobs:
echo "setup ghc-version: ${{ steps.setup.outputs.ghc-version }}"
echo "setup cabal-version: ${{ steps.setup.outputs.cabal-version }}"
echo "setup cabal-store: ${{ steps.setup.outputs.cabal-store }}"
which ghc
ghc --version
which cabal
cabal --version

- name: Install non-Haskell dependencies (ubuntu)
if: contains(matrix.os, 'ubuntu')
run: sudo apt-get install -y libgflags-dev liblz4-dev libzstd-dev libsnappy-dev libbz2-dev libmpfr-dev
Expand Down Expand Up @@ -314,19 +329,6 @@ jobs:
ghc-options: -debug
EOF

# dist cache
# the cache-key forces uploading of cache at least once a day, which ensures that
# upstream dependency changes are captured regularly.
- uses: actions/cache@v4
name: Cache dist-newstyle
with:
path: |
dist-newstyle
key: ${{ matrix.os }}-${{ matrix.ghc }}-${{ matrix.use-freeze-file }}-2-${{ hashFiles('cabal.*', '*.cabal', 'src/**', 'test/**') }}
restore-keys: |
${{ matrix.os }}-${{ matrix.ghc }}-${{ matrix.use-freeze-file }}-2-
${{ matrix.os }}-${{ matrix.ghc }}-

# Build
- name: Delete Freeze file if it exists
if: matrix.use-freeze-file != 'true'
Expand Down Expand Up @@ -354,13 +356,11 @@ jobs:
store_path: ${{ steps.setup.outputs.cabal-store }}
aws_access_key_id: "${{ secrets.kadena_cabal_cache_aws_access_key_id }}"
aws_secret_access_key: "${{ secrets.kadena_cabal_cache_aws_secret_access_key }}"
- name: Install build dependencies
run: cabal build chainweb --only-dependencies
- name: Build chainweb library
run: cabal build --ghc-options=-j2 lib:chainweb
- name: Build chainweb
run: cabal build -j --semaphore lib:chainweb
- name: Build chainweb applications
run: |
cabal build -j2 --ghc-options=-j2 \
cabal build -j --semaphore \
chainweb:bench:bench \
exe:b64 \
exe:calculate-release \
Expand All @@ -379,7 +379,7 @@ jobs:
test:remote-tests \
test:chainweb-storage-tests
- name: Build chainweb-node application
run: cabal build -j2 --ghc-options=-j2 chainweb-node:exe:chainweb-node
run: cabal build -j --semaphore chainweb-node:exe:chainweb-node

# Checks
- name: Check that working directory tree is clean
Expand Down Expand Up @@ -525,7 +525,7 @@ jobs:
rocksdb-library-tests:
name: RocksDB library tests
needs: [config, build]
runs-on: $${ matrix.os }}
runs-on: ${ matrix.os }}
env:
ARTIFACTS_NAME: chainweb-applications.${{ matrix.use-freeze-file }}.${{ matrix.ghc }}.${{ matrix.os }}
strategy:
Expand Down
15 changes: 6 additions & 9 deletions .github/workflows/macos.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
key: ${{ runner.os }}-${{ runner.arch }}-ghc-cabal
with:
path: |
/Users/runner/.ghcup
/home/runner/.ghcup
key: ${{ env.key }}-${{ hashFiles('bin/cabal', 'bin/ghc') }}
restore-keys: ${{ env.key }}-

Expand Down Expand Up @@ -71,7 +71,7 @@ jobs:
if: steps.ghc-cabal-cache.outputs.cache-hit != 'true'
with:
path: |
/Users/runner/.ghcup
/home/runner/.ghcup
key: ${{ steps.ghc-cabal-cache.outputs.cache-primary-key }}

- name: Install non-Haskell dependencies (macOS)
Expand Down Expand Up @@ -111,13 +111,11 @@ jobs:
store_path: ${{ steps.setup.outputs.cabal-store }}
aws_access_key_id: "${{ secrets.kadena_cabal_cache_aws_access_key_id }}"
aws_secret_access_key: "${{ secrets.kadena_cabal_cache_aws_secret_access_key }}"
- name: Install build dependencies
run: cabal build chainweb --only-dependencies
- name: Build chainweb library
run: cabal build lib:chainweb
- name: Build chainweb
run: cabal build -j --semaphore chainweb
- name: Build chainweb applications
run: |
cabal build -j \
cabal build -j --semaphore \
chainweb:bench:bench \
exe:b64 \
exe:calculate-release \
Expand All @@ -135,7 +133,7 @@ jobs:
test:multi-node-network-tests \
test:remote-tests
- name: Build chainweb-node application
run: cabal build -j chainweb-node:exe:chainweb-node
run: cabal build -j --semaphore chainweb-node:exe:chainweb-node
- uses: actions/cache/save@v4
name: Save dist-newstyle cache
if: steps.cabal-dist-cache.outputs.cache-hit != 'true'
Expand Down Expand Up @@ -171,4 +169,3 @@ jobs:
exit 1
fi
mv cabal.project.freeze.backup cabal.project.freeze

Loading