Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
4fe69cf
Create restore_spm_cache
spencertransier Feb 8, 2023
702c16e
Add separate save_spm_cache action
spencertransier Feb 9, 2023
5fb1859
Try different path to get relative path
spencertransier Feb 9, 2023
bf61231
Revert "Try different path to get relative path"
spencertransier Feb 9, 2023
0697fb3
Remove --quiet from aws commands
spencertransier Feb 10, 2023
ca48379
Test different tar option
spencertransier Feb 10, 2023
1f19980
Restore cache to correct location
spencertransier Feb 10, 2023
dc1ef56
Move the commands over to `install_swiftpm_dependencies`
spencertransier Feb 10, 2023
9c1565a
Fix linter issue
spencertransier Feb 10, 2023
4855a65
Automatically find Package.resolved
spencertransier Feb 10, 2023
ac1697b
Package.resolved variable isn't initialized yet
spencertransier Feb 11, 2023
776efe7
Exclude 'checkouts' and 'artifacts' from SPM cache
spencertransier Feb 11, 2023
e6813ff
Switch back to root project directory after restoring cache
spencertransier Feb 11, 2023
74e046b
Test with explicit workspace path
spencertransier Feb 11, 2023
7237304
Change location of SPM cache
spencertransier Feb 18, 2023
cdd1414
Change where to save cache from
spencertransier Feb 18, 2023
38c1a99
Install the SPM packages in the correct location
spencertransier Feb 18, 2023
543042e
Use `swift` or `xcodebuild` depending on the project type
spencertransier Feb 23, 2023
8bfe4f7
Use Apple Archive for archiving and extracting the cache
spencertransier Mar 1, 2023
68b361b
Update install_swiftpm_dependencies
spencertransier Mar 1, 2023
b3fe073
Add lines to verify archive/extract method
spencertransier Mar 1, 2023
d988855
Re-add --quiet to AWS upload
spencertransier Mar 1, 2023
1423e71
Add custom path
spencertransier Mar 1, 2023
9432ed6
Collapse aa archive into one line
spencertransier Mar 1, 2023
71f18ab
Use uname to determine whether to use Apple Archive
spencertransier Mar 1, 2023
f71842b
Continue iterating on Apple Archive support
spencertransier Mar 1, 2023
f3ba705
Remove Apple Archive option
spencertransier Mar 1, 2023
f08a599
Use $HOME instead of ~
spencertransier Mar 1, 2023
1b518d5
Use the correct TAR_OPTIONS
spencertransier Mar 1, 2023
bdaf289
Cleaned up debugging echos and added explanatory comments
spencertransier Mar 1, 2023
48bc2ac
Fix shellcheck warnings
spencertransier Mar 1, 2023
f8942a8
Fix shellcheck issue - SC2128
spencertransier Mar 1, 2023
8a6377f
Simplify saving cache by deleting unneeded cache files beforehand
spencertransier Mar 2, 2023
e8784f5
Fix indenting
spencertransier Mar 2, 2023
c05d5e8
Add curly braces around CACHE_KEY variables
spencertransier Mar 2, 2023
82af9b6
Add quotes around all variable names in `if` statement
spencertransier Mar 2, 2023
44785e4
Use default SPM location
spencertransier Mar 2, 2023
b919255
Update tar custom flag name
spencertransier Mar 2, 2023
9d49421
Exit if unable to find Package.resolved or if BUILD_TYPE is not set
spencertransier Mar 9, 2023
4cec047
Merge branch 'trunk' into add/spm-caching-actions
spencertransier Mar 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

.DS_Store
58 changes: 57 additions & 1 deletion bin/install_swiftpm_dependencies
Original file line number Diff line number Diff line change
@@ -1,9 +1,65 @@
#!/bin/bash -eu

CUSTOM_PACKAGE_RESOLVED_PATH=${1-}
BUILD_TYPE=${2-}
SPM_CACHE_LOCATION="$HOME/Library/Caches/org.swift.swiftpm"

WORKSPACE_PACKAGE_RESOLVED_PATH=(*.xcworkspace/xcshareddata/swiftpm/Package.resolved)
ROOT_PACKAGE_RESOLVED_PATH=Package.resolved

# Find where Package.resolved is located
if [[ -n "$CUSTOM_PACKAGE_RESOLVED_PATH" ]]; then
PACKAGE_RESOLVED_LOCATION="$CUSTOM_PACKAGE_RESOLVED_PATH"
elif [[ -f "${WORKSPACE_PACKAGE_RESOLVED_PATH[0]}" ]]; then
BUILD_TYPE="XCODEBUILD"
PACKAGE_RESOLVED_LOCATION="${WORKSPACE_PACKAGE_RESOLVED_PATH[0]}"
elif [[ -f "$ROOT_PACKAGE_RESOLVED_PATH" ]]; then
BUILD_TYPE="SWIFT"
PACKAGE_RESOLVED_LOCATION="$ROOT_PACKAGE_RESOLVED_PATH"
else
echo "Unable to find Package.resolved"
exit 1
fi

PACKAGE_RESOLVED_HASH=$(hash_file "$PACKAGE_RESOLVED_LOCATION")
CACHE_KEY="${BUILDKITE_PIPELINE_SLUG}-spm-cache-${PACKAGE_RESOLVED_HASH}"

# Restore SPM cache if it's available
mkdir -p "$SPM_CACHE_LOCATION"
cd "$SPM_CACHE_LOCATION"
restore_cache "$CACHE_KEY"
cd -

# This will let Xcode use the system SSH config for downloading packages
sudo defaults write com.apple.dt.Xcode IDEPackageSupportUseBuiltinSCM YES
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not part of this PR. Is this preference same as xcodebuild -scmProvider system?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@crazytonyli I think so but I'm not sure. I will look into this and see about adding in the next update of the SPM caching action.


# Trust all GitHub.com and BitBucket.org keys – this allows checking out dependencies via SSH
add_host_to_ssh_known_hosts bitbucket.org
add_host_to_ssh_known_hosts github.com

xcodebuild -resolvePackageDependencies
# Based on the project type, resolve the packages using the correct build type
if [[ "$BUILD_TYPE" == "XCODEBUILD" ]]; then
echo " Resolving packages with \`xcodebuild\`"
xcodebuild -resolvePackageDependencies
elif [[ "$BUILD_TYPE" == "SWIFT" ]]; then
echo " Resolving packages with \`swift\`"
swift package resolve
else
echo "BUILD_TYPE must be set"
exit 1
fi

# `checkouts` can be removed because the system can quickly generate them
# instead of needing to download them in the cache each time.
#
# `artifacts` should be removed because it causes issues when downloading
# certain packages to have the artifacts already present after extracting
# cache
echo " Cleaning up cache files before saving cache"
rm -rf "$SPM_CACHE_LOCATION/checkouts" "$SPM_CACHE_LOCATION/artifacts"

# If this is the first time we've seen this particular cache key, save it for the future
echo " Saving SPM Cache"
save_cache "$SPM_CACHE_LOCATION" "$CACHE_KEY" false --use_relative_path_in_tar
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker. This sequence of argument feels prone to errors, it's probably worth looking into parsing the arguments and options, using getopts for example. Statement as below would look much nicer 😸

save_cache --cache-key "$CACHE_KEY" --overwrite --use_relative_path_in_tar "$SPM_CACHE_LOCATION"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the suggestion! I am going to land this PR and work on improving this in my next iteration of the SPM caching and save_cache improvements.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #54



12 changes: 11 additions & 1 deletion bin/save_cache
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,17 @@ if ! aws s3api head-object --bucket "$S3_BUCKET_NAME" --key "$CACHE_KEY" > /dev/
echo "No existing cache entry for $CACHE_KEY – storing in cache"

echo " Compressing"
tar -czf "$CACHE_KEY" "$CACHE_FILE"
TAR_CONFIG=${4-}
if [[ "$TAR_CONFIG" == '--use_relative_path_in_tar' ]]; then
# This is used by actions such as `install_swiftpm_dependencies`
# This configuration allows the tar to not include the full system path of the
# directory that's being archived. For example, this will save only the
# "DIRECTORY_BEING_ARCHIVED" in `/User/builder/DIRECTORY_BEING_ARCHIVED`
# instead of also creating `/User/builder` when extracting the archive
tar -czf "$CACHE_KEY" -C "$CACHE_FILE" .
else
tar -czf "$CACHE_KEY" "$CACHE_FILE"
fi

echo " Uploading"
# If the bucket has transfer acceleration enabled, use it!
Expand Down