Skip to content

Commit 40d4ab3

Browse files
thomas11iwahbe
andauthored
CopyFile: support assets, archives, and recursive copying (#423)
This PR allows the [CopyFile](https://www.pulumi.com/registry/packages/command/api-docs/remote/copyfile/) resource to recursively copy directories as well, similar to `scp -r`. This PR enhances the CopyFile resource in a few ways. - It can now copy directories as well, recursively with their contents. - It now takes the standard Pulumi [asserts and archives](https://www.pulumi.com/docs/concepts/assets-archives/) as input, allowing for seamless interop with other resources. - It now checks whether the specified file or directory have changed (in content, not timestamp) and copies only if they did. In light of these changes, I've renamed `CopyFile` to just `CopyToRemote` which is a **breaking change**. Resolves #23 Resolves #33 Resolves #42 ## TODO - [x] ~~The current implementation fails as soon as a file or directory already exists on the remote, like the previous `CopyFile`.~~ - [x] Before GA, should we rename `CopyFile` to `Copy` or `RemoteCopy` or _insert your proposal here_? - [ ] The integration test copies a bunch of EC2 setup code with `TestEc2RemoteTs`, which we should share instead. _What would be a good way to do that, given the code is in the TS tests, not in the Go driver?_ ## Design considerations The behavior of the copy operation was modeled after `cp` and `scp`. ``` source | dest - exists as dir | dest - does not exist | dest - exists as file -------|----------------------|-----------------------|----------------------- dir | dest/dir | dest/dir | error dir/ | dest/x for x in dir | dest/dir | error file | dest/file | dest | dest (overwritten) ``` Specifically: - When copying a directory, we overwrite existing files. (The current CopyFile resource for single files does that, too.) - When copying a directory, we _don't_ clear the remote directory first so that no left-over files from previous copies will be around. We can always add a flag for that if needed. ## Implementation notes - I've looked around for open source Go packages implementing scp of folders, but to my surprise, I couldn't find one with an acceptable license that 1) wasn't ancient and 2) was cross-platform. - The current implementation copies files sequentially and is therefore probably slow for large trees. If we replaced the implementation, I don't think the shape of the resource would change, so this shouldn't be a GA blocker. - The `size.ts` file in both steps of the integration test is useless because it always has the same value. However, I found that without a TS file present in the additional step, it would not be run. Haven't debugged further. --------- Co-authored-by: Ian Wahbe <[email protected]>
1 parent b424533 commit 40d4ab3

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

66 files changed

+692846
-847
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ sdk/dotnet/version.txt
1414
.DS_Store
1515
/provider/cmd/pulumi-resource-command/schema-embed.json
1616
dist/
17+
.vscode
1718

1819
rsa*
1920
password.txt

examples/aws-py-ec2-command/__main__.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -69,10 +69,10 @@ def decode_key(key):
6969
)
7070

7171
# Copy a config file to our server.
72-
cp_config = command.remote.CopyFile(
72+
cp_config = command.remote.Copy(
7373
'config',
7474
connection=conn,
75-
local_path='myapp.conf',
75+
local_asset='myapp.conf',
7676
remote_path='myapp.conf',
7777
opts=pulumi.ResourceOptions(depends_on=[server])
7878
)

examples/dir_copy_ec2/Pulumi.yaml

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
name: copy-remote-ec2
2+
runtime: nodejs
3+
description: A command example copying a directory via sFTP
4+
config:
5+
pulumi:tags:
6+
value:
7+
pulumi:template: ""

0 commit comments

Comments
 (0)