Skip to content

Commit

Permalink
MINOR: Apply extra serialized rsync step to both parallel and serial …
Browse files Browse the repository at this point in the history
…paths

Author: Ewen Cheslack-Postava <[email protected]>

Reviewers: Ismael Juma <[email protected]>, Eno Thereska <[email protected]>

Closes apache#3546 from ewencp/fix-rsync
  • Loading branch information
ewencp committed Jul 20, 2017
1 parent b04bed0 commit 9b7a6ee
Showing 1 changed file with 17 additions and 15 deletions.
32 changes: 17 additions & 15 deletions vagrant/vagrant-up.sh
Original file line number Diff line number Diff line change
Expand Up @@ -226,31 +226,33 @@ function bring_up_aws {

if [[ ! -z "$worker_machines" ]]; then
echo "Bringing up test worker machines in parallel"
# Currently it seems that the AWS provider will always run
# rsync as part of vagrant up. However,
# https://github.com/mitchellh/vagrant/issues/7531 means
# it is not safe to do so. Since the bug doesn't seem to
# cause any direct errors, just missing data on some
# nodes, follow up with serial rsyncing to ensure we're in
# a clean state. Use custom TMPDIR values to ensure we're
# isolated from any other instances of this script that
# are running/ran recently and may cause different
# instances to sync to the wrong nodes
# Try to isolate this job in its own /tmp space. See note
# below about vagrant issue
local vagrant_rsync_temp_dir=$(mktemp -d);
TMPDIR=$vagrant_rsync_temp_dir vagrant_batch_command "vagrant up $debug --provider=aws" "$worker_machines" "$max_parallel"
rm -rf $vagrant_rsync_temp_dir
vagrant hostmanager
for worker in $worker_machines; do
local vagrant_rsync_temp_dir=$(mktemp -d);
TMPDIR=$vagrant_rsync_temp_dir vagrant rsync $worker;
rm -rf $vagrant_rsync_temp_dir
done
fi
else
vagrant up --provider=aws --no-parallel --no-provision $debug
vagrant hostmanager
vagrant provision
fi

# Currently it seems that the AWS provider will always run rsync
# as part of vagrant up. However,
# https://github.com/mitchellh/vagrant/issues/7531 means it is not
# safe to do so. Since the bug doesn't seem to cause any direct
# errors, just missing data on some nodes, follow up with serial
# rsyncing to ensure we're in a clean state. Use custom TMPDIR
# values to ensure we're isolated from any other instances of this
# script that are running/ran recently and may cause different
# instances to sync to the wrong nodes
for worker in $worker_machines; do
local vagrant_rsync_temp_dir=$(mktemp -d);
TMPDIR=$vagrant_rsync_temp_dir vagrant rsync $worker;
rm -rf $vagrant_rsync_temp_dir
done
}

function main {
Expand Down

0 comments on commit 9b7a6ee

Please sign in to comment.