Merge pull request #87 from fgogolli/fix_docs_deployment

Fix docs, workshop and deployment scripts to reflect finos/htc-grid

deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml

@@ -3,7 +3,7 @@ AWSTemplateFormatVersion: '2010-09-09'
 Description: AWS CloudFormation template to create a Cloud9 environment and prepare the HTC-Grid setup
 Metadata:
   Author:
-    Description: Pierre-Louis Gounod <gpilouis@amazon.com>
+    Description: Flamur Gogolli <flamurg@amazon.co.uk>
   License:
     Description: 'Copyright 2024 Amazon.com, Inc. and its affiliates. 
 
@@ -55,7 +55,7 @@ Parameters:
   HTCGridRepo:
     Description: HTC Grid repo to pull on Cloud9
     Type: String
-    Default: https://github.com/awslabs/aws-htc-grid
+    Default: https://github.com/finos/htc-grid
     ConstraintDescription: Must be a valid HTC Grid version.
   ParticipantRoleARN:
     Description: "ARN of the Participant Role"
@@ -536,16 +536,16 @@ Resources:
                   echo -e "========================================================"
                   echo -e "                   Downloading HTC-Grid"
                   echo -e "========================================================"
-                  export DEFAULT_HTCGRID_REPO="https://github.com/awslabs/aws-htc-grid"
+                  export DEFAULT_HTCGRID_REPO="https://github.com/finos/htc-grid"
                   export HTCGRID_HOME="/home/ec2-user/environment"
                   cd "${HTCGRID_HOME}"
-                  if [ -e "aws-htc-grid" ]; then
-                    export OLD_HTCGRID_DIR="aws-htc-grid.$(date "+%F-%T" | sed 's|:|_|g')"
-                    mv aws-htc-grid "${OLD_HTCGRID_DIR}"
+                  if [ -e "htc-grid" ]; then
+                    export OLD_HTCGRID_DIR="htc-grid.$(date "+%F-%T" | sed 's|:|_|g')"
+                    mv htc-grid "${OLD_HTCGRID_DIR}"
                     if [ $? -eq 0 ]; then
-                      echo -e "[WARN] Found existing aws-htc-grid directory. Renaming to:\n${OLD_HTCGRID_DIR}"
+                      echo -e "[WARN] Found existing htc-grid directory. Renaming to:\n${OLD_HTCGRID_DIR}"
                     else
-                      echo -e "[ERROR] Found existing aws-htc-grid directory, however failed to rename it as:\n${OLD_HTCGRID_DIR}"
+                      echo -e "[ERROR] Found existing htc-grid directory, however failed to rename it as:\n${OLD_HTCGRID_DIR}"
                     fi
                   fi
 
@@ -563,26 +563,26 @@ Resources:
                     fi
 
                     curl --silent --location \
-                          -o "${HTCGRID_HOME}/aws-htc-grid.tar.gz" \
+                          -o "${HTCGRID_HOME}/htc-grid.tar.gz" \
                           "${HTCGRID_REPO}/archive/refs/${HTCGRID_URI_REF}.tar.gz"
 
-                    cd "${HTCGRID_HOME}"; tar -zxf aws-htc-grid.tar.gz; rm -fr aws-htc-grid.tar.gz
-                    mv "aws-htc-grid-${HTCGRID_VERSION}" "aws-htc-grid"
+                    cd "${HTCGRID_HOME}"; tar -zxf htc-grid.tar.gz; rm -fr htc-grid.tar.gz
+                    mv "htc-grid-${HTCGRID_VERSION}" "htc-grid"
 
                     if [ $? -eq 0 ]; then
-                      echo -e "[INFO] Successfully downloaded HTC-Grid to [${HTCGRID_HOME}/aws-htc-grid] with version:\n${HTCGRID_VERSION}"
+                      echo -e "[INFO] Successfully downloaded HTC-Grid to [${HTCGRID_HOME}/htc-grid] with version:\n${HTCGRID_VERSION}"
                     else
-                      echo -e "[ERROR] Failed to rename aws-htc-grid-${HTCGRID_VERSION} to aws-htc-grid .."
+                      echo -e "[ERROR] Failed to rename htc-grid-${HTCGRID_VERSION} to htc-grid .."
                     fi
                   else
                     echo -e "[INFO] Cloning HTC-Grid from:\n${HTCGRID_REPO}/tree/${HTCGRID_VERSION}"
-                    mkdir -p "${HTCGRID_HOME}/aws-htc-grid"; cd "${HTCGRID_HOME}/aws-htc-grid"
+                    mkdir -p "${HTCGRID_HOME}/htc-grid"; cd "${HTCGRID_HOME}/htc-grid"
                     git init && git branch -m main || true && \
                     git remote add origin "${HTCGRID_REPO}.git" -f && git reset --hard origin/main && \
                     git checkout "${HTCGRID_VERSION}"
 
                     if [ $? -eq 0 ]; then
-                      echo -e "[INFO] Successfully downloaded HTC-Grid to [${HTCGRID_HOME}/aws-htc-grid] with version:\n${HTCGRID_VERSION}"
+                      echo -e "[INFO] Successfully downloaded HTC-Grid to [${HTCGRID_HOME}/htc-grid] with version:\n${HTCGRID_VERSION}"
                     else
                       echo -e "[ERROR] Failed to clone ${HTCGRID_REPO}.git with version ${HTCGRID_VERSION}. Exiting ...";
                       exit 254

deployment/init_grid/scripts/python/delete_bucket.py

@@ -17,11 +17,11 @@
 
 if versioning.get("Status") == "Enabled":
     bucket.object_versions.delete()
-    print(f"Succesfully deleted all bucket object versions in bucket: {bucket_name}")
+    print("Succesfully deleted all bucket object versions in bucket: {0}".format(bucket_name))
 else:
     bucket.objects.delete()
-    print(f"Succesfully deleted all bucket objects in bucket: {bucket_name}")
+    print("Succesfully deleted all bucket objects in bucket: {0}".format(bucket_name))
 
 
 s3_client.delete_bucket(Bucket=bucket_name)
-print(f"Succesfully deleted bucket: {bucket_name}")
+print("Succesfully deleted bucket: {0}".format(bucket_name))

docs/project/getting_started/happy-path.md

@@ -55,7 +55,7 @@ The HTC-Grid project has external software dependencies that are deployed as con
     make ecr-login REGION=$HTCGRID_REGION
     ```
 
-2. The following command will go to the `~/environment/aws-htc-grid/deployment/image_repository/terraform` and initialize the  project:
+2. The following command will go to the `~/environment/htc-grid/deployment/image_repository/terraform` and initialize the  project:
     ```bash
     make init-images TAG=$TAG REGION=$HTCGRID_REGION
     ```

docs/project/user_guide/creating_your_a_client.md

@@ -47,7 +47,7 @@ We assume that all previous steps have been successfully completed, there is at
     pip3 install -r requirements.txt
     ```
 
-6. Sample client and workload generator is located here [`./examples/client/python/client.py`](https://github.com/awslabs/aws-htc-grid/blob/main/examples/client/python/client.py). Read help and browse through the code to be able to submit tasks and sessions, see examples below:
+6. Sample client and workload generator is located here [`./examples/client/python/client.py`](https://github.com/finos/htc-grid/blob/main/examples/client/python/client.py). Read help and browse through the code to be able to submit tasks and sessions, see examples below:
 
     To show the example client application help
 

docs/workshop/content/10_architecture/10_high_level_arch/_index.en.md

@@ -24,4 +24,3 @@ At the moment HTC-Grid Compute plane does only provide an EKS Implementation. Th
 Inter module communication is implemented using standardized AWS APIs which facilitates independent development and provide further customization options.
 
 Internally, each of the 4 functional components (API, Data & Control Planes, and Compute Resources) are built using exclusively cloud native building blocks such as: serverless functions and fully managed services. These blocks require no human maintenance (zero administration), are highly available by design, and can scale horizontally in response to demand.
-

docs/workshop/content/10_architecture/20_lambda_compute_unit/_index.en.md

@@ -39,4 +39,3 @@ Within the EKS implementation that the project uses by default, the agent provid
 {{% notice info %}}
 At time of writing this, the project is undergoing changes to move from **[lambci](https://github.com/lambci/lambci)** to the **[aws-lambda-runtime-interface-emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator)**. *aws-lambda-runtime-interface-emulator* is an AWS project that simplifies the integration, reduce the code needed and allows us to provide the lambda simulation integrated with other architecture, enabling us in the future to support **[Graviton](https://aws.amazon.com/ec2/graviton/)** instances.
 {{% /notice %}}
-

docs/workshop/content/10_architecture/30_task_lifecycle/_index.en.md

@@ -38,4 +38,4 @@ During a normal usage, client application would either (i) save the returned ses
 
 {{< img "htc-tasks-state-transition-diagram.png" "State Transition" >}}
 
-{{< img "htc-tasks-state-transition-table.png" "State Transition" >}}
+{{< img "htc-tasks-state-transition-table.png" "State Transition" >}}

docs/workshop/content/10_architecture/40_failure_detection_recovery/_index.en.md

@@ -17,4 +17,4 @@ Depending on the job definition, failed tasks can be retried up to a fixed numbe
 When the task is completed, the Engine updates DynamoDB for the last time and sets task status to “completed”. Afterwards, the Engine tries to acquire a next task from an SQS queue.
 
 
-{{< img "simplified-flow-failure.png"  "Simplified-flow-failure" >}}
+{{< img "simplified-flow-failure.png"  "Simplified-flow-failure" >}}

docs/workshop/content/10_architecture/50_data_plane/_index.en.md

@@ -18,4 +18,4 @@ At the time of writing this, the team has been testing the integration of a Data
 {{% /notice %}}
 
 
-The data plane does not have a data retention mechanism – though S3 data lifecycle policies can be applied to reduce the cost. If the size of task payload is small (in the order of a few KB), this data embedded the task definition and the data-plane bypassed. Also, if required, large amounts of common input data can be preloaded into the data-plane prior to start of workload execution.
+The data plane does not have a data retention mechanism – though S3 data lifecycle policies can be applied to reduce the cost. If the size of task payload is small (in the order of a few KB), this data embedded the task definition and the data-plane bypassed. Also, if required, large amounts of common input data can be preloaded into the data-plane prior to start of workload execution.

docs/workshop/content/10_architecture/70_compute_plane/_index.en.md

@@ -19,4 +19,4 @@ As HTC-Agents are treated as a [Kubernetes deployment](https://kubernetes.io/doc
 
 The corresponding scale down procedure is shown in Figure 4. Pod scale down is triggered when metrics imply a target cluster size that is smaller than the current cluster size. In response, the Kubernetes control plane sends a SIGTERM signal  to selected containers/pods. The SIGTERM is intercepted by the agent, so providing the opportunity to finish the current task and exit gracefully. The pod terminates after the agent exits, or once the terminationGracePeriod has expired . Instance scale down then occurs targeting the removal of inactive instances.
 
-{{< img "scale-down-pods.png"  "scale-down-pods" >}}
+{{< img "scale-down-pods.png"  "scale-down-pods" >}}

docs/workshop/content/10_architecture/80_full_architecture_diagram/_index.en.md

@@ -10,4 +10,4 @@ The following depicts a larger diagram of HTC-Grid:
 
 We have attached the blueprint so you can re-use and apply the changes that you decide to apply to your HTC-Grid setup.
 
-{{% attachments title="Drawio Architecture Files" pattern=".*(drawio)" /%}}
+{{% attachments title="Drawio Architecture Files" pattern=".*(drawio)" /%}}

docs/workshop/content/10_architecture/_index.en.md

@@ -18,4 +18,3 @@ HTC-Grid’s design tenets have been moulded by the requirements of early adopte
 1.	**Simplify re-Platforming**: Support client APIs that are familiar to AWS customers.
 1.	**All compute looks like lambda**: Tasks API’s are lambdas irrespective of the backend compute resource (Lambda Service, Container or EC2) being used.
 1.	**Cloud-native**: Fully leverage operationally hardened AWS core services to optimize robustness and performance while minimizing operational management.
-

docs/workshop/content/20_deploy_htc/10_self_paced/_index.en.md

@@ -52,7 +52,7 @@ Just follow the process below in your account to deploy the Cloud9 Cloudformatio
 If you would like to use the current version of the project, change the value of the `HTCGridVersion` field to `main` in step 4 below.
 {{% /notice %}}
 
-1. Download the latest HTC-Grid Cloud9 Cloudformation Stack. **[The stack is available on this link](https://raw.githubusercontent.com/awslabs/aws-htc-grid/main/deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml)**.
+1. Download the latest HTC-Grid Cloud9 Cloudformation Stack. **[The stack is available on this link](https://raw.githubusercontent.com/finos/htc-grid/main/deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml)**.
 
 1. On the AWS Console, select **CloudFormation** and **Create a Stack**. Select the (with new resources, Standard)
 
@@ -70,7 +70,7 @@ If you would like to use the current version of the project, change the value of
 
     {{< img "Cloud9-stack-creation-3.png" "Cloud9 stack creation 3" >}}
 
-1. Finally wait for the CloudFormation Stack to complete. This may take a few minutes (From 10 to 15 minutes) You can use the time to [Read the CloudFormation stack](https://github.com/awslabs/aws-htc-grid/blob/main/deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml) and understand which resources have been deployed.
+1. Finally wait for the CloudFormation Stack to complete. This may take a few minutes (From 10 to 15 minutes) You can use the time to [Read the CloudFormation stack](https://github.com/finos/htc-grid/blob/main/deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml) and understand which resources have been deployed.
 
 
-Once the CloudFormation Stack creation is complete, go to the **[Configure your Workspace]({{< ref "20_deploy_htc/30_configure_your_workspace/_index.en.md" >}})** section
+Once the CloudFormation Stack creation is complete, go to the **[Configure your Workspace]({{< ref "20_deploy_htc/30_configure_your_workspace/_index.en.md" >}})** section

docs/workshop/content/20_deploy_htc/20_aws_event/_index.en.md

@@ -22,5 +22,5 @@ If you are at an AWS event, an AWS account was created for you to use throughout
 You are now logged in to the AWS console in an account that was created for you, and will be available only throughout the workshop run time.
 
 {{% notice info %}}
-In the interest of time, for the event we have pre-deployed a set of resources for you, starting with a Cloud9 environment that you can use to run the workshop. The [Cloud9 Environment is available here](https://github.com/ruecarlo/aws-htc-grid/blob/main/deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml).  In some workshops, this might be extendeed with a few extra command and resulting in skipping a few other sections in the workshop (such us skipping the download and copy of container images). Just follow the instructions at the event. The instructor will point you where to start.
-{{% /notice %}}
+In the interest of time, for the event we have pre-deployed a set of resources for you, starting with a Cloud9 environment that you can use to run the workshop. The [Cloud9 Environment is available here](https://github.com/finos/htc-grid/blob/main/deployment/dev_environment_cloud9/cfn/cloud9-htc-grid.yaml).  In some workshops, this might be extendeed with a few extra command and resulting in skipping a few other sections in the workshop (such us skipping the download and copy of container images). Just follow the instructions at the event. The instructor will point you where to start.
+{{% /notice %}}

docs/workshop/content/20_deploy_htc/30_configure_your_workspace/_index.en.md

@@ -80,5 +80,3 @@ If the _Arn_ contains the role name from above and an Instance ID, you may proce
     "Arn": "arn:aws:sts::123456789012:assumed-role/htc-grid-workshop-C9Role-XXXXXXXXXXXXXXXX/i-01234567890abcdef"
 }
 ```
-
-

docs/workshop/content/20_deploy_htc/40_prerequisites/_index.en.md

@@ -9,18 +9,18 @@ weight: 40
 The current release of HTC requires python3.7, and the documentation assumes the use of virtualenv. Set this up as follows:
 
 ```
-cd aws-htc-grid
+cd htc-grid
 virtualenv venv
 ```
 
 This will create a virtual environment for python execution with dependencies and libraries required. To activate the environment run:
 
 ```
 source ./venv/bin/activate
-echo "source ~/environment/aws-htc-grid/venv/bin/activate" >> ~/.bashrc
+echo "source ~/environment/htc-grid/venv/bin/activate" >> ~/.bashrc
 ```
 
-To validate the version of python we are using is the one from the virtual environment we just created you can run, and it should point to `~/environment/aws-htc-grid/venv/bin/python`
+To validate the version of python we are using is the one from the virtual environment we just created you can run, and it should point to `~/environment/htc-grid/venv/bin/python`
 
 ```
 which python
@@ -37,7 +37,7 @@ export HTCGRID_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity
 
 for var in TAG HTCGRID_REGION  ; do echo "export $var=$(eval "echo \"\$$var\"")" >> load_variables.sh ; done
 echo -e "===\nYour variables and configuration have been setup as follows\n===\n$(cat load_variables.sh)"
-echo "source ~/environment/aws-htc-grid/load_variables.sh" >> ~/.bashrc
+echo "source ~/environment/htc-grid/load_variables.sh" >> ~/.bashrc
 ```
 
 The code above first set the variables and then, saves all of them within the file `load_variables.sh` file. Finally it creates a line within the `bashrc` file so that whenever we create new terminal the variables get pre-loaded.

docs/workshop/content/20_deploy_htc/50_deploying_htc_grid_images/_index.en.md

@@ -16,7 +16,7 @@ HTC-Grid uses a few open source project with container images stored at Dockerhu
     make ecr-login
     ```
 
-1. Go into the terraform image deployment directory. This contains the terraform project that will help to create a copy of the required images to your ECR repository. The following command will go to the `~/environment/aws-htc-grid/deployment/image_repository/terraform` and initialize the terraform project using the bucket `$S3_IMAGE_TFSTATE_HTCGRID_BUCKET_NAME` as the bucket that will hold the terraform state:
+1. Go into the terraform image deployment directory. This contains the terraform project that will help to create a copy of the required images to your ECR repository. The following command will go to the `~/environment/htc-grid/deployment/image_repository/terraform` and initialize the terraform project using the bucket `$S3_IMAGE_TFSTATE_HTCGRID_BUCKET_NAME` as the bucket that will hold the terraform state:
 
     ```
     make init-images TAG=$TAG REGION=$HTCGRID_REGION

docs/workshop/content/20_deploy_htc/60_building_htc_grid_artifacts/_index.en.md

@@ -18,15 +18,15 @@ A few notes on this command:
  - If `TAG` is omitted then *mainline* will be chosen as the default value.
  - If `REGION` is omitted then eu-west-1 will be used.
 
- Once the command above gets executed, A folder named `generated` will be created at `~/environment/aws-htc-grid/generated`. This folder will contain some important files, like the following:
+ Once the command above gets executed, A folder named `generated` will be created at `~/environment/htc-grid/generated`. This folder will contain some important files, like the following:
 
 * **grid_config.json**: A configuration file that contains the deployment settings for HTC-Grid.
 * **single-task-test.yaml**:  A Kubernetes job that can be used to test the installation and submit a single task.
 * **batch-task-test.yaml**:  A Kubernetes job that can be used to test the installation and submit multiple sessions of tasks.
 
 ## Configuring HTC-Grid Runtime
 
-The `~/environment/aws-htc-grid/generated/grid_config.json` file contains the configuration file that we will use to deploy HTC-Grid, let's explore a few sections:
+The `~/environment/htc-grid/generated/grid_config.json` file contains the configuration file that we will use to deploy HTC-Grid, let's explore a few sections:
 
 #### Configuration of Data-plane and Control-Plane DynamoDB Read/Write Capacity Modes
 

docs/workshop/content/20_deploy_htc/70_deploying_htc/_index.en.md

@@ -4,7 +4,7 @@ chapter: false
 weight: 80
 ---
 
-We are now ready to deploy HTC-Grid using the terraform stack provided by the project. The terraform stack uses `~/environment/aws-htc-grid/generated/grid_config.json` to drive the configuration of the deployment.
+We are now ready to deploy HTC-Grid using the terraform stack provided by the project. The terraform stack uses `~/environment/htc-grid/generated/grid_config.json` to drive the configuration of the deployment.
 
 Before we deploy the project, we need to initialize the terraform state. Remember we will be using the `$S3_TFSTATE_HTCGRID_BUCKET_NAME` bucket to hold the state. You can read more about the S3 terraform backend [here](https://www.terraform.io/docs/language/settings/backends/s3.html).
 
@@ -33,7 +33,7 @@ If `terraform apply` is successful then in the terraform folder two files are cr
 Let's validate that the Compute Plane has been set up accordingly. First of all, we need to configure our environment with [Kubectl](https://kubernetes.io/docs/tasks/tools/) configuration pointing to our cluster. We will also read from the terraform output the Agent config file and prepare our environment to select the configuration on newly created terminals.
 
   ```
-  cd ~/environment/aws-htc-grid/deployment/grid/terraform
+  cd ~/environment/htc-grid/deployment/grid/terraform
   export AGENT_CONFIG_FILE=$(terraform output -raw agent_config)
   echo "export AGENT_CONFIG_FILE=$AGENT_CONFIG_FILE" >> ~/.bashrc
   ```