new pattern - lambda-sftp-connector-step-function

lambda-sftp-connector-stepfunction/README.md

@@ -0,0 +1,60 @@
+# SFTP Server to Amazon S3
+
+This pattern shows how to use AWS Transfer Family SFTP connector to transfer files from SFTP server to Amazon S3. EventBridge is used to schedule periodic execution of the Step functions. The Step functions orchastrates the workflow.  It invokes a Lambda function to retrieve the list of files created or updated after the previous execution of the workflow, followed by another Lambda function which creates file transfer jobs in SFTP connector. SFTP Connector transfers these files from source SFTP server to S3. The status of each transfer job is pushed to DynamoDB using EventBridge and Lambda function. Retry mechanism for failed files is implemented using Lambda. 
+
+Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/lambda-sftp-connector-stepfunction
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+1. Change directory to the pattern directory:
+    ```
+    cd lambda-sftp-connector-stepfunction
+    ```
+1. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
+    ```
+    sam deploy --guided
+    ```
+1. During the prompts:
+    * Enter a stack name
+    * Enter the desired AWS Region
+    * Allow SAM CLI to create IAM roles with the required permissions.
+
+    Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults.
+
+1. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing.
+
+## How it works
+
+Explain how the service interaction works.
+
+## Testing
+
+Go to EventBridge schedule and check the configured time of Step functions execution. You may manually execute the Step functions. Once the execution is complete, check S3 buckets. All files present in the source SFTP server are transferred to given S3 bucket.
+
+## Cleanup
+
+1. Delete the stack
+    ```bash
+    aws cloudformation delete-stack --stack-name STACK_NAME
+    ```
+1. Confirm the stack has been deleted
+    ```bash
+    aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
+    ```
+----
+Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

lambda-sftp-connector-stepfunction/example-pattern.json

@@ -0,0 +1,59 @@
+{
+  "title": "SFTP Connector, Lambda, Step Functions to periodically transfer files from SFTP server to S3",
+  "description": "Create a Step Functions for orchastration and Lambda functions to execute the business logic. Use EventBridge for scheduling and other events mnagament.",
+  "language": "Python",
+  "level": "200",
+  "framework": "CDK",
+  "introBox": {
+    "headline": "How it works",
+    "text": [
+      "This sample project demonstrates how to use an AWS Step Functions state machine to sequentially execute Lambda functions to create SFTP connector jobs to transfer files. This pattern is leveraging .",
+      "With AWS Transfer SFTP Connector you can retrieve list of directories, subdirectories and files from your remote SFTP server. It can also transfer these files to S3. The Step Function is used to retrieve file list in first step and transfer those files in following step. The failure events are sent to EventBridge and SQS. A Lambda function retrieve these events and retries file transfer by creating new transfer job with SFTP connector.",
+      "This pattern deploys one Step Functions, one SFTP Connector, three Lambda functions, one S3 bucket and three EventBridge events."
+    ]
+  },
+  "gitHub": {
+    "template": {
+      "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-sftp-connector-stepfunction",
+      "templateURL": "serverless-patterns/lambda-sftp-connector-stepfunction",
+      "projectFolder": "slambda-sftp-connector-stepfunction",
+      "templateFile": "lambda-sftp-connector-stepfunction_stack.py"
+    }
+  },
+  "resources": {
+    "bullets": [
+      {
+        "text": "Call Athena with Step Functions",
+        "link": "https://docs.aws.amazon.com/step-functions/latest/dg/connect-athena.html"
+      },
+      {
+        "text": "Amazon Athena - Serverless Interactive Query Service",
+        "link": "https://aws.amazon.com/athena/"
+      }
+    ]
+  },
+  "deploy": {
+    "text": [
+      "sam deploy"
+    ]
+  },
+  "testing": {
+    "text": [
+      "See the GitHub repo for detailed testing instructions."
+    ]
+  },
+  "cleanup": {
+    "text": [
+      "Delete the stack: <code>cdk delete</code>."
+    ]
+  },
+  "authors": [
+    {
+      "name": "Your name",
+      "image": "link-to-your-photo.jpg",
+      "bio": "Your bio.",
+      "linkedin": "linked-in-ID",
+      "twitter": "twitter-handle"
+    }
+  ]
+}

lambda-sftp-connector-stepfunction/src/app.js

@@ -0,0 +1,10 @@
+/*! Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *  SPDX-License-Identifier: MIT-0
+ */
+
+'use strict'
+
+exports.handler = async (event) => {
+  // Lambda handler code
+  console.log(JSON.stringify(event, 0, null))
+}

lambda-sftp-connector-stepfunction/template.yaml

@@ -0,0 +1,16 @@
+AWSTemplateFormatVersion: '2010-09-09'
+Transform: AWS::Serverless-2016-10-31
+Description: Serverless patterns - Service to Service description
+
+# Comment on each global 
+Globals:
+
+
+# Comment each resource section to explain usage
+Resources:
+
+
+# List all common outputs for usage
+Outputs:
+
+