Skip to content

Commit b038d00

Browse files
leandrodamascenaleandrodamascena
committed
Java versions for the-efs-lambda stack. Update TS and Python Stack to CDK 1.82.0
1 parent 95fa039 commit b038d00

File tree

12 files changed

+644
-293
lines changed

12 files changed

+644
-293
lines changed

‎the-efs-lambda/csharp/src/TheEfsLambda/TheEfsLambda.csproj

+3-3
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,9 @@
99

1010
<ItemGroup>
1111
<!-- CDK Construct Library dependencies -->
12-
<PackageReference Include="Amazon.CDK" Version="1.80.0" />
13-
<PackageReference Include="Amazon.CDK.AWS.APIGatewayv2.Integrations" Version="1.80.0" />
14-
<PackageReference Include="Amazon.CDK.AWS.EFS" Version="1.80.0" />
12+
<PackageReference Include="Amazon.CDK" Version="1.82.0" />
13+
<PackageReference Include="Amazon.CDK.AWS.APIGatewayv2.Integrations" Version="1.82.0" />
14+
<PackageReference Include="Amazon.CDK.AWS.EFS" Version="1.82.0" />
1515

1616
<!-- jsii Roslyn analyzers (un-comment to obtain compile-time checks for missing required props
1717
<PackageReference Include="Amazon.Jsii.Analyzers" Version="*" PrivateAssets="all" />

‎the-efs-lambda/java/.gitignore

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
.classpath.txt
2+
target
3+
.classpath
4+
.project
5+
.idea
6+
.settings
7+
.vscode
8+
*.iml
9+
10+
# CDK asset staging directory
11+
.cdk.staging
12+
cdk.out
13+

‎the-efs-lambda/java/README.md

+57
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# The EFS Lambda Pattern
2+
3+
![efs lambda overview](img/overview.png)
4+
5+
This is a pattern that attaches an EFS file system to your lambda function to give it expandable, persistent storage. Having this level of storage in a Lambda Function opens the door to many new possibilities (multiple functions can even use the same file system).
6+
7+
Some Useful References:
8+
9+
| Author | Link |
10+
| ------------- | ------------- |
11+
| AWS Docs | [Using Amazon EFS with Lambda](https://docs.aws.amazon.com/lambda/latest/dg/services-efs.html) |
12+
| AWS Docs | [Configuring file system access for Lambda functions](https://docs.aws.amazon.com/lambda/latest/dg/configuration-filesystem.html) |
13+
| AWS Samples | [EFS for Lambda - Example SAM applications](https://github.com/aws-samples/aws-lambda-efs-samples) |
14+
| James Beswick | [Using Amazon EFS for AWS Lambda in your serverless applications](https://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/) |
15+
| Danilo Poccia | [A Shared File System for Your Lambda Functions](https://aws.amazon.com/blogs/aws/new-a-shared-file-system-for-your-lambda-functions/) |
16+
| Yan Cui | [Unlocking New Serverless Use Caes With EFS and Lambda](https://lumigo.io/blog/unlocking-more-serverless-use-cases-with-efs-and-lambda/) |
17+
18+
19+
## What's Included In This Pattern?
20+
This pattern covers the first half of Danilo Poccia's awesome [blog post](https://aws.amazon.com/blogs/aws/new-a-shared-file-system-for-your-lambda-functions/). After deployment you will have an API Gateway HTTP API where any url you hit gets directed to a Lambda Function that is integrated with EFS.
21+
22+
### VPC
23+
A VPC is bundled in this pattern because EFS requires it, this is using the default settings from CDK so if you want to put this in production you will have to review / refine this
24+
25+
### EFS FileSystem
26+
A FileSystem is included in the above VPC with a removal policy of destroy. In a production system you probably would want to retain your storage on stack deletion.
27+
28+
POSIX permissions are also setup for this File System
29+
30+
### Lambda Function
31+
A simple Python lambda function that interacts with the file system - storing, retrieving and deleting messages
32+
33+
### API Gateway HTTP API
34+
This is configured with the Lambda Function as the default handler for any url you hit.
35+
36+
## How Do I Test This Pattern?
37+
38+
Our deployed Lambda Function is acting as a shared message broker. It allows you to send messages to it which it stores in EFS, then you can retrieve all messages to read them or delete all messages after you have finished reading.
39+
40+
The Lambda Function will behave differently based on the RESTful verb you use:
41+
42+
- GET - Retrieve messages
43+
- POST - Send a message (whatever you send in the body is the message)
44+
- DELETE - Deletes all stored messages
45+
46+
The URL for the HTTP API to use these commands will be printed in the CloudFormation stack output after you deploy
47+
48+
Note - After deployment you may need to wait 60-90 seconds before the implementation works as expected. There are a lot of network configurations happening so you need to wait on propagation
49+
50+
## Useful commands
51+
52+
* `mvn package` compile and run tests
53+
* `cdk ls` list all stacks in the app
54+
* `cdk synth` emits the synthesized CloudFormation template
55+
* `cdk deploy` deploy this stack to your default AWS account/region
56+
* `cdk diff` compare deployed stack with current state
57+
* `cdk docs` open CDK documentation

‎the-efs-lambda/java/cdk.json

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"app": "mvn -e -q compile exec:java",
3+
"context": {
4+
"@aws-cdk/core:enableStackNameDuplicates": "true",
5+
"aws-cdk:enableDiffNoFail": "true",
6+
"@aws-cdk/core:stackRelativeExports": "true",
7+
"@aws-cdk/aws-ecr-assets:dockerIgnoreSupport": true,
8+
"@aws-cdk/aws-secretsmanager:parseOwnedSecretName": true,
9+
"@aws-cdk/aws-kms:defaultKeyPolicies": true
10+
}
11+
}

‎the-efs-lambda/java/lambda_fns/message_wall.py

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
import os
2+
import fcntl
3+
4+
MSG_FILE_PATH = '/mnt/msg/content'
5+
6+
7+
def get_messages():
8+
try:
9+
with open(MSG_FILE_PATH, 'r') as msg_file:
10+
fcntl.flock(msg_file, fcntl.LOCK_SH)
11+
messages = msg_file.read()
12+
fcntl.flock(msg_file, fcntl.LOCK_UN)
13+
except:
14+
messages = 'No message yet.'
15+
return messages
16+
17+
18+
def add_message(new_message):
19+
with open(MSG_FILE_PATH, 'a') as msg_file:
20+
fcntl.flock(msg_file, fcntl.LOCK_EX)
21+
msg_file.write(new_message + "\n")
22+
fcntl.flock(msg_file, fcntl.LOCK_UN)
23+
24+
25+
def delete_messages():
26+
try:
27+
os.remove(MSG_FILE_PATH)
28+
except:
29+
pass
30+
31+
32+
def lambda_handler(event, context):
33+
method = event['requestContext']['http']['method']
34+
if method == 'GET':
35+
messages = get_messages()
36+
elif method == 'POST':
37+
new_message = event['body']
38+
add_message(new_message)
39+
messages = get_messages()
40+
elif method == 'DELETE':
41+
delete_messages()
42+
messages = 'Messages deleted.'
43+
else:
44+
messages = 'Method unsupported.'
45+
return messages

‎the-efs-lambda/java/pom.xml

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
3+
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
4+
<modelVersion>4.0.0</modelVersion>
5+
6+
<groupId>com.cdkpatterns</groupId>
7+
<artifactId>the-efs-lambda</artifactId>
8+
<version>0.1</version>
9+
10+
<properties>
11+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
12+
<cdk.version>1.82.0</cdk.version>
13+
<junit.version>5.7.0</junit.version>
14+
</properties>
15+
16+
<build>
17+
<plugins>
18+
<plugin>
19+
<groupId>org.apache.maven.plugins</groupId>
20+
<artifactId>maven-compiler-plugin</artifactId>
21+
<version>3.8.1</version>
22+
<configuration>
23+
<source>1.8</source>
24+
<target>1.8</target>
25+
</configuration>
26+
</plugin>
27+
28+
<plugin>
29+
<groupId>org.codehaus.mojo</groupId>
30+
<artifactId>exec-maven-plugin</artifactId>
31+
<version>3.0.0</version>
32+
<configuration>
33+
<mainClass>com.cdkpatterns.TheEfsLambdaApp</mainClass>
34+
</configuration>
35+
</plugin>
36+
</plugins>
37+
</build>
38+
39+
<dependencies>
40+
<!-- AWS Cloud Development Kit -->
41+
<dependency>
42+
<groupId>software.amazon.awscdk</groupId>
43+
<artifactId>core</artifactId>
44+
<version>${cdk.version}</version>
45+
</dependency>
46+
<dependency>
47+
<groupId>software.amazon.awscdk</groupId>
48+
<artifactId>lambda</artifactId>
49+
<version>${cdk.version}</version>
50+
</dependency>
51+
<dependency>
52+
<groupId>software.amazon.awscdk</groupId>
53+
<artifactId>ec2</artifactId>
54+
<version>${cdk.version}</version>
55+
</dependency>
56+
<dependency>
57+
<groupId>software.amazon.awscdk</groupId>
58+
<artifactId>efs</artifactId>
59+
<version>${cdk.version}</version>
60+
</dependency>
61+
<dependency>
62+
<groupId>software.amazon.awscdk</groupId>
63+
<artifactId>apigatewayv2</artifactId>
64+
<version>${cdk.version}</version>
65+
</dependency>
66+
<dependency>
67+
<groupId>software.amazon.awscdk</groupId>
68+
<artifactId>apigatewayv2-integrations</artifactId>
69+
<version>${cdk.version}</version>
70+
</dependency>
71+
72+
<dependency>
73+
<groupId>org.junit.jupiter</groupId>
74+
<artifactId>junit-jupiter-api</artifactId>
75+
<version>${junit.version}</version>
76+
<scope>test</scope>
77+
</dependency>
78+
<dependency>
79+
<groupId>org.junit.jupiter</groupId>
80+
<artifactId>junit-jupiter-engine</artifactId>
81+
<version>${junit.version}</version>
82+
<scope>test</scope>
83+
</dependency>
84+
<dependency>
85+
<groupId>org.assertj</groupId>
86+
<artifactId>assertj-core</artifactId>
87+
<version>3.18.1</version>
88+
<scope>test</scope>
89+
</dependency>
90+
</dependencies>
91+
</project>

‎the-efs-lambda/java/src/main/java/com/cdkpatterns/TheEfsLambdaApp.java

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
package com.cdkpatterns;
2+
3+
import software.amazon.awscdk.core.App;
4+
5+
import java.util.Arrays;
6+
7+
public class TheEfsLambdaApp {
8+
public static void main(final String[] args) {
9+
App app = new App();
10+
11+
new TheEfsLambdaStack(app, "TheEfsLambdaStack");
12+
13+
app.synth();
14+
}
15+
}

‎the-efs-lambda/java/src/main/java/com/cdkpatterns/TheEfsLambdaStack.java

+74
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
package com.cdkpatterns;
2+
3+
import software.amazon.awscdk.core.*;
4+
import software.amazon.awscdk.services.lambda.Code;
5+
import software.amazon.awscdk.services.lambda.Function;
6+
import software.amazon.awscdk.services.lambda.Runtime;
7+
import software.amazon.awscdk.services.ec2.Vpc;
8+
import software.amazon.awscdk.services.efs.FileSystem;
9+
import software.amazon.awscdk.services.efs.AccessPoint;
10+
import software.amazon.awscdk.services.efs.AccessPointOptions;
11+
import software.amazon.awscdk.services.efs.Acl;
12+
import software.amazon.awscdk.services.efs.PosixUser;
13+
import software.amazon.awscdk.services.apigatewayv2.HttpApi;
14+
import software.amazon.awscdk.services.apigatewayv2.integrations.LambdaProxyIntegration;
15+
16+
public class TheEfsLambdaStack extends Stack {
17+
public TheEfsLambdaStack(final Construct scope, final String id) {
18+
this(scope, id, null);
19+
}
20+
21+
public TheEfsLambdaStack(final Construct scope, final String id, final StackProps props) {
22+
super(scope, id, props);
23+
24+
// EFS needs to be setup in a VPC
25+
Vpc vpc = Vpc.Builder.create(this, "Vpc")
26+
.maxAzs(2)
27+
.build();
28+
29+
// Create a file system in EFS to store information
30+
FileSystem fileSystem = FileSystem.Builder.create(this, "FileSystem")
31+
.vpc(vpc)
32+
.removalPolicy(RemovalPolicy.DESTROY)
33+
.build();
34+
35+
// Create a access point to EFS
36+
AccessPoint accessPoint = fileSystem.addAccessPoint("AccessPoint",
37+
AccessPointOptions.builder()
38+
.createAcl(
39+
Acl
40+
.builder()
41+
.ownerGid("1001").ownerUid("1001").permissions("750")
42+
.build())
43+
.path("/export/lambda")
44+
.posixUser(
45+
PosixUser
46+
.builder()
47+
.gid("1001").uid("1001")
48+
.build())
49+
.build());
50+
51+
// Create the lambda function
52+
Function functionProxyHandler = Function.Builder.create(this, "rdsProxyHandler")
53+
.runtime(Runtime.PYTHON_3_8)
54+
.code(Code.fromAsset("lambda_fns"))
55+
.handler("message_wall.lambda_handler")
56+
.vpc(vpc)
57+
.filesystem(software.amazon.awscdk.services.lambda.FileSystem.fromEfsAccessPoint(accessPoint, "/mnt/msg"))
58+
.build();
59+
60+
// Api Gateway HTTP integration
61+
HttpApi httpApi = HttpApi.Builder.create(this, "EFS Lambda")
62+
.defaultIntegration(
63+
LambdaProxyIntegration
64+
.Builder
65+
.create()
66+
.handler(functionProxyHandler)
67+
.build())
68+
.build();
69+
70+
CfnOutput.Builder.create(this, "HTTP API Url")
71+
.value(httpApi.getUrl())
72+
.build();
73+
}
74+
}

‎the-efs-lambda/java/src/test/java/com/cdkpatterns/TheEfsLambdaTest.java

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
package com.cdkpatterns;
2+
3+
import software.amazon.awscdk.core.App;
4+
5+
import com.cdkpatterns.TheEfsLambdaStack;
6+
import com.fasterxml.jackson.databind.JsonNode;
7+
import com.fasterxml.jackson.databind.ObjectMapper;
8+
import com.fasterxml.jackson.databind.SerializationFeature;
9+
10+
import java.io.IOException;
11+
12+
import org.junit.jupiter.api.Test;
13+
import static org.assertj.core.api.Assertions.assertThat;
14+
15+
public class TheEfsLambdaTest {
16+
private final static ObjectMapper JSON =
17+
new ObjectMapper().configure(SerializationFeature.INDENT_OUTPUT, true);
18+
19+
@Test
20+
public void testStack() throws IOException {
21+
App app = new App();
22+
TheEfsLambdaStack stack = new TheEfsLambdaStack(app, "test");
23+
24+
// synthesize the stack to a CloudFormation template and compare against
25+
// a checked-in JSON file.
26+
JsonNode actual = JSON.valueToTree(app.synth().getStackArtifact(stack.getArtifactId()).getTemplate());
27+
28+
// After synth, performs some basic tests
29+
30+
// AWS::EC2::VPC exists test
31+
assertThat(actual.toString()).contains("AWS::EC2::VPC");
32+
33+
// AWS::EFS::AccessPoint exists test
34+
assertThat(actual.toString()).contains("AWS::EFS::AccessPoint");
35+
36+
// AWS::EFS::FileSystem exists test
37+
assertThat(actual.toString()).contains("AWS::EFS::FileSystem");
38+
39+
}
40+
}

‎the-efs-lambda/python/requirements.txt

+6-6
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
-e .
2-
aws-cdk.core==1.80.0
3-
aws-cdk.aws-apigatewayv2==1.80.0
4-
aws-cdk.aws-apigatewayv2-integrations==1.80.0
5-
aws-cdk.aws-ec2==1.80.0
6-
aws-cdk.aws-lambda==1.80.0
7-
aws-cdk.aws-efs==1.80.0
2+
aws-cdk.core==1.82.0
3+
aws-cdk.aws-apigatewayv2==1.82.0
4+
aws-cdk.aws-apigatewayv2-integrations==1.82.0
5+
aws-cdk.aws-ec2==1.82.0
6+
aws-cdk.aws-lambda==1.82.0
7+
aws-cdk.aws-efs==1.82.0

0 commit comments

Comments
 (0)