How to use Centralized Logging with OpenSearch without using public subnets?

[Paul-AWS]

For network security reasons, I can't use a public or private subnet with a NAT gateway. How do I use Centralized Logging with OpenSearch in this case?

[Paul-AWS]

In this case, additional configuration is required to complete log collection in the private network. You can follow the steps below to configure it.

Step 1. Decide whether to use the Console.

You can use the Centralized Logging with OpenSearch console to visually and easily create your log ingestion pipelines, or you can create them using CloudFormation stacks for each log pipeline. The console creates resources that are open to the internet, such as CloudFront.

• If your security regulations require that these public resources cannot be created, you can use CloudFormation stack directly to create a log ingestion pipeline without deploying a console.
• If your security regulations allow the creation of public resources, this will simplify the creation of log ingestion pipeline. You need to create an OpenSearch cluster in your existing VPC in advance, then use the “Deploy from an Existing VPC” mode to deploy Centralized Logging with OpenSearch. For both the parameter “public subnets” and “private subnets”, select two subnets that belongs to the same VPC withOpenSearch cluster. After the deployment, you can create log ingestion pipeline using the console.

No matter which way you choose, you will have one or more log ingestion stacks in CloudFormation.

Step 2. Adding STS endpoints and S3 endpoints to the VPC allows Lambda to access log files in S3 through the intranet.

Step 3. Move the helper Lambda out of the VPC to create a role for OpenSearch.

1. Find the stack of log ingestion pipeline in CloudFormation, then search in the “Resources” tab on the right using the keyword “Helper”, find the Lambda function with Helper in its name, and click to jump to this Lambda function. This Lambda function provisions resources such as index patterns, visualization, and dashboards in OpenSearch.
2. On the Lambda function's “Configuration” tab, modify the configuration of the “VPC”, change the VPC option to None (record the value before the modification).

3. Modify the configuration of the “Environment Variable” to add a new set of parameters that the key is “AWS_STS_REGIONAL_ENDPOINTS” and the value is “regional”.

4. Switch to the Lambda function's "Code" tab, change several lines (line 92-102 in lambda_function.py) of code that create ISM, index and template to comments (they need to be executed in VPC), and click "Deploy" and and wait for the function to be deployed successfully.

5. Then click “Test” to execute the Lambda function. If the execution is successful, you'll see that the execution record shows that two backend roles have been created.

Step 4. Restore the Helper Lambda configuration and modify the code.

1. On the Lambda function's “Configuration” tab, modify the “VPC” configuration and modify the VPC and subnet to their previous values.
2. On the Lambda function's “Code” tab, restore the comments that create ISM, index and template in line 92-102 into code. Then change several lines of code that create backend role (line 84-89, line 106-108) to comments, then click "Deploy", and wait for the function to be deployed successfully.

3. Click “Test” to execute this Lambda function. If the execution is successful, you will see that the execution record shows that the index template was successfully inserted. Now you can see that the index pattern and dashboard have been created in the OpenSearch Dashboard.

Step 5. Add the STS endpoint configuration for the Processor Lambda.

1. Find the stack of log ingestion pipeline in CloudFormation, then search in the “Resources” tab on the right using the keyword “Processor”, find the Lambda function with Processor in its name, and click to jump to this Lambda function. This Lambda function ingests logs from S3 and write them into OpenSearch after processing.
2. On the Lambda function's “Configuration” tab, modify the configuration of the “Environment Variable” to add a new set of parameters that the key is “AWS_STS_REGIONAL_ENDPOINTS” and the value is “regional”.
3. Switch to the Lambda function's “Monitoring” tab and click “View CloudWatch Logs” to go to the execution log page to view the latest execution log. If the execution is successful, you will observe that the log was successfully written to the log.

Tips: You may need to use bastion, dedicated line or VPN on VPC to access the OpenSearch Dashboard, cause you can’t use AOS proxy in this mode.

Now you can ingest logs without using public subnets.