Cloud Integration Showcase

This project is a demonstration of a serverless integration platform built on AWS, designed to showcase skills relevant to the Integration Engineer (AWS) role. It mirrors the responsibilities and technologies from the job description, including data querying with AWS Glue and Athena, Lambda-based interfaces with API Gateway, and event-driven architecture with EventBridge.

Core Features

• Advanced Data Transformation: An AWS Glue ETL job that transforms nested JSON into a canonical data model in Parquet format for optimized querying.
• Decoupled API Processing: A secure API Gateway endpoint that uses a Lambda authorizer and queues requests in RabbitMQ for asynchronous processing.
• Bidirectional Salesforce Integration: The system creates Case records in Salesforce and is prepared to receive outbound messages from Salesforce.
• Polyglot Development: The project includes Lambda functions written in both TypeScript and C#/.NET.
• Infrastructure as Code (IaC) & CI/CD: The entire infrastructure is defined using the AWS CDK, with a CodePipeline for automated deployments.
• Comprehensive Monitoring: A CloudWatch dashboard provides a centralized view of key application metrics.
• Security by Design: The project follows the principle of least privilege, with fine-grained IAM policies for all services.
• Unit Testing: The project includes unit tests for the Lambda functions.

Prerequisites

Before you begin, ensure you have the following:

• An AWS account with the AWS CLI configured.
• Node.js and the .NET SDK installed.
• A GitHub account and a personal access token with repo and workflow scopes.
• A Salesforce Developer Edition account.

Deployment

For detailed instructions on how to set up your local environment and deploy the project, please see the Developer's Guide.

Testing

Data Querying Engine (Glue & Athena)

1. Navigate to the Glue service in the AWS Console and run the LicenseDataTransformJob ETL job.
2. Navigate to the Athena service in the AWS Console.
3. Select the LicenseQueryWorkGroup workgroup.
4. In the query editor, run the following query to see the transformed Parquet data:

   SELECT * FROM "license_history_db"."processed";

Document Processing Interface (API Gateway, RabbitMQ, Salesforce)

1. Prerequisites:
   • Ensure you have configured your Salesforce and API key credentials in AWS Secrets Manager as described in the Developer's Guide.
2. Testing:
   • Get the API Gateway endpoint from the output of the cdk deploy command.
   • Make a POST request to the /license-print-request endpoint with your API key in the Authorization header.

     curl -X POST \
       -H "Authorization: <your-api-key>" \
       -H "Content-Type: application/json" \
       -d '{ "licenseId": "LIC-001", "ownerName": "John Smith", "issueDate": "2022-01-15", "expiryDate": "2025-01-14", "status": "Active", "address": { "street": "123 Main St", "city": "Anytown", "postalCode": "12345" } }' \
       https://<your-api-id>.execute-api.<your-region>.amazonaws.com/prod/license-print-request

   • You should receive a 202 Accepted response.
   • Check the logs for the RabbitMqProcessorLambda in CloudWatch to see the message being processed.
   • Log in to your Salesforce account to see the new Case record that was created.

Event-Driven Integration (EventBridge & .NET Lambda)

1. Navigate to the EventBridge service in the AWS Console.
2. Select the IntegrationEventBus event bus.
3. Click "Send events" and enter the following event details:
   • Event source: com.my-app
   • Detail type: document.requested
   • Event detail: { "documentId": "12345" }
4. Navigate to the CloudWatch service and view the logs for the DocumentAutomationLambda function. You should see the event logged.

Further Reading

• Architecture Overview: A high-level look at the project's design.
• Architecture Diagram: A visual representation of the architecture.
• Developer's Guide: A comprehensive guide for developers working on this project.
• Troubleshooting Guide: Solutions to common problems.
• Contributing Guide: Guidelines for contributing to the project.