Skip to content

Commit 3ba0d61

Browse files
leggetterleggetter
authored
Merge pull request #447 from hookdeck/feat/azure-setup-example
feat: Add deployment scripts and configuration for Azure Container Apps
2 parents 4051cd9 + 75ea09d commit 3ba0d61

File tree

7 files changed

+1045
-0
lines changed

7 files changed

+1045
-0
lines changed

‎examples/azure/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.env.*

‎examples/azure/README.md‎

Lines changed: 336 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,336 @@
1+
# Deploy Outpost on Azure with Azure Service Bus
2+
3+
This example demonstrates how to deploy Outpost on Azure, using Azure Service Bus as the message queue.
4+
5+
## Prerequisites
6+
7+
Before you begin, ensure you have the following:
8+
9+
* [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) installed.
10+
* You are logged into your Azure account (`az login`).
11+
12+
## Architecture
13+
14+
This example deploys a distributed architecture on Azure, leveraging managed services for dependencies and Azure Container Apps for the Outpost services.
15+
16+
```mermaid
17+
graph TD
18+
subgraph "External Source"
19+
WebhookSource[Event Source]
20+
end
21+
22+
subgraph "Azure Container Apps"
23+
APIService["api service (Container)"]
24+
DeliveryService["delivery service (Container)"]
25+
LogService["log service (Container)"]
26+
end
27+
28+
subgraph "Azure Dependencies"
29+
PostgresDB["PostgreSQL (Log Storage)"]
30+
ServiceBusDelivery["Azure Service Bus (Delivery Queue)"]
31+
ServiceBusLog["Azure Service Bus (Log Queue)"]
32+
RedisCache["Redis (Entity Storage)"]
33+
end
34+
35+
subgraph "External Destinations"
36+
Delivery[Destination]
37+
end
38+
39+
WebhookSource -- "HTTPS Request" --> APIService
40+
41+
APIService -- " " --> PostgresDB
42+
APIService -- "Stores/Retrieves Session Data" --> RedisCache
43+
APIService -- "Enqueues Message" --> ServiceBusDelivery
44+
45+
ServiceBusDelivery -- "Consumes Message" --> DeliveryService
46+
DeliveryService -- " " --> RedisCache
47+
DeliveryService -- "Enqueues Log" --> ServiceBusLog
48+
49+
ServiceBusLog -- "Consumes Log" --> LogService
50+
LogService -- "Writes Log" --> PostgresDB
51+
52+
DeliveryService -- "Sends Events" --> Delivery
53+
```
54+
55+
### Components
56+
57+
#### Dependencies
58+
The deployment relies on Azure-managed services for its core dependencies:
59+
* **PostgreSQL**: Used for persistent log storage (`log storage`).
60+
* **Redis**: Used for entity storage and caching (`entity storage`).
61+
* **Azure Service Bus**: Used as the message queue for both the delivery (`delivery queue`) and log (`log queue`) services.
62+
63+
#### Outpost Services
64+
The Outpost application itself is deployed as three distinct services in Azure Container Apps:
65+
* **api**: The public-facing API that receives webhooks (`API Service`).
66+
* **delivery**: A backend service that processes and delivers webhooks from the queue (`Delivery Service`).
67+
* **log**: A backend service that processes and stores logs (`log service`).
68+
69+
## Scripts
70+
71+
This example includes three main scripts to manage the deployment:
72+
73+
* `dependencies.sh`: Provisions all the necessary Azure resources, including PostgreSQL for storage, Redis for caching, and Azure Service Bus for the message queue. It also configures the required permissions for the services to interact with each other. If a PostgreSQL password is not provided via the `PG_PASS` environment variable, a secure one will be generated automatically.
74+
* `local-deploy.sh`: Deploys the Outpost services using Docker Compose. It uses the Azure resources provisioned by the `dependencies.sh` script.
75+
* `diagnostics.sh`: Runs checks to validate deployments. Use `--local` for the Docker deployment or `--azure` for the Azure Container Apps deployment. The script requires a webhook URL for testing, which can be provided via the `--webhook-url` flag or the `WEBHOOK_URL` environment variable.
76+
* `azure-deploy.sh`: Deploys the Outpost services to Azure Container Apps.
77+
78+
## Deployment Steps using Outpost Locally
79+
80+
To deploy Outpost, you must run the scripts in the following order:
81+
82+
1. **Provision Dependencies:**
83+
```bash
84+
# To use a specific password (optional):
85+
# export PG_PASS=<YOUR_POSTGRES_PASSWORD>
86+
./dependencies.sh
87+
```
88+
89+
2. **Deploy Outpost:**
90+
```bash
91+
./local-deploy.sh
92+
```
93+
94+
3. **Run Diagnostics:**
95+
This command specifically targets the local Docker deployment. You will need a public webhook URL for the test.
96+
```bash
97+
export WEBHOOK_URL=<YOUR_PUBLIC_WEBHOOK_URL>
98+
bash ./diagnostics.sh --local
99+
```
100+
Alternatively, you can use the `--webhook-url` flag:
101+
```bash
102+
bash ./diagnostics.sh --local --webhook-url <YOUR_PUBLIC_WEBHOOK_URL>
103+
```
104+
105+
## Deploying Outpost to Azure Container Apps
106+
107+
### 1. Deploy with the Deployment Script (Recommended)
108+
109+
1. **Prepare Environment Files:**
110+
Before deploying, you need the `.env.outpost` and `.env.runtime` files.
111+
112+
* **To provision new dependencies (Recommended):** Run the scripts to generate the files automatically.
113+
```bash
114+
# To use a specific password (optional):
115+
# export PG_PASS=<YOUR_POSTGRES_PASSWORD>
116+
./dependencies.sh
117+
./local-deploy.sh
118+
```
119+
> **Note:** The `local-deploy.sh` script will also start services locally via Docker Compose. You can stop them with `docker-compose down` after the script finishes if you only intend to deploy to ACA.
120+
121+
* **To use existing dependencies:** Create `.env.outpost` and `.env.runtime` manually. Refer to the [Environment Variable Reference](#environment-variable-reference) section for details on the required variables.
122+
123+
2. **Run the Deployment Script:**
124+
Once the environment files are generated, run the `azure-deploy.sh` script to deploy the Outpost services to Azure Container Apps.
125+
126+
```bash
127+
./azure-deploy.sh
128+
```
129+
130+
### 2. Deploy Manually
131+
132+
These instructions outline how to manually deploy Outpost to Azure Container Apps (ACA).
133+
134+
#### 1. Prepare Environment Files
135+
136+
Before deploying manually, ensure you have the `.env.outpost` and `.env.runtime` files.
137+
138+
* **To provision new dependencies:** Run the scripts to generate the files automatically.
139+
```bash
140+
# To use a specific password (optional):
141+
# export PG_PASS=<YOUR_POSTGRES_PASSWORD>
142+
./dependencies.sh
143+
./local-deploy.sh
144+
```
145+
> **Note:** The `local-deploy.sh` script will also start services locally via Docker Compose. You can stop them with `docker-compose down` after the script finishes if you only intend to deploy to ACA.
146+
147+
* **To use existing dependencies:** Create `.env.outpost` and `.env.runtime` manually. Refer to the [Environment Variable Reference](#environment-variable-reference) section for details on the required variables.
148+
149+
#### 2. Load Environment Variables
150+
151+
Load the environment variables from both `.env.outpost` and `.env.runtime` into your shell session. Sourcing `.env.runtime` last ensures that it can override any common variables.
152+
```bash
153+
source .env.outpost && source .env.runtime
154+
```
155+
156+
#### 3. Create Azure Container Apps Environment
157+
158+
Create the ACA Environment using the variables loaded in the previous step.
159+
160+
```bash
161+
az containerapp env create \
162+
--name outpost-environment \
163+
--resource-group $RESOURCE_GROUP \
164+
--location $LOCATION
165+
```
166+
167+
#### 4. Deploy Each Container
168+
169+
Deploy each service as a separate container app. First, the environment variables from `.env.runtime` must be formatted into a string that can be passed to the Azure CLI.
170+
171+
```bash
172+
ENV_VARS_STRING=$(grep -v '^#' .env.runtime | sed 's/^export //g' | sed 's/"//g' | tr '\n' ' ')
173+
```
174+
175+
Now, deploy the containers using the created string.
176+
177+
**Deploy `api` service:**
178+
```bash
179+
az containerapp create \
180+
--name outpost-api \
181+
--resource-group $RESOURCE_GROUP \
182+
--environment outpost-environment \
183+
--image hookdeck/outpost:v0.4.0 \
184+
--target-port 3333 \
185+
--ingress external \
186+
--env-vars "SERVICE=api" $ENV_VARS_STRING
187+
```
188+
189+
**Deploy `delivery` service:**
190+
```bash
191+
az containerapp create \
192+
--name outpost-delivery \
193+
--resource-group $RESOURCE_GROUP \
194+
--environment outpost-environment \
195+
--image hookdeck/outpost:v0.4.0 \
196+
--ingress internal \
197+
--env-vars "SERVICE=delivery" $ENV_VARS_STRING
198+
```
199+
200+
**Deploy `log` service:**
201+
```bash
202+
az containerapp create \
203+
--name outpost-log \
204+
--resource-group $RESOURCE_GROUP \
205+
--environment outpost-environment \
206+
--image hookdeck/outpost:v0.4.0 \
207+
--ingress internal \
208+
--env-vars "SERVICE=log" $ENV_VARS_STRING
209+
```
210+
211+
### 3. Validate the Deployment
212+
213+
After deploying with either method, you can validate the deployment by running the following script. You will need a public webhook URL for the test.
214+
215+
```bash
216+
export WEBHOOK_URL=<YOUR_PUBLIC_WEBHOOK_URL>
217+
bash ./diagnostics.sh --azure
218+
```
219+
Alternatively, you can use the `--webhook-url` flag:
220+
```bash
221+
bash ./diagnostics.sh --azure --webhook-url <YOUR_PUBLIC_WEBHOOK_URL>
222+
```
223+
224+
## Azure Deployment Options: Overview, Pros & Cons
225+
226+
There are several ways to deploy Docker containers to Azure. Below is a summary of the main options, their advantages, and trade-offs:
227+
228+
### 1. Azure Container Apps (ACA)
229+
230+
**Description:** Managed service for running containerized applications with built-in scaling, ingress, and environment management. Each service is deployed as a separate container app.
231+
**Pros:**
232+
- No infrastructure management (serverless experience)
233+
- Built-in scaling and ingress
234+
- Easy integration with Azure resources
235+
- Good fit for microservices and distributed architectures
236+
**Cons:**
237+
- Limited support for multi-container orchestration compared to Docker Compose
238+
- Some advanced networking features may require extra configuration
239+
- Not suitable for very complex container topologies
240+
241+
### 2. Azure Container Instances (ACI)
242+
243+
**Description:** Directly run containers in Azure without managing VMs. Supports single containers or simple container groups.
244+
**Pros:**
245+
- Fast and simple deployment
246+
- No infrastructure management
247+
- Good for short-lived or simple workloads
248+
**Cons:**
249+
- Limited orchestration (multi-container support is basic)
250+
- No built-in scaling or ingress
251+
- Not ideal for production microservices
252+
253+
### 3. Azure Kubernetes Service (AKS)
254+
255+
**Description:** Full Kubernetes orchestration for complex, scalable deployments. Can convert Docker Compose files to Kubernetes manifests using tools like `kompose`.
256+
**Pros:**
257+
- Full orchestration and scaling
258+
- Advanced networking and service discovery
259+
- Suitable for large-scale, complex applications
260+
**Cons:**
261+
- Requires Kubernetes knowledge and management
262+
- More operational overhead
263+
- Overkill for simple deployments
264+
265+
### 4. Azure Web App for Containers
266+
267+
**Description:** Deploy container images to Azure App Service. Best for web apps and APIs.
268+
**Pros:**
269+
- Simple deployment for web-facing apps
270+
- Built-in scaling and monitoring
271+
**Cons:**
272+
- Limited control over networking and container lifecycle
273+
- Not suitable for multi-service architectures
274+
275+
## Using Docker Compose for Azure Deployments
276+
277+
Azure does not natively support `docker-compose.yml` for multi-container deployments in the same way as local Docker Compose. You can use conversion tools (e.g., `kompose` for AKS, or limited support for ACI) to generate Azure-compatible deployment files, but manual adjustments are often required.
278+
279+
## Why `azure-deploy.sh` Is a Reasonable Approach
280+
281+
The provided `azure-deploy.sh` script is a practical solution for deploying Outpost to Azure Container Apps because:
282+
283+
- It automates the deployment of multiple services, mirroring the local Docker Compose setup.
284+
- It leverages environment files generated by your local setup, ensuring consistency between local and cloud deployments.
285+
- It uses Azure CLI commands, which are well-supported and documented.
286+
- It validates the deployment and provides immediate feedback (logs, health checks).
287+
- It aligns with ACA’s architecture by deploying each service as a separate container app, following best practices for microservices.
288+
- It handles Azure resource provider registration and environment creation, reducing manual setup steps and potential errors.
289+
- It supports repeatable, scriptable deployments, making it easy to update, redeploy, or share the deployment process with others.
290+
- It enables troubleshooting and monitoring by fetching logs and checking health endpoints immediately after deployment.
291+
- It is flexible and can be adapted for different environments or configurations by simply updating environment files or script parameters.
292+
293+
**Summary:**
294+
For most users, `azure-deploy.sh` offers a balance of automation, reliability, maintainability, and alignment with Azure Container Apps best practices. It is recommended for Outpost deployments unless you require advanced orchestration (AKS) or have very simple workloads (ACI).
295+
296+
**Summary:**
297+
For most users, `azure-deploy.sh` offers a balance of automation, reliability, and maintainability. It is recommended for Outpost deployments unless you require advanced orchestration (AKS) or have very simple workloads (ACI).
298+
## Environment Variable Reference
299+
300+
If you are not using the `dependencies.sh` and `local-deploy.sh` scripts to provision your infrastructure, you will need to create the `.env.outpost` and `.env.runtime` files manually.
301+
302+
See the [Configure Azure Service Bus as the Outpost Internal Message Queue](https://outpost.hookdeck.com/docs/guides/service-bus-internal-mq) guide for more details on the environment variables required for Outpost and how to create the values.
303+
304+
### `.env.outpost`
305+
306+
This file contains variables related to the Azure infrastructure where the services will be deployed.
307+
308+
| Variable | Description | Example |
309+
| --- | --- | --- |
310+
| `LOCATION` | The Azure region for your resources. | `westeurope` |
311+
| `RESOURCE_GROUP` | The name of the Azure resource group. | `outpost-azure` |
312+
| `POSTGRES_URL` | The full connection URL for your PostgreSQL database. | `postgres://user:pass@host:5432/dbname` |
313+
| `REDIS_HOST` | The hostname of your Redis instance. | `outpost-redis.redis.cache.windows.net` |
314+
| `REDIS_PORT` | The port for your Redis instance. | `6379` |
315+
| `REDIS_PASSWORD` | The password for your Redis instance. | `your-redis-password` |
316+
| `REDIS_DATABASE` | The Redis database number. | `0` |
317+
| `AZURE_SERVICEBUS_CLIENT_ID` | The Client ID of the service principal for Service Bus access. | `...` |
318+
| `AZURE_SERVICEBUS_CLIENT_SECRET` | The Client Secret of the service principal. | `...` |
319+
| `AZURE_SERVICEBUS_SUBSCRIPTION_ID` | Your Azure Subscription ID. | `...` |
320+
| `AZURE_SERVICEBUS_TENANT_ID` | Your Azure Tenant ID. | `...` |
321+
| `AZURE_SERVICEBUS_NAMESPACE` | The name of the Service Bus namespace. | `outpost-internal` |
322+
| `AZURE_SERVICEBUS_RESOURCE_GROUP` | The resource group for the Service Bus. | `outpost-azure` |
323+
| `AZURE_SERVICEBUS_DELIVERY_TOPIC` | The name of the delivery topic. | `outpost-delivery` |
324+
| `AZURE_SERVICEBUS_DELIVERY_SUBSCRIPTION` | The name of the delivery subscription. | `outpost-delivery-sub` |
325+
| `AZURE_SERVICEBUS_LOG_TOPIC` | The name of the log topic. | `outpost-log` |
326+
| `AZURE_SERVICEBUS_LOG_SUBSCRIPTION` | The name of the log subscription. | `outpost-log-sub` |
327+
328+
### `.env.runtime`
329+
330+
This file contains secrets and runtime configuration for the Outpost services. It includes all variables from `.env.outpost` plus the following application-specific secrets.
331+
332+
| Variable | Description |
333+
| --- | --- |
334+
| `API_KEY` | A secret key for securing the Outpost API. |
335+
| `API_JWT_SECRET` | A secret for signing JWTs. |
336+
| `AES_ENCRYPTION_SECRET` | A secret for data encryption. |

0 commit comments

Comments
 (0)