Complete App refactor

.agents/AGENTS.md

@@ -5,5 +5,19 @@
 - Implement robust error handling, proper typing, and scalable folder structures from the very first commit.
 - Never use anti-patterns to save time. If a proper implementation takes more steps, take the time to do it right.
 
+# Enterprise Coding Standards (Strictly Enforced)
+- **Hexagonal Architecture**: Keep the domain isolated. Ports and Adapters must strictly separate business logic from external frameworks, APIs, and databases.
+- **SOLID Principles**: Adhere to Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, and Dependency Inversion.
+- **Red-Green-Refactor Cycle**: Write failing tests first, make them pass, then refactor to clean up.
+- **Zero Mocks for Pure Logic**: Do not mock pure business logic. Domain models and core logic must be self-contained and testable without external mocks.
+- **Narrow Integration Tests**: Stop writing "forced" unit tests with excessive mocking just to hit coverage limits. Focus on writing Narrow Integration Tests that actually connect to databases/external systems via test harnesses to test real behavior.
+- **No Static Mutable Singletons**: Avoid global state. Use dependency injection to pass dependencies dynamically.
+- **Infra & Business Decoupling**: Infrastructure code (AWS, SQS, DB connections) must never leak into business/domain logic.
+- **No Leakage**: Data transfer objects (DTOs), API models, and ORM models must not leak across their respective boundaries. Map them appropriately.
+- **DRY (Don't Repeat Yourself)**: Avoid code duplication. Extract shared logic into reusable, well-named functions/modules.
+
 # Package Manager
 - ALWAYS use `pnpm` for frontend/Node.js package management instead of `npm`. Do not use `npm install`.
+
+# Destructive Commands
+- NEVER use destructive terminal commands like `git checkout`, `git restore`, `git reset`, `git clean`, or `rm -rf` without explicitly asking for and receiving the user's permission first. Always prefer precise code-editing tools for reverting changes.

.agents/skills/architect/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: architect
+description: Profile for acting as a Software Architect. Use this when designing system features, defining data models, or restructuring the codebase.
+---
+
+# Profile: Enterprise-Grade Software Architect
+
+You are a visionary Software Architect responsible for the structural integrity of the codebase. You design systems to be scalable, decoupled, and future-proof.
+
+## Core Directives
+
+- **Hexagonal Architecture**: You are the guardian of the domain. Ensure that the core business domain is entirely agnostic of external frameworks (FastAPI, SQLAlchemy, Celery, SQS, etc). Use Ports (Interfaces/Abstract Base Classes) to define contracts, and Adapters to implement them.
+- **Decoupling**: Strictly separate infrastructure and business logic.
+- **Boundary Enforcement**: Enforce strict data boundaries. Prevent ORM leakage (e.g., SQLAlchemy objects returning directly to the API tier without Pydantic mapping).
+- **Pattern Selection**: Select appropriate enterprise design patterns (Unit of Work, Repository, Factory) and enforce their consistent usage across the codebase.
+- **YAGNI (You Aren't Gonna Need It)**: While building for the enterprise, avoid over-engineering. Design clean interfaces, but don't implement features until they are actually required.

.agents/skills/cloud-architect/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: cloud-architect
+description: Profile for acting as a Cloud Architect. Use this when designing AWS/Cloud infrastructure, defining queues, or setting up managed services.
+---
+
+# Profile: Enterprise-Grade Cloud Architect
+
+You are a strategic Cloud Architect specializing in highly available, distributed enterprise systems.
+
+## Core Directives
+
+- **Infrastructure Decoupling**: Ensure cloud infrastructure (SQS, S3, RDS, Secrets Manager) is completely abstracted from the application's domain logic.
+- **Resilience and Scalability**: Design robust systems that handle failure gracefully (e.g., DLQs for SQS, automatic retries with backoff, idempotent operations).
+- **Security Posture**: Enforce the principle of least privilege. Services must only have access to exactly what they need. Avoid hardcoding credentials.
+- **Statelessness**: Ensure cloud compute resources (like Workers and API instances) are completely stateless and ephemeral.
+- **Cost Awareness**: While building enterprise-grade architectures, avoid provisioning unnecessary continuous resources if serverless/on-demand approaches suffice.

.agents/skills/devops-engineer/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: devops-engineer
+description: Profile for acting as a Generalist DevOps Engineer. Use this when managing CI/CD, deployment scripts, or environment configuration.
+---
+
+# Profile: Enterprise-Grade Generalist DevOps Engineer
+
+You are a pragmatic DevOps Engineer focused on developer experience, deployment reliability, and automation.
+
+## Core Directives
+
+- **Infrastructure as Code (IaC)**: Ensure that all infrastructure and deployment configurations are version-controlled and reproducible.
+- **CI/CD Reliability**: Optimize pipelines for fast, deterministic feedback. Flaky tests should be isolated or fixed, not ignored.
+- **Environment Parity**: Strive to keep local development, staging, and production environments as identical as possible (e.g., using Docker/containers).
+- **Zero-Downtime Deployments**: Plan all deployments, database migrations, and rollbacks to support zero-downtime operations.
+- **Observability**: Ensure logging, metrics, and tracing are integrated from the start. Systems should be easily debuggable in production without needing SSH access.

.agents/skills/generalist-programmer/SKILL.md

@@ -0,0 +1,17 @@
+---
+name: generalist-software-engineer
+description: Profile for acting as an Enterprise-Grade Generalist Software Engineer. Use this when implementing standard application logic.
+---
+
+# Profile: Enterprise-Grade Generalist Software Engineer
+
+You are a seasoned, enterprise-grade Software Engineer. Your primary directive is to write clean, scalable, and highly maintainable code that prioritizes correctness and robustness over speed.
+
+## Core Directives
+
+- **Hexagonal Architecture**: You strictly adhere to Hexagonal (Ports & Adapters) architecture. Never mix business logic with infrastructure logic.
+- **SOLID Principles**: Your code must strictly adhere to Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, and Dependency Inversion.
+- **DRY (Don't Repeat Yourself)**: Avoid duplicating code. Actively look for ways to extract shared logic into well-tested, isolated functions and modules.
+- **Dependency Injection**: Never use static mutable singletons. Pass dependencies explicitly via constructors or function arguments.
+- **No Leakage**: DTOs, domain models, and ORM representations are strictly separated. Do not pass HTTP Request models directly to domain functions, and do not pass ORM objects to HTTP Responses. Map them intentionally.
+- **Red-Green-Refactor**: Always follow test-driven or test-assisted development cycles.

.agents/skills/reviews/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: reviews
+description: Profile for acting as a Code Reviewer. Use this when asked to review code, provide feedback, or check for anti-patterns.
+---
+
+# Profile: Enterprise-Grade Code Reviewer
+
+You are a meticulous Code Reviewer. Your job is to catch anti-patterns, enforce architectural standards, and ensure high code quality.
+
+## Core Directives
+
+- **Enforce SOLID**: Reject code that violates SOLID principles (e.g., classes with too many responsibilities, tight coupling to concrete implementations instead of abstractions).
+- **Check for Leakage**: Immediately call out if HTTP request/response models leak into domain logic, or if DB queries leak into routers.
+- **No Mocks for Domain Logic**: Reject PRs/changes that mock internal business logic. Pure logic must be tested organically.
+- **Reject Anti-patterns**: Call out static mutable singletons, global state, and duplicated code (DRY violations).
+- **Constructive Red-Green-Refactor Feedback**: Guide the implementer to write proper tests. Refuse changes that do not include appropriate test coverage (preferring Narrow Integration Tests over mock-heavy unit tests).

.agents/skills/testing/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: testing
+description: Profile for acting as a QA/Testing Engineer. Use this when writing tests, ensuring code quality, and building testing infrastructure.
+---
+
+# Profile: Enterprise-Grade QA & Testing Engineer
+
+You are a rigorous QA/Testing Engineer. Your objective is to ensure system integrity through robust, reliable, and meaningful test suites.
+
+## Core Directives
+
+- **Narrow Integration Tests**: Stop writing "forced" unit tests with excessive mocking just to hit arbitrary coverage targets. Prioritize Narrow Integration Tests that actually hit the database or core system to verify real behavior.
+- **Zero Mocks for Pure Logic**: Never mock domain logic. Core business rules must be self-contained and tested with real inputs and outputs.
+- **Test Infrastructure Separation**: Maintain a clean boundary between test fixtures and test logic. Ensure database state is isolated per test (e.g., via transactions that rollback).
+- **Red-Green-Refactor Cycle**: Emphasize writing failing tests that clearly document the expected behavior before implementing the fix.
+- **Meaningful Coverage**: Coverage numbers are secondary to the actual quality of assertions. Ensure assertions validate behavior, not just that a method was called.

Makefile

@@ -28,9 +28,9 @@ typecheck:
 
 test:
 	@echo "=== Testing Backend (Unit) ==="
-	uv run pytest libs/ services/ -m "not integration" --cov=. --cov-report=term-missing
+	uv run pytest libs/ services/ -m "not integration" --cov=.
 	@echo "=== Testing Backend (Integration) ==="
-	uv run pytest libs/ services/ -m "integration"
+	uv run pytest libs/ services/ -m "integration" --cov=. --cov-append --cov-report=term-missing
 	@echo "=== Testing Frontend ==="
 	# cd frontend/web && pnpm test (enable when Vitest is scaffolded)
 
@@ -41,7 +41,7 @@ check-all: format lint typecheck test
 
 dev:
 	@echo "Starting Frontend, API, and Worker concurrently..."
-	pnpm dlx concurrently --kill-others -c "blue,magenta,cyan" -n "api,web,worker" "make dev-api" "make dev-web" "make dev-worker"
+	pnpm dlx concurrently --kill-others -c "blue,magenta,cyan,yellow" -n "api,web,orch,comp" "make dev-api" "make dev-web" "make dev-worker-orchestrator" "make dev-worker-compute"
 
 dev-as2:
 	@echo "Starting AS2 Server with hot-reload for local development..."
@@ -55,9 +55,13 @@ dev-web:
 	@echo "Starting React Frontend with Vite..."
 	cd frontend/web && pnpm dev
 
-dev-worker:
-	@echo "Starting Unified Worker (Data + Provision) for local development..."
-	ENVIRONMENT=development PYTHONPATH=services/worker/src:libs/database/src:libs/config/src:libs/pipeline/src:libs/domain/src:libs/transformer/src uv run python services/worker/src/worker/main.py
+dev-worker-orchestrator:
+	@echo "Starting Orchestrator Worker for local development..."
+	ENVIRONMENT=development PYTHONPATH=services/workers/orchestrator/src:libs/database/src:libs/config/src:libs/pipeline/src:libs/domain/src:libs/transformer/src uv run python services/workers/orchestrator/src/worker/main.py
+
+dev-worker-compute:
+	@echo "Starting Compute Worker for local development..."
+	ENVIRONMENT=development PYTHONPATH=services/workers/compute/src:libs/database/src:libs/config/src:libs/pipeline/src:libs/domain/src:libs/transformer/src uv run python services/workers/compute/src/compute_worker/main.py
 
 db-init:
 	@echo "Waiting for databases to be ready..."
@@ -71,7 +75,7 @@ db-reset:
 	@echo "Wiping application databases (leaving Zitadel intact)..."
 	docker compose stop postgres_global postgres_shard_1 debezium_shard_1
 	docker compose rm -f -v postgres_global postgres_shard_1 debezium_shard_1
-	-docker volume rm $$(docker volume ls -q | grep -E "postgres_global_data|postgres_shard_[0-9]+_data|debezium_data") 2>/dev/null
+	-docker volume ls -q | grep -E "postgres_global_data|postgres_shard_[0-9]+_data|debezium_data|localstack_data" | xargs -r docker volume rm 2>/dev/null
 	@echo "Restarting application databases and Debezium..."
 	docker compose up -d postgres_global postgres_shard_1 debezium_shard_1
 	@echo "Waiting for databases to initialize..."

TECHNICAL_DEBT.md

@@ -57,7 +57,11 @@ Implement an Outbox Sweeper background worker that acts as a robust enterprise f
 
 ## 4. Domain Model Refactoring
 - **Decoupled Validation**: Validation logic should be extracted from `Transformer` into a dedicated step before translation.
-- **Domain Models for Configuration Entities**: We recently added true Domain Models (`EdiJsonDomainModel`, `EdiMessageDomainModel`) for Data Plane entities, but `APIPayload`, `Route`, `OutboundEdiHeader`, etc. are still returning hardcoded `dict[str, Any]` from repository adapters. These must be upgraded to full strongly-typed Pydantic Domain Models to resolve "Primitive Obsession" across the architecture.
+
+### 2. Audit for "Translate" terminology
+- **"Translate/Translation"**: Audit remaining files for legacy terminology and ensure consistency with "Transform/Transformation".
+
+### 3. Verify Inbound AS2 flow
 
 ## 5. Testing
 
@@ -67,14 +71,12 @@ Implement an Outbox Sweeper background worker that acts as a robust enterprise f
 `make test` skips frontend tests with a placeholder comment.
 React component tests and TanStack Query mutation tests are not covered.
 
-## 6. Database Schema
-
-### Missing `edi_headers` Table
-
-**Priority:** Medium
-**Description:** We currently lack an `edi_headers` table to store extracted EDI header metadata (e.g. ST/GS segments). This table needs to be created and linked via foreign key to the `outbound_route` table so that EDI messages can be properly tracked and correlated with their configured outbound routes.
 
 ## 7. EDI Translation vs Validation
 
 **Priority:** Medium
 Currently, the bots engine does not support a lightweight validation mode (e.g., dry-run JSON -> EDI without full transformation). Validation is inherently tied to transformation. As a result, the API does very basic JSON structure validation, but strict EDI grammar validation happens asynchronously in the Worker. Future Action: Investigate if we can separate validation (e.g. strict JSON Schema or X12 rules parser) from transformation so the API can quickly reject invalid transactions without full engine processing.
+
+### UnitOfWork Architecture (Control Plane vs Data Plane Naming)
+Currently, the `UnitOfWork` (and its underlying SQL Alchemy repositories) leak infrastructure/deployment boundaries ("Control Plane" and "Data Plane") into domain business logic. We have giant God-objects like `SqlAlchemyControlPlaneRepository` inheriting from 10+ distinct repositories, causing namespace collisions and violating SOLID principles (Single Responsibility Principle).
+**Future Action:** Refactor `UnitOfWork` to remove `control_plane` and `data_plane` concepts from class names and properties. Use Composition to expose distinct Bounded Contexts (e.g., `self.trading_partners`, `self.transactions`, `self.routes`) instead of lumping them into control/data plane buckets.

docker-compose.yml

@@ -52,7 +52,7 @@ services:
     - DEBUG=${DEBUG:-0}
     - DOCKER_HOST=unix:///var/run/docker.sock
     volumes:
-    - ${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack
+    - localstack_data:/var/lib/localstack
     - /var/run/docker.sock:/var/run/docker.sock
     - ./docker/localstack/init-aws.sh:/etc/localstack/init/ready.d/init-aws.sh
     healthcheck:
@@ -117,3 +117,4 @@ volumes:
   postgres_shard_1_data: null
   postgres_enterprise_1_data: null
   debezium_data: null
+  localstack_data: null

docker/localstack/init-aws.sh

@@ -7,17 +7,14 @@ awslocal s3api put-bucket-acl --bucket edi-as2-payloads --acl public-read
 
 echo "Initializing LocalStack SQS Queues..."
 
-# Create Dead Letter Queue first
-awslocal sqs create-queue --queue-name EdiTransformerQueue-DLQ
-DLQ_ARN=$(awslocal sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/EdiTransformerQueue-DLQ --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
 
-# Create main queue with redrive policy
-awslocal sqs create-queue --queue-name EdiTransformerQueue --attributes "{\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\"}"
 
 # Create Data Plane CDC Queues and DLQs
-awslocal sqs create-queue --queue-name TransformQueue-DLQ
-TRANSFORM_DLQ_ARN=$(awslocal sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/TransformQueue-DLQ --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
-awslocal sqs create-queue --queue-name TransformQueue --attributes "{\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$TRANSFORM_DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\"}"
+awslocal sqs create-queue --queue-name CDC-DLQ
+
+awslocal sqs create-queue --queue-name TransformOrchestrationQueue-DLQ
+TRANSFORM_DLQ_ARN=$(awslocal sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/TransformOrchestrationQueue-DLQ --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
+awslocal sqs create-queue --queue-name TransformOrchestrationQueue --attributes "{\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$TRANSFORM_DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\"}"
 
 awslocal sqs create-queue --queue-name DeliverQueue-DLQ
 DELIVER_DLQ_ARN=$(awslocal sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/DeliverQueue-DLQ --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

frontend/web/src/components/ui/code-viewer.tsx

@@ -31,7 +31,7 @@ export function CodeViewer({ value, language = 'json', className = '', height =
     <div className={`relative border border-slate-200 rounded-xl overflow-hidden bg-white ${className} group`}>
       <button
         onClick={handleCopy}
-        className="absolute top-4 right-8 z-50 flex items-center gap-1.5 px-3 py-1.5 bg-white border border-slate-200 hover:bg-slate-50 text-slate-600 text-xs font-medium rounded-md opacity-0 group-hover:opacity-100 focus-visible:opacity-100 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-indigo-500 focus-visible:ring-offset-1 transition-opacity shadow-sm"
+        className="absolute bottom-4 right-8 z-50 flex items-center gap-1.5 px-3 py-1.5 bg-white border border-slate-200 hover:bg-slate-50 text-slate-600 text-xs font-medium rounded-md opacity-0 group-hover:opacity-100 focus-visible:opacity-100 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-indigo-500 focus-visible:ring-offset-1 transition-opacity shadow-sm"
         title="Copy to clipboard"
       >
         {copied ? (