Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
78 lines (52 loc) · 5.89 KB

ARCHITECTURE.md

File metadata and controls

78 lines (52 loc) · 5.89 KB
Raw

Architecture

A high-level overview of how MEGZO Integrator Vortex is built. This describes the shape of the platform — the runtime, the connector model, and the reliability & security posture. It is not an implementation guide; the engine and connector code are private.

Runtime

MEGZO IV runs entirely on the Cloudflare edge:

Concern Technology
Compute TypeScript on Cloudflare Workers
Control-plane state Postgres behind Hyperdrive (connection pooling at the edge)
Blobs (invoice PDFs, rendered feeds) R2
Async work Cloudflare Queues
Coordination (rate-limit, cursors, locks, token sessions) Durable Objects
Schedules & reconciliation Cron Triggers
Observability OpenTelemetry — OTLP traces/metrics/logs
Admin UI React SPA behind Cloudflare Access
Egress a fixed-IP hop, so channels that IP-whitelist see one stable address

Business data stays in the ERP. The platform's own database is a control plane only — connector instances, flow definitions, sealed credentials, field-mapping configs, idempotency links, sync cursors, an event log and a dead-letter queue. It is multi-tenant with row-level security keyed on the tenant.

The connector model

Every integrated system is a connector that ships a JSON-Schema-validated manifest declaring its key, kind (erp / marketplace / ecommerce / courier / feed / invoicing), authentication scheme, capabilities, events, rate limits and quirks. The catalogue, rate-limiter, mapping UI and provisioning funnel all read that one manifest.

Capabilities are the vocabulary the engine speaks:

  • Source capabilities — read_catalog, sync_stock_source, emit_orders, provide_invoice, provide_fulfillment
  • Sink capabilities — ingest_orders, publish_offers, sync_stock, acknowledge_orders, push_invoice, push_awb
  • Feed capabilities — render_feed, handle_returns

A connector implements only the methods matching its declared capabilities — enforced by a conformance harness. The engine dispatches purely on capabilities and canonical entities; it never special-cases a particular system.

Adding a system = manifest + adapter + schema catalogue + a passing conformance test. No engine change.

The canonical model & mapping

Everything flows through versioned canonical types — offer, order, shipment, stock level. Each connector translates its native shape ⇄ canonical through per-(tenant, connector, entity, direction) mapping configuration. That configurability is the product: a maintainer can remap a field, pick a transform, preview it against a sample record, and save a versioned mapping — without a deploy.

Transforms are a whitelist of small, pure, typed functions (no arbitrary code execution). Mapping errors are collected per field and routed to the dead-letter queue — never dropped silently.

Sync flows — configuration, not code

A tenant's integration is a set of flows. A flow wires one connector's source capability → the canonical model → another connector's sink capability, and the engine executes them generically. Adding a channel is a connector instance plus a few flow rows.

The decisive consequence: the orders flow for any marketplace pours into the same ERP ingest_orders sink. Dozens of marketplaces, one order sink, zero per-channel engine code.

  emit_orders            canonical Order            ingest_orders
  (marketplace)  ───────────────────────────────▶  (ERP → sales order)
                         de-dup + idempotency

Reliability

  • Idempotency — a channel order can never create a second ERP sales order (a unique link plus an order-lock).
  • Oversell protection — stock is pushed with a safety buffer (channel = max(0, available − buffer)) and reconciled.
  • Retries — exponential backoff with jitter, honouring 429 / Retry-After; exhausted work lands in the dead-letter queue with an alert.
  • Reconciliation crons — a daily order sweep, an offer/stock drift check, and orphan detection heal divergence.
  • Async actions are awaited — where a system accepts a write and processes it asynchronously, the platform polls to completion before reporting success.
  • Circuit breakers & alerts — per-instance breakers and fleet-wide alert evaluation (DLQ depth, oversell, price-swing, error-rate) — all connector-agnostic, so every connector inherits them automatically.

Security

  • Secret hygiene — channel and ERP credentials are encrypted at rest (AES-256-GCM, per-tenant ciphertext), decrypted only in-memory at the point of use, and never logged, persisted in plaintext, or returned to the UI. Rotation is supported.
  • Webhook authentication — every inbound webhook is verified at the boundary before processing (signature, timestamp, replay window). Unauthenticated requests are rejected.
  • Logging scrub — no bodies, auth headers, tokens or credentials in logs; references and hashes only.
  • Observability — one trace per flow execution, propagated source → hub → sink, with the trace id written to the event log and the dead-letter queue.
  • Least privilege — each integration's service user is scoped to only what its flows need; the admin UI runs behind Cloudflare Access with a locked content-security policy.

Compliance posture

MEGZO IV acts as a data processor: marketplace orders carry personal data, handled under a per-client data-processing agreement, with data minimisation, encryption in transit and at rest, and a retention/deletion policy. The platform never generates fiscal or legal invoices — it only fetches already-produced documents and attaches them.

For the catalogue of systems this architecture connects, see CONNECTORS.md. To evaluate MEGZO IV, contact office@megzo.biz.