Skip to content

Architecture Overview

MeshFlows Engine is a service-oriented runtime for workflow orchestration.

Core Components

  • Gateway: external API and request routing layer.
  • Orchestrator: workflow execution runtime and policy enforcement.
  • Transformers: XML/JSON and template transformation service.
  • Storage: workflow/config/secrets persistence and runtime key-value data.
  • Identity: token issuance and identity functions.
  • Scheduler: timed workflow invocation.
  • Egress services: protocol-specific outbound adapters.

Logical Request Path

  1. Client calls Gateway (/v1/api/*, /v1/run/*, or /v1/transform).
  2. Gateway resolves edge route mapping for /v1/api/* using exact method + path matching.
  3. Gateway forwards to Orchestrator and preserves correlation headers.
  4. Orchestrator executes workflow steps.
  5. Steps call Transformers, Storage, and egress services.
  6. Trace and trigger data are stored and exposed via API.

Data and Control Planes

  • Control plane: workflow definitions, connections, policies, schedules, runtime config.
  • Data plane: workflow payloads and step outputs.
  • Ops plane: health, readiness, traces, and admin endpoints.

Deployment Model

  • Containerized microservices, typically deployed on Kubernetes.
  • Service discovery through cluster DNS and service names.
  • Configuration via environment variables and mounted files/secrets.
  • Persistent data for Storage and message broker state.

Runtime Characteristics

  • Stateless API services where possible.
  • Explicit request correlation using X-Request-ID and correlation headers.
  • Async HTTP communication between services.
  • Retry/idempotency/throttling controls in orchestration runtime.

Messaging and Persistence Direction

  • Broker: RabbitMQ (durable work queues, retry, DLQ).
  • State: PostgreSQL for workflow run/step lifecycle and idempotency receipts.
  • Payload offload: MinIO for large message bodies by reference.
  • Two-track policy:
  • small track (<= 512KB): inline payload, optional in-flow transforms, CEL content routing.
  • large track (> 512KB): payload-by-reference, transforms only via provider services.

See ADR 0006 for normative details.

  • Services: 02-services.md
  • Orchestrator API: 04-orchestrator-api.md
  • CI/CD architecture: 11-cicd-pipeline.md
  • ADR: ../adr/0006-rabbitmq-persistence-two-track-routing.md