Services Overview

This page describes the Engine services with a technical focus on the orchestration component.

Service Map

Service	Default Port	Role
Gateway	8080	External API entrypoint, edge alias route mapping, authentication boundary, request routing
Orchestrator	8083	Workflow runtime and execution control plane
Transformers	8081	Data transformation operations (XSLT, XML/JSON, Liquid)
Storage	8086	Runtime state, documents, secrets, and configuration data
Identity	8088	Authentication, authorization, user and token services
Scheduler	8089	Timed and named schedule management
Egress HTTP	8082	Outbound HTTP requests
Egress FTP	8084	FTP and FTPS operations
Egress SSH	8085	SSH and SFTP operations
Egress RabbitMQ	8087	Message publication and optional RabbitMQ-trigger subscriber mode
Egress IMAP	8091	IMAP mailbox access and email retrieval
Egress SMTP	8090	Outbound email delivery via SMTP
Egress Google Workspace	8080	Gmail, Calendar, and Drive API operations via service-account auth

Orchestration Component (Orchestrator)

The Orchestrator is the core execution runtime for MeshFlows workflows.

Responsibilities

• Loads workflow, connection, and policy definitions at startup and on reload.
• Validates and executes workflow steps in a deterministic order.
• Enforces invocation constraints such as HTTP-only or schedule-only workflows.
• Integrates with downstream services (transformers, storage, and egress).
• Captures run traces and trigger-event records for observability.
• Applies runtime concerns such as retry, idempotency, throttling, and OAuth token handling.

Runtime Flow

1. Receive invocation via HTTP or scheduler endpoint.
2. Resolve workflow and invocation policy.
3. Build trigger context from headers and request metadata.
4. Execute workflow steps through the workflow runner.
5. Persist trace and trigger-event data.
6. Return transformed result and include X-Request-ID for correlation.

Key Interfaces

Health and readiness:

• GET /healthz
• GET /readyz

Workflow discovery:

• GET /workflows
• GET /workflows/http
• GET /workflows/{workflow_name}

Execution entrypoints:

• POST /run/{workflow_name}
• POST /run
• POST /invoke/scheduled

Administration:

• POST /admin/validate
• POST /admin/reload

Observability and diagnostics:

• GET /api/traces
• GET /api/traces/{request_id}
• GET /api/trigger-events

Reliability Controls

• Idempotency support is available for duplicate-request protection.
• Throttling controls can limit request rate and in-flight concurrency per workflow partition.
• Retry behavior is workflow-driven and enforced by the runtime.
• Trace masking and secret-aware handling are applied to reduce sensitive-data leakage.

Note: idempotency and throttle stores are in-memory in the service process, so behavior is pod-local unless externalized.

Security and Access

• HTTP workflow invocation can use OAuth2/JWT scope checks and optional bearer-token fallback.
• Scheduled invocation has a dedicated auth path and can also enforce OAuth2 when configured.
• Admin endpoints are protected by reload token configuration.
• Connection secret references can be resolved from storage-backed secrets.

Integration with Other Services

• Uses Transformers for in-flow data conversion and mapping.
• Uses Storage for runtime config resolution and secret retrieval.
• Uses egress services for outbound communication protocols.
• Proxies schedule management operations to the Scheduler service.
• Executes workflows started by external trigger adapters (Gateway, Scheduler, RabbitMQ subscriber, future IMAP/SFTP trigger services).

Operational Notes

• Keep WORKFLOWS_DIR, WORKFLOW_ARTIFACTS_DIR, CONNECTIONS_DIR, and POLICIES_DIR mounted and versioned.
• Use POST /admin/validate before rollout to catch schema errors early.
• Use POST /admin/reload after updating flows/connections/policies.
• Track run behavior with trace APIs and correlate with X-Request-ID.