Skip to content

Workflows (Flows) — Workflow Definitions & Deployment

The Flows component contains all workflow definitions, connections, policies, and configurations for task automation and integration.

What are Flows?

Flows represent the application logic layer of MeshFlows: - Workflow Definitions — YAML files describing execution steps - Connections — Secure credential storage (HTTP, OAuth, FTP, SSH) - Policies — Execution rules and access control - Schedules — CronJob trigger configurations - Settings — Runtime configuration overrides

Flows are version controlled in Git and can be: - Deployed alongside the Engine (integrated deployment) - Deployed independently (as ConfigMaps or directly) - Updated without restarting other Engine services

Workflow Types

HTTP-Triggered Workflows

User or external system initiates via HTTP: 

curl -X POST http://meshflows/v1/run/my-workflow

Scheduled Workflows

Run automatically on a schedule: 

cron: "0 2 * * *"  # Daily at 2 AM

Event-Driven Workflows

Triggered by storage changes or RabbitMQ messages.

Start Here

Build a New Workflow

  1. First Workflow
  2. Workflow YAML Syntax
  3. Step Types
  4. Step Contract (Fields & Options)
  5. Testing

Integrate External Systems

  1. Connections
  2. Transformations
  3. Policies
  4. Examples

Run on Schedule or Events

  1. Schedules
  2. Deployment
  3. Error Handling

I’m a Developer — What Do I Need to Know?

Getting Started?

  1. First Workflow — Create your first workflow in 5 minutes
  2. Connections & Credentials — Store credentials securely
  3. Deployment Methods — Deploy to the Engine

Writing Workflows?

Testing & Debugging?

  1. Local Testing — Test locally before deploying
  2. Viewing Traces — Debug execution with Tempo (Forgejo mirror: Observability)

Key Concepts

Workflow Structure

name: my-workflow
invocation:
  type: [http] # use explicit list, e.g. [http, schedule]

steps:
  - name: step_1
    type: http
    method: GET
    url: https://api.example.com/data

  - name: step_2
    type: storage_write
    key: result
    body_from: step_1

Step Types

Type Purpose
http HTTP GET/POST/PUT/DELETE
transform XSLT, xml2json, json2xml, Liquid
storage_read Read key-value or documents
storage_write Write key-value or documents
if Conditional execution
for_each Loop over collections
parallel Execute steps concurrently
wait Delay execution
log Log and capture output
cancel Cancel workflow
compensation Rollback on failure

Connections (Secure Credentials)

type: http
name: my_api
properties:
  base_url: https://api.example.com
  headers:
    Authorization: "Bearer {secret-token}"

Policies (Execution Rules)

name: require_auth_header
type: required_headers
apply_to:
  - "sensitive-*"
apply_to_triggers:
  - http
headers:
  - Authorization
on_violation: reject

Common Workflows

1. Data Integration

Fetch data from external API, transform, store locally: 

HTTP GET → XML2JSON → Storage Write

2. File Processing

Upload file, process, send results: 

HTTP GET (file) → FTP Upload → Storage Write

3. Scheduled Reports

Generate reports on schedule, email results: 

CronJob Trigger → Transform Data → HTTP POST (email service)

4. Event-Driven Processing

React to storage changes or messages: 

Storage Change Event → Process → RabbitMQ Publish

5. Multi-Step Integration

Complex workflow with branching logic: 

HTTP GET → IF (check status) → Parallel (notify teams) → Compensate (on error)

Deployment Methods

1. ConfigMap (Development)

Quick deployment for testing: 

kubectl create configmap workflows --from-file=workflows/
kubectl create configmap connections --from-file=connections/

Workflows stored in Git, CI/CD manages deployment: - Push workflow YAML to Git - Forgejo Actions validates and builds - GitOps controller syncs changes to the cluster

3. API Upload (Advanced)

curl -X POST http://storage:8080/internal/upload/workflows/demo \
  -H "Authorization: Bearer <runtime-admin-token>" \
  -H "Content-Type: application/yaml" \
  --data-binary @workflows/demo.yaml

Other single-item uploads:

  • POST /internal/upload/connections/{name}
  • POST /internal/upload/policies/{name} (limited to required_headers)
  • POST /internal/upload/schedules/{name}
  • POST /internal/upload/config (runtime settings YAML)

After uploading a workflow, trigger a reload in orchestrator (POST /admin/reload).

Workflow Lifecycle

Edit YAML
   ↓
Local Testing
   ↓
Push to Git
   ↓
CI/CD Validation
   ↓
Deploy to Cluster
   ↓
Monitor in Tempo/Dashboard
   ↓
Update if needed

Common Tasks

Create First Workflow

→ First Workflow How-to

Add Credentials

→ Connections How-to

Deploy Workflows

→ Deployment How-to

Learn All Step Types

→ Step Types Reference

Debug Execution

→ Observability Guide (Forgejo mirror: Observability)

Understand Transformations

→ Transformations Reference

Next Steps

  1. First Time?: Start with First Workflow
  2. Learning: Check out Examples & Recipes
  3. Production: Review Deployment Methods and Security (Forgejo mirror: Security) (Forgejo mirror: Security)
  4. Advanced: Explore Error Handling & Compensation