Step Contract (Fields, Options, Rules)¶
This page documents the step-level contract for workflow YAML.
Scope: - Field-level options per step type. - Required vs optional fields. - Common validation and behavior rules.
Source of truth:
- engine/services/orchestrator/app/workflow_runner.py
- WorkflowStep discriminated union and corresponding step models.
Terminology¶
A capability provider is an HTTP service that registers with the capability-registry. Workflows use exclusively type: service_call plus capability_name and path for integrations, transforms, and scripts. Built-in control/context steps remain orchestrator-owned. Do not add new first-class integration types.
Shared Contract (Most Step Types)¶
Common fields used by many steps through shared mixins:
| Field | Required | Type | Description |
|---|---|---|---|
id |
yes | string |
Unique step identifier within the workflow. |
type |
yes | string |
Step discriminator (for example service_call, for_each). |
when |
no | object | Conditional execution rule based on context values. |
run_after |
no | list | Dependency gating based on previous step status. |
compensate |
no | object | Compensation steps for rollback behavior. |
retry_policy |
no | object | Per-step retry override (where supported). |
Notes:
- Use id:, not name:, for step identity.
- when supports equals, not_equals, or one_of (exactly one).
- run_after supports concise YAML {step_id: succeeded} and a status list {step_id: [succeeded, failed]} (OR within one predecessor, AND across predecessors).
- Status values: succeeded, failed, skipped, timedout. Combinations match Logic Apps Configure run after.
- Control-flow steps may handle retries differently than service_call.
Step Type Registry¶
Supported values for type (exact WorkflowStep union):
context_setcontext_incrementcontext_bindcontext_extract_jsoncontext_extract_xmlmilestonejson_setxml_set_textterminatewaitlogworkflow_callcollectfor_eachrepeat_untiliftryscopeparallelservice_call
Former first-class integration/transform/script types (http, xslt, ftp, storage_write, …) are not in the union. Map them via service_call — see Step Types.
Contract by Step Type¶
service_call¶
Required:
- id
- type: service_call
- capability_name
- path
Optional:
- capability_version (recommended, e.g. "^1.0")
- connection
- payload / payload_from
- first-class provider fields (capability-specific; see below)
- output_key
- timeout_seconds
- values (placeholder substitution)
- variable-binding fields (variable_from, operand, apply, result_to) for variables.*
- retry_policy
- shared fields
Validation rules:
- Do not mix exclusive payload modes (see model validators in workflow_runner.py).
- Capability must be discoverable via the capability registry at runtime.
Common capability contracts¶
script.execute¶
Required first-class fields: runtime, code. Path: /v1/execute.
Optional: payload_from (SES input), timeout_ms, legacy JSON payload.
See Script steps howto.
For C# / .NET custom logic, use a capability provider + service_call (not SES):
C# Custom Process.
egress.http.call¶
Path: /call. First-class: url (or legacy payload.url), optional method, headers, body, request_timeout_seconds, fail_on_error_status, accepted_status_codes, values.
egress.rabbitmq.topic.send¶
Path: /publish/topic. First-class: domain, service, action, version; optional content_type, message, payload_from.
egress.smtp.call¶
Path: /call. First-class: to_emails, subject, body / body_from, is_html, attachments, …
egress.imap.call¶
Path: /call. First-class: action (peek | list | read), mailbox, filter, limit, optional message_id, include_body, include_attachments, mark_seen, attachment_max_bytes.
action |
Provider behaviour |
|---|---|
peek |
Headers of matches without setting \Seen |
list |
Compact summary list |
read |
Full message(s); honour include_* / mark_seen |
See Egress IMAP. Legacy JSON payload remains a compose fallback.
egress.rabbitmq.publish¶
Path: /publish. Generic publish (exchange/queue routing in payload/connection).
Transforms (transformers.*)¶
| Capability | Path |
|---|---|
transformers.apply_xslt |
/applyXSLT |
transformers.xml2json |
/xml2json |
transformers.json2xml |
/json2xml |
transformers.apply_liquid |
/applyLiquid |
transformers.csv_to_json |
/csv2json |
transformers.json_to_csv |
/json2csv |
transformers.xml_to_csv |
/xml2csv |
transformers.csv_to_xml |
/csv2xml |
transformers.flatfile_to_xml |
/flatfile2xml |
transformers.xml_to_flatfile |
/xml2flatfile |
transformers.validate_json_schema |
/validateJsonSchema |
transformers.validate_xsd |
/validateXsd |
Demo: flows/workflows/transform_testing_demo.yaml, flows/workflows/csv_transform_demo.yaml, flows/workflows/xml_csv_demo.yaml, flows/workflows/flatfile_demo.yaml, flows/workflows/json_schema_validate_demo.yaml, flows/workflows/xsd_validate_demo.yaml, flows/workflows/soap_http_xslt_demo.yaml.
Object storage (egress.s3.*)¶
| Capability | Path |
|---|---|
egress.s3.put |
/put |
egress.s3.get |
/get |
egress.s3.list |
/list |
egress.s3.delete |
/delete |
There are no native storage_read / storage_write steps. Demo: flows/workflows/storage_minio_demo.yaml.
Other frequent capabilities¶
| Capability | Path (typical) |
|---|---|
ftp.call |
/ftp |
egress.ssh.exec |
/exec |
egress.ssh.sftp |
/sftp |
sftp.list / download / upload |
/sftp/list, /sftp/download, /sftp/upload |
egress.imap.call |
/call |
egress.smtp.call |
/call |
egress.google_workspace.* |
e.g. /gmail/list |
variables.string_concat / number_add / … |
/variables/... |
Naming note: not every capability uses the egress. prefix.
context_set¶
Required:
- id
- type: context_set
- context_key
- exactly one of value or value_from
Optional:
- retry_policy
- shared fields
Validation rules:
- Fails validation if both value and value_from are set, or both missing.
milestone¶
Required:
- id
- type: milestone
- exactly one of value or value_from
Optional:
- retry_policy
- shared fields
Writes context.milestone and context.milestone_at (UTC). Operations
list-API rolls the latest written value in the related-run tree onto the
runtime-root row.
Validation rules:
- Fails validation if both value and value_from are set, or both missing.
context_increment¶
Required:
- id
- type: context_increment
- context_key
Optional:
- by (default 1)
- shared fields
context_bind¶
Parse JSON, XML, or YAML once and register it under an alias for path references (request:json:/field). See Bound documents.
Required:
idtype: context_bindalias
Optional:
input_fromformat(auto,json,xml,yaml)- shared fields
context_extract_json¶
Required:
- id
- type: context_extract_json
- context_key
- json_path
Optional:
- input_from
- retry_policy
- shared fields
context_extract_xml¶
Required:
- id
- type: context_extract_xml
- context_key
- xpath
Optional:
- input_from
- shared fields
json_set¶
Required:
- id
- type: json_set
- json_path
- value_from
Optional:
- input_from
- mirror_to_context
- shared fields
xml_set_text¶
Required:
- id
- type: xml_set_text
- xpath
- value_from
Optional:
- input_from
- attribute
- mirror_to_context
- shared fields
terminate¶
Required:
- id
- type: terminate
- status in: success, failed, ignored, canceled, skipped
Optional:
- reason
- http_status (range 200..599)
- shared fields
Behavior: - Stops workflow execution immediately and returns a controlled termination state.
wait¶
Required:
- id
- type: wait
- seconds (range 0..3600)
Optional: - shared fields
Behavior: - Pauses execution for the configured duration.
log¶
Required:
- id
- type: log
- message
Optional:
- values (default {})
- level (debug, info, warning, error; default info)
- tags (default [])
- into
- shared fields
Behavior:
- Writes structured log event and can store rendered message in context via into.
- Tag business-event marks a business event (Loki marker meshflows.business_event).
See Business events.
workflow_call¶
Required:
- id
- type: workflow_call
- target
Optional:
- payload
- output_key
- timeout_seconds (range 1..3600)
- shared fields
Behavior:
- Invokes another workflow as a child run.
- Applies cycle detection and maximum call depth.
- Inherits parent timeout budget, optionally capped by timeout_seconds.
- Emits trace linkage metadata (parent_run_id, child_run_id, depth/call-chain context).
collect¶
Required:
- id
- type: collect
- from_steps (at least one entry)
Optional:
- merge_mode (object or array, default object)
- try_parse_json (default true)
- shared fields
Notes:
- from_steps entries can be:
- step_id
- {step_id: alias}
Validation rules:
- Mapping entries in from_steps must contain exactly one key.
for_each¶
Required:
- id
- type: for_each
- steps (at least one nested step)
Optional:
- input_from
- items_path (default /)
- as_key (default item)
- index_as
- max_iterations (default 100, range 1..10000)
- shared fields
Behavior: - Iterates over array items and executes nested steps per item.
repeat_until¶
Required:
- id
- type: repeat_until
- until
- steps (at least one nested step)
Optional:
- max_iterations (default 20, range 1..10000)
- shared fields
Behavior: - Repeats nested steps until condition matches or limit is reached.
if¶
Required:
- id
- type: if
- condition
- then (or then_steps)
Optional:
- else (or else_steps, default empty)
- shared fields
condition is a WhenCondition object:
| Field | Required | Description |
|---|---|---|
context_key (alias variable) |
yes | Run context key to read |
equals |
exactly one of | Expected string value |
not_equals |
these three | Must not equal |
one_of |
Value must be in list |
Behavior:
- Executes then branch when condition matches, otherwise else.
- Compares context strings only — no JSON Pointer or XPath in context_key.
- To branch on a field inside trigger JSON, XML, or service_call output, use context_extract_json / context_extract_xml first (see Conditions & Path Expressions).
try¶
Required:
- id
- type: try
- steps (at least one nested step)
Optional:
- catch (or catch_steps, default empty)
- rethrow (default false)
- error_context_key (default {step_id}.error)
- shared fields
Behavior:
- Runs steps sequentially; on failure stores the error message in context and runs catch substeps when present.
- When catch is empty, failures propagate like a normal step failure.
- When rethrow is false (default), the try step is marked successful after catch and the workflow continues.
- When rethrow is true, catch substeps still run but the original error is raised afterward (workflow fails).
- WorkflowTerminatedByStep is never caught and always stops the run immediately.
- Failed service_call steps that received an HTTP error envelope (e.g. egress fail_on_error_status) still write that JSON to outputs[<step_id>] and context:<step_id>.response, so catch can use input_from: <step_id> / extract /body / /headers / /status_code.
scope¶
Required:
- id
- type: scope
- steps (at least one nested step)
Optional:
- shared fields (when, run_after, compensate)
Behavior:
- Runs steps sequentially in the same run context as the parent (sub-workflow grouping).
- Output of the scope step is the output of the last inner step.
- Inner failures propagate; use type: try when you need catch.
- Distinct from parallel.scopes (named concurrent lanes) and from scoped_context (workflow variables).
Workflow-level on_error / finally¶
Optional lists on the workflow document (not a step type):
steps:
- id: main
type: log
message: "work"
on_error:
- id: handle
type: log
message: "failed: {err}"
values:
err: context:workflow.error
finally:
- id: always
type: log
message: "cleanup"
on_errorruns after the mainrun_aftergraph if a step failure is still pending. Setscontext:workflow.error.on_error_rethrow(defaulttrue): raise the original error after the handler.falseswallows so the run succeeds.finallyalways runs after main (andon_error), before return or re-raise.- Distinct from
try/catch(local recovery on one step) and fromrun_after(sibling graph).
parallel¶
Required:
- id
- type: parallel
- scopes (at least one named scope; each scope has nested steps)
Optional:
- max_concurrency (default 10, range 1..100)
- outputs_map (auto-generated from scope names if omitted)
- fail_fast (default true)
- shared fields
Validation rules:
- Flat steps: on parallel is not supported — use scopes: only.
- Do not set both scopes and legacy steps.
- outputs_map values must be scope:<name> or step:<id> targeting scopes/steps that exist.
Contract Tips¶
- Prefer connection-based
service_callsteps over inline credentials. - Keep step IDs stable; IDs are used by references (
payload_from,run_after,collect,input_from). - Use
workflow_callfor composition, but avoid deep nesting unless required. - Validate workflows locally with:
python engine/scripts/validate-workflow.py flows/workflows/<your-file>.yaml
Or via orchestrator: POST /admin/validate?strict=true.
Related Pages¶
- Workflow YAML syntax: Workflow YAML
- Step type overview: Step Types
- Error handling and compensation: Error Handling
- Capability provider contract: Contract 17