Storage Service

Storage provides both runtime key-value operations and internal configuration persistence.

Base URL examples:

• Local: http://localhost:8086
• Cluster: http://storage:8086

Capability Areas

• Runtime key-value storage for workflow data.
• Internal storage for workflows, connections, policies, and schedules.
• Runtime config hierarchy (global, group, workflow scopes).
• Secret upload/download and secret value retrieval.

Endpoint Groups

Health

• GET /healthz
• GET /readyz

Runtime key-value API

• GET /v1/storage/{bucket}/{key}
• PUT /v1/storage/{bucket}/{key}
• GET /v1/storage
• GET /v1/storage/{bucket}

Internal artifact APIs

• /internal/workflows*
• /internal/connections*
• /internal/policies*
• /internal/schedules*

Internal runtime config APIs

• /internal/config/global*
• /internal/config/groups*
• /internal/config/workflows*
• /internal/config/resolved

Internal secret APIs

• /internal/upload/secrets
• /internal/download/secrets
• /internal/secrets*

Internal upload APIs (single-item)

All upload endpoints require the runtime-admin token.

• POST /internal/upload/workflows/{name} (YAML body)
• POST /internal/upload/connections/{name} (YAML body)
• POST /internal/upload/policies/{name} (YAML body)
• POST /internal/upload/schedules/{name} (YAML body)
• POST /internal/upload/config (YAML body, scoped runtime settings)

After uploading a workflow, trigger a reload in the orchestrator to pick up the new definition (POST /admin/reload).

Usage Notes

• Internal endpoints are intended for trusted services and operational tooling.
• Runtime config resolution is used by Orchestrator for merged effective config.
• Secret data should only be accessed by least-privileged service identities.

Startup Seeding

Storage supports file-based startup seeding, so workflows/connections/policies/schedules/secrets do not need to be hardcoded in service code.

• STORAGE_RUNTIME_SEED_PATH: absolute path to a YAML/JSON seed file.
• STORAGE_RUNTIME_SEED_OVERWRITE: true/false (default false).

Supported top-level seed sections:

• workflows
• connections
• policies
• schedules
• secrets

Each section can be either:

• A list of documents (each document requires name), or
• A mapping keyed by name (name is inferred from the key).

Example seed file: engine/config/runtime.seed.yaml.example.

Runtime config scope seeding remains supported via:

• STORAGE_CONFIG_SEED_PATH
• STORAGE_CONFIG_SEED_OVERWRITE

Operational Guidance

• Keep backups/snapshots for persistent storage data.
• Restrict network access to internal routes.
• Monitor latency and error rates on key-value operations.
• Use explicit bucket/key naming conventions per domain.