Manual Kubernetes Deployment (Without CI Pipelines)¶

Use this guide when you deploy MeshFlows Engine to your own minikube or Kubernetes cluster and cannot rely on the Forgejo/Woodpecker build and deploy pipelines.

The repository already contains the manifests, helper scripts, and image mappings you need. This page describes how to bootstrap from scratch and how to roll forward when new container packages are published.

When to Use This Guide¶

You run a local lab on minikube and want a repeatable setup without a CI runner.
You operate your own cluster (kind, k3s, AKS, EKS, bare metal, …) and pull images from the public MeshFlows container packages on Forgejo.
You build images yourself and push them to a private registry.
You want to understand what the automated pipeline does, so you can reproduce it manually.

What the CI Pipeline Normally Does¶

The automated pipeline (documented in CI/CD Pipeline) roughly performs:

Build changed engine services and the wiki, tag them (for example v2026.07.02.42), and push to code.meshflows.org/<owner>/meshflows/<service>.
Deploy Kubernetes manifests from engine/deploy/.
Update running deployments to the new image tag via deploy_engine_images.py.
Sync workflow artifacts from flows/ into the shared PVC.
Reload the orchestrator so new workflows and settings are picked up.

Everything below is the manual equivalent of those steps.

Published Container Images¶

Engine images are published as OCI containers under:

code.meshflows.org/meshflowsgpl/meshflows/<service>:<tag>

Examples:

Service	Image
Gateway	`code.meshflows.org/meshflowsgpl/meshflows/gateway`
Orchestrator	`code.meshflows.org/meshflowsgpl/meshflows/orchestrator`
Storage	`code.meshflows.org/meshflowsgpl/meshflows/storage`
Wiki	`code.meshflows.org/meshflowsgpl/meshflows/wiki`

The canonical mapping between build names, image names, and Kubernetes deployments lives in engine/deploy/engine-images-manifest.yaml.

Browse available packages and tags on Forgejo: MeshFlowsGPL/meshflows packages.

Image tags

CI publishes immutable version tags such as v2026.07.02.<run-number> (and environment suffixes like -dev on the develop branch). The linux-cluster overlay defaults to latest for convenience. For production-like environments, pin an explicit version tag instead of latest.

Choose Your Deployment Path¶

graph TD
    A[Target cluster] --> B{Image source?}
    B -->|Local only| C[Minikube overlay<br/>build in minikube Docker]
    B -->|Pull from Codeberg| D[Base k8s manifests<br/>+ Codeberg image refs]
    B -->|Private registry| E[Build locally<br/>push to your registry]
    C --> F[kubectl apply -k overlays/minikube]
    D --> G[kubectl apply -k custom overlay]
    E --> G

Path	Best for	Registry needed?
A — Minikube (local build)	Laptop / dev machine	No — images are built inside the minikube Docker daemon
B — Pull from Codeberg	Own cluster with internet access	No (public pull) or login for private forks
C — Private registry	Air-gapped or corporate registry	Yes

Path A — Minikube (Recommended for Local Labs)¶

This is the fastest way to get a working stack without any registry or CI.

Prerequisites¶

minikube
kubectl
Docker (or another driver supported by minikube)

One-command bootstrap (Windows)¶

From the repository root:

.\deploy-minikube.ps1

Optional flags:

-SkipBuild — reuse images already in the minikube Docker daemon
-SkipStart — minikube profile is already running
-Tunnel — open port-forward tunnels to dashboard, orchestrator, and RabbitMQ UI

What the script does¶

Starts minikube profile meshflows.
Points your Docker client at the minikube daemon (minikube docker-env).
Builds all engine service images as meshflows/<service>:latest.
Creates namespace meshflows and the meshflows-identity-auth secret.
Installs Gateway API CRDs if missing.
Applies engine/deploy/overlays/minikube/.

The minikube overlay sets imagePullPolicy: Never for MeshFlows images and exposes NodePorts 31100 (dashboard), 31101 (orchestrator), and 31102 (RabbitMQ management UI).

Manual equivalent (Linux / macOS)¶

minikube start -p meshflows
eval "$(minikube -p meshflows docker-env)"

# Build images (example for one service; repeat for each service you need)
docker build -t meshflows/gateway:latest engine/services/gateway

kubectl create namespace meshflows --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret generic meshflows-identity-auth \
  -n meshflows \
  --from-literal=jwt_secret=dev-local-minikube-secret-key \
  --from-literal=bootstrap_admin_user=admin \
  --from-literal=bootstrap_admin_password=admin \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl apply -k engine/deploy/overlays/minikube --load-restrictor=LoadRestrictionsNone

Sync workflow data (minikube)¶

cd engine
./scripts/sync-flows-pv.sh meshflows

Then reload the orchestrator (see Reload after changes below).

Path B — Own Kubernetes Cluster (Pull from Codeberg)¶

Use this when your cluster can pull container images from Codeberg and you do not want to build locally.

Prerequisites¶

Kubernetes 1.24+ with a default StorageClass for PVCs
kubectl configured for your cluster
Gateway API CRDs (install if not present):

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml

1. Create namespace and secrets¶

Pick a namespace (example: meshflows):

export NAMESPACE=meshflows
kubectl create namespace "$NAMESPACE"

Create the identity bootstrap secret (replace values):

kubectl create secret generic meshflows-identity-auth \
  -n "$NAMESPACE" \
  --from-literal=jwt_secret="<min-32-char-secret>" \
  --from-literal=bootstrap_admin_user="admin" \
  --from-literal=bootstrap_admin_password="<strong-password>"

Create the orchestrator reload token:

kubectl create secret generic meshflows-orchestrator-auth \
  -n "$NAMESPACE" \
  --from-literal=reload_token="$(openssl rand -hex 24)"

If your cluster requires authentication to pull from Codeberg, create an image pull secret and reference it in your overlay or namespace default.

2. Apply the standalone overlay¶

Use engine/deploy/overlays/standalone-cluster/. It extends the full base stack (engine/deploy/k8s/) and maps every service to code.meshflows.org/meshflowsgpl/meshflows/*.

To pin a specific version, edit newTag in that overlay (default: latest).

kubectl apply -k engine/deploy/overlays/standalone-cluster

This deploys infrastructure (RabbitMQ, Postgres, MinIO, observability) and all microservices in namespace meshflows — without lab-specific extras (Forgejo, webmail, Longhorn, central Postgres).

Alternative — lab overlay: linux-cluster/components targets namespace meshflows-lab with additional patches (central Postgres, MFA, SMTP). Use that only if you mirror the full lab topology.

4. Wait for rollouts¶

kubectl get pods -n "$NAMESPACE" -w
kubectl rollout status deployment/orchestrator -n "$NAMESPACE"
kubectl rollout status deployment/gateway -n "$NAMESPACE"

5. Sync flows and reload¶

cd engine
./scripts/sync-flows-pv.sh "$NAMESPACE"

See Deploying Workflows for artifact details.

Path C — Build Images Yourself¶

Use this when you modify engine code, run air-gapped, or mirror images to a corporate registry.

Build and tag¶

Each service has a Dockerfile under engine/services/<name>/. Build and push using the same naming convention as CI:

export REGISTRY=code.meshflows.org          # or your private registry
export OWNER=meshflowsgpl                  # your org/user
export TAG=v2026.07.02.local1

docker build -t "$REGISTRY/$OWNER/meshflows/gateway:$TAG" engine/services/gateway
docker push "$REGISTRY/$OWNER/meshflows/gateway:$TAG"
# repeat for each changed service

The full service list matches deploy-minikube.ps1 and engine/deploy/engine-images-manifest.yaml.

Deploy¶

Point your kustomize overlay images: entries at your registry and tag, then kubectl apply -k … as in Path B.

Updating to New Package Versions¶

When new containers are published on Forgejo packages, choose one of these update strategies.

Strategy 1 — `deploy_engine_images.py` (closest to CI)¶

After manifests are applied at least once, use the same script the deploy workflow uses:

cd engine/deploy/scripts

python3 deploy_engine_images.py \
  --namespace meshflows \
  --image-tag v2026.07.02.42 \
  --registry-owner meshflowsgpl \
  --registry-host code.meshflows.org \
  --deploy-services all

Deploy only specific services:

python3 deploy_engine_images.py \
  --namespace meshflows \
  --image-tag v2026.07.02.42 \
  --registry-owner meshflowsgpl \
  --deploy-services gateway,orchestrator \
  --selective-deploy

The script:

Runs kubectl set image on each deployment
Sets APP_VERSION and APP_BUILD_TIMESTAMP env vars
Updates the engine-images-lock ConfigMap with the deployed versions

Strategy 2 — Kustomize tag bump + re-apply¶

Change newTag in your overlay kustomization.yaml.
Re-apply:

kubectl apply -k engine/deploy/overlays/standalone-cluster

Watch rollouts:

kubectl rollout status deployment -n meshflows --timeout=300s

Strategy 3 — `kubectl set image` (quick one-off)¶

kubectl set image deployment/gateway \
  gateway=code.meshflows.org/meshflowsgpl/meshflows/gateway:v2026.07.02.42 \
  -n meshflows

Repeat per deployment. Refer to engine/deploy/engine-images-manifest.yaml for deployment and container names.

Strategy 4 — Minikube local rebuild¶

.\deploy-minikube.ps1 -SkipStart   # rebuilds images and re-applies manifests

After an engine image update¶

Sync flows if flows/ changed in the release you are deploying.
Reload the orchestrator (below).
Run health checks from Operations & Monitoring.

Updating the wiki container¶

The wiki is deployed separately from wiki/deploy/k8s/:

# Example: patch the wiki deployment image
kubectl set image deployment/wiki-main \
  wiki=code.meshflows.org/meshflowsgpl/meshflows/wiki:v2026.07.02.42 \
  -n meshflows

Adjust deployment name and namespace to match your overlay.

Reload After Changes¶

When workflow artifacts, connections, policies, or schedules change, sync the PVC and reload:

cd engine
./scripts/sync-flows-pv.sh meshflows

RELOAD_TOKEN=$(kubectl get secret meshflows-orchestrator-auth \
  -n meshflows -o jsonpath='{.data.reload_token}' | base64 -d)

kubectl exec -n meshflows deploy/orchestrator -- \
  curl -s -X POST http://localhost:8080/admin/reload \
  -H "X-Reload-Token: ${RELOAD_TOKEN}"

What This Guide Does Not Cover¶

The linux-cluster overlay used by the lab pipeline includes optional platform extras that are not required for a minimal MeshFlows runtime:

Component	Location	Needed for basic runtime?
Forgejo (`code.meshflows.org`)	`overlays/linux-cluster/infra/forgejo-*`	No
Roundcube webmail	deploy workflow + Helm	No
Longhorn storage / backups	`overlays/linux-cluster/infra-longhorn/`	No — use your cluster default `StorageClass`
Central Postgres (`platform-data`)	`overlays/linux-cluster/platform-data/`	No — base manifests include in-namespace Postgres
Lab TLS hostnames / Gateway patches	`overlays/linux-cluster/infra/`	Optional — adjust hostnames for your DNS

For constrained clusters, also see Small Cluster Checklist.

Verification Checklist¶

After bootstrap or an upgrade:

kubectl get pods -n meshflows
kubectl rollout status deployment/orchestrator -n meshflows
kubectl rollout status deployment/gateway -n meshflows

From inside the cluster or via port-forward:

curl -s http://orchestrator:8080/readyz
curl -s http://gateway:8080/v1/status

Confirm:

All pods are Running / Ready.
Orchestrator readyz reports loaded workflows.
RabbitMQ and storage pods are healthy.
A test workflow invocation succeeds and appears in traces.

Rollback¶

Revert a single deployment:

kubectl rollout undo deployment/orchestrator -n meshflows
kubectl rollout undo deployment/gateway -n meshflows

Or redeploy a known-good image tag with deploy_engine_images.py or kubectl set image.

Kubernetes Deployment — high-level cluster deploy overview
CI/CD Pipeline — what the automated pipeline does
Secrets Management
Deploying Workflows
Operations & Monitoring
Forgejo container packages

Manual Kubernetes Deployment (Without CI Pipelines)¶

When to Use This Guide¶

What the CI Pipeline Normally Does¶

Published Container Images¶

Choose Your Deployment Path¶

Path A — Minikube (Recommended for Local Labs)¶

Prerequisites¶

One-command bootstrap (Windows)¶

What the script does¶

Manual equivalent (Linux / macOS)¶

Sync workflow data (minikube)¶

Path B — Own Kubernetes Cluster (Pull from Codeberg)¶

Prerequisites¶

1. Create namespace and secrets¶

2. Apply the standalone overlay¶

4. Wait for rollouts¶

5. Sync flows and reload¶

Path C — Build Images Yourself¶

Build and tag¶

Deploy¶

Updating to New Package Versions¶

Strategy 1 — deploy_engine_images.py (closest to CI)¶

Strategy 2 — Kustomize tag bump + re-apply¶

Strategy 3 — kubectl set image (quick one-off)¶

Strategy 4 — Minikube local rebuild¶

After an engine image update¶

Updating the wiki container¶

Reload After Changes¶

What This Guide Does Not Cover¶

Verification Checklist¶

Rollback¶

Related Documentation¶

Strategy 1 — `deploy_engine_images.py` (closest to CI)¶

Strategy 3 — `kubectl set image` (quick one-off)¶