Deployments Overview

Deployments is the part of LyftData that turns workflows (graphs) into running systems across one or more workers.

Note

A workflow is a control-plane description of what should run and how it is wired. It does not run as one long-lived process; the Deployment Manager expands it into concrete jobs deployed onto workers.

If Jobs teaches you how to author a single job (one input → actions → one output), Deployments teaches you how to:

• compose many jobs into a system,
• choose how data moves between steps (transports),
• decide where each step runs (placement),
• and safely plan + apply changes with a clear diff (“no surprises”).

When to use Deployments

Use Deployments when you need any of the following:

• A pipeline that is naturally multi-stage (ingest → route/transform → deliver).
• Fan-out (send the same events to multiple consumers) or fan-in (combine multiple sources).
• A durable hop between stages (spooling/landing before downstream processing).
• Placement and scaling: run different parts of the pipeline on different worker groups and scale them independently.
• A repeatable rollout flow with plan → apply → reconcile and an audit trail.

If you just need a single job running on one worker, you can stay in Jobs.

The Deployments lifecycle (high level)

1. Pick a starting point from the Catalog (a workflow, perspective, or blueprint).
2. If you chose a perspective, clone/start from it, complete bindings in Workflows, and publish a deployable workflow version.
3. Click Plan & register to open the “create deployment” flow.
4. Choose a name, a worker group, and fill in wizard fields when present.
5. Plan: preview what will be created/updated and where it will run.
6. Apply: stage and deploy the concrete jobs onto workers.
7. Operate: watch health/status, and use reconcile to converge drift back to desired state.

Where this lives in the UI

• Catalog: discover workflows, perspectives, blueprints, and library jobs (local or portal).
• Workflows: author drafts and publish versions.
• Deployment Manager: plan/apply/reconcile deployments.
• Workloads / Workers: see what is actually running and where.

Note

The Catalog can include locally installed items and (when licensed) Portal items. Some authoring primitives are server-shipped “builtins” that appear in the workflow editor palette rather than the Catalog.

Next pages

• Start with Catalog to understand what each catalog item type is used for.
• Read Core Concepts for the primitives (nodes, edges, channels, messages, KV state, control plane).
• Read Workflows for authoring/versioning and transports.
• Read Deployment Manager for plan/apply/reconcile, placement, and drift.
• Follow Tutorial: Quickstart File Store Roundtrip for an end-to-end walkthrough.

Note

Deployments is a licensed surface. In Community Edition, deployments APIs/UI flows are unavailable and external-worker deployment flows are blocked. See Licensing and Community Edition.