Deployments Overview

Deployments is the part of LyftData that turns workflows (graphs) into running systems across one or more workers.

Note

A workflow is a control-plane description of what should run and how it is wired. It does not run as one long-lived process; the Deployment Manager expands it into concrete jobs deployed onto workers.

If Pipelines teaches you how to author a single job (one input → actions → one output), Deployments teaches you how to:

• compose many jobs into a system,
• choose how data moves between steps (transports),
• decide where each step runs (placement),
• and safely plan + apply changes with a clear diff (“no surprises”).

When to use Deployments

Use Deployments when you need any of the following:

• A pipeline that is naturally multi-stage (ingest → route/transform → deliver).
• Fan-out (send the same events to multiple consumers) or fan-in (combine multiple sources).
• A durable hop between stages (spooling/landing before downstream processing).
• Placement and scaling: run different parts of the pipeline on different worker groups and scale them independently.
• A repeatable rollout flow with plan → apply → reconcile and an audit trail.

If you just need a single job running on one worker, you can stay in Pipelines.

The Deployments lifecycle (high level)

1. Pick a starting point from the Catalog (usually a workflow or blueprint).
2. Click Plan & register to open the “create deployment” flow.
3. Choose a name, a worker group, and (when available) fill in any wizard fields for the selected workflow.
4. Plan: preview what will be created/updated and where it will run.
5. Apply: stage and deploy the concrete jobs onto workers.
6. Operate: watch health/status, and use reconcile to converge drift back to desired state.

Where this lives in the UI

• Catalog: discover workflows, blueprints, templates, and library jobs (local or portal).
• Workflows: author drafts and publish versions.
• Deployment Manager: plan/apply/reconcile deployments.
• Workloads / Workers: see what is actually running and where.

Note

The Catalog can include locally installed items and (when licensed) Portal items. Some authoring primitives are server-shipped “builtins” that appear in the workflow editor palette rather than the Catalog.

Next pages

• Start with Catalog to understand what each catalog item type is used for.
• Read Core Concepts for the primitives (nodes, edges, channels, messages, KV state, control plane).
• Read Workflows for authoring/versioning and transports.
• Read Deployment Manager for plan/apply/reconcile, placement, and drift.
• Follow Tutorial: Quickstart File Store Roundtrip for an end-to-end walkthrough.

Note

Some deployments capabilities (especially anything involving external workers and worker groups) may require a licensed deployment. See Licensing and Community Edition.