Author Jobs with MCP

This runbook uses an assistant to author a job without collapsing discovery, preview, execution, persistence, and deployment into one opaque action.

Applies to LyftData 3.0.0-beta.5

The named tools are present in the beta.5 MCP inventory. Bounded tests and lifecycle mutations require --max-tool-tier write; preview and discovery are available at read_only.

Prepare a bounded task

Use a non-production profile. Choose a finite synthetic sample, a run-owned job name, the intended worker, and a cleanup rule. Do not provide raw credentials or customer data in the prompt.

Ask the assistant to repeat the session URL, tenant, environment, identity, maximum tool tier, proposed writes, and evidence target before proceeding.

1. Discover the supported shape

   Use workflow_recommend for the desired outcome, tool_inventory and tool_get for current schemas, and DSL/spec discovery tools for exact component fields. If starting from a file, use job_auto_detect_file; do not paste a large or sensitive file into the conversation.

   Expected output: a proposed job shape, assumptions, unsupported fields, and the exact next preview tool. No server state should change.

2. Preview and lint the job

   Call job_yaml_preview, then inspect the YAML yourself. Run the available lint/schema tool and resolve every error. Confirm source bounds, output target, credential references, schedule, and redaction behavior.

   Preview and lint are authoring evidence only. They do not prove runtime connectivity, throughput, or delivery.

3. Run a bounded transient test

   Switch to a separately named write-tier MCP session and call job_test_run only after reviewing its live schema. Set finite event and time limits, use the synthetic sample, and cancel any run that exceeds the stated bound.

   Caution

   A transient test is execution. An input may read and an output may write. Substitute a local evidence sink unless an external side effect was explicitly approved.

4. Persist the reviewed definition

   Call job_create with the exact YAML that passed review. Record the returned job identity and compare its rendered form with the preview. A saved job is editable authoring state, not a staged or running artifact.

5. Stage and deploy separately

   Call job_stage to create the deployable version, then review the returned identity. Call job_deploy only for the named worker after checking its authorization, connectivity, capability, credentials, and current workload. Record each write separately.

6. Inspect the lifecycle and evidence

   Use job_lifecycle_snapshot_get, worker/job status, recent notifications, logs, metrics, and traces. Correlate evidence with the recorded job, deployment, worker, and time window.

   A worker heartbeat proves control-plane connectivity. A running job proves more, but still does not prove that an external destination accepted the expected records. Use provider readback or a destination receipt for that claim.

7. Clean up the run-owned work

   Stop any transient run, undeploy only the recorded deployment, and remove run-owned authoring state only when the available cleanup tool and its tier have been reviewed. Never delete by a broad prefix. Re-list the namespace and verify zero run-owned residue.

A prompt that preserves the gates

Work against tenant example, environment staging, using the synthetic file at the approved path. First report session scope and discover the current tool schemas. Recommend and preview a finite job, but stop before execution. List every assumption and proposed write. After I approve the YAML, run a bounded transient test and stop again. Do not create, stage, or deploy until each action receives separate approval. Record immutable identities and distinguish worker evidence from destination acceptance.

Failure and recovery

• Missing mutation tools: confirm this is the intended session, then start a separate --max-tool-tier write process. Do not widen a production reader.
• Permission denied: fix server RBAC or use the correct identity; a local tier cannot grant permission.
• Test inventory stuck: inspect current transient runs and cancel the exact run before starting another.
• Worker online but deployment unhealthy: inspect serviceability, credentials, connector capability, placement, and logs—not just heartbeat.
• External count differs: preserve run evidence, stop, and reconcile input, output, quarantine, retry, and duplicate counts before retrying in a fresh namespace.

Continue with Author and Deploy Workflows when the outcome requires multiple jobs, branching, or controlled placement.