Delivery Semantics

LyftData inputs, outputs, actions, and transports expose component-specific controls for batching, retry, timeout, acknowledgement, and state. There is no single delivery guarantee that applies to every component and topology.

Applies to

Output delivery, batching, and retry controls are beta capabilities across Community and Enterprise single-node and distributed-worker deployments. Exact controls and guarantees remain component- and topology-specific.

Caution

LyftData does not make a universal exactly-once delivery claim or a universal persistent-queue-per-edge claim. Verify the selected component, source, destination, and recovery path against your loss, duplication, ordering, and replay requirements.

Questions to answer before deployment

For every important handoff, record:

1. What does the source consider acknowledged?
2. When does LyftData advance its cursor or checkpoint?
3. Which failures are retried, how often, and with what backoff?
4. What timeout stops one attempt?
5. Can a partial batch be accepted?
6. Can retries create duplicates?
7. Is ordering required, and at what scope?
8. Where is unacknowledged data held, and is that storage durable?
9. How is replay started and bounded?
10. What evidence proves the destination accepted the result?

The generated DSL page for the selected input or output is the authority for its fields. This guide supplies the reasoning model; it does not replace those component tables.

Batching

Batching trades fewer destination calls for more data held per attempt. Component fields can control batch size, flush conditions, request wrapping, or write mode. Larger batches can improve throughput while increasing latency, memory use, duplicate scope, and the impact of a partial failure.

Test with representative payloads and a deliberately failing destination. Confirm whether the component retries the whole batch, reports individual failures, or stops the job.

Retries and timeouts

A retry repeats an attempt after a failure the component classifies as retryable. It does not guarantee eventual success. A timeout bounds an attempt; it does not prove the destination did no work before the connection ended.

Use bounded retries and explicit timeouts. For non-idempotent destinations, assume a timeout or lost response can leave an ambiguous result: the destination may have accepted the operation even though LyftData did not receive the acknowledgement.

Where the destination supports idempotency keys, derive a stable key from the business operation and preserve it across retries. Confirm the destination’s scope and expiry rules; an idempotency key is only as strong as the service that enforces it.

Acknowledgement, loss, and duplication

At-least-once behavior prioritizes retry and can create duplicates. At-most-once behavior avoids retrying an ambiguous attempt and can lose an operation. Exactly-once requires coordinated source, processing, and destination semantics and must be proven for the exact path.

Do not infer exactly-once from:

• a successful job status;
• an output retry setting;
• a checkpoint or cursor;
• a checksum;
• a workflow checkpoint; or
• one destination receipt without source-side correlation.

Use destination-side idempotency, deduplication keys, transactional APIs, or a reconciliation job when duplicates are unacceptable.

Ordering

Ordering can change when work is batched, retried, parallelized, fanned out, or placed on multiple workers. A component that preserves order in one process does not automatically preserve global order across workers or destination partitions.

Define the required ordering scope: per key, partition, file, job instance, worker, or the whole stream. Then configure concurrency and partitioning to that scope and test recovery after a mid-stream failure.

Cursors, runtime artifacts, and replay

Some inputs persist a cursor or next-request value so later runs can continue. The storage can be component-specific; for example, an HTTP pagination cursor can use runtime artifacts, while another component can expose a different checkpoint field.

Before resetting state:

• identify the exact worker, job, state, and resource;
• capture the current value securely;
• decide the replay start and end;
• account for duplicate destination writes; and
• pause other producers that could advance the same state.

See Runtime Evidence and Receipts before clearing a runtime artifact.

Transport choices

Transport	Use	Reliability boundary
Worker channel	Low-latency composition between jobs on the runtime substrate	Not equivalent to an external durable broker; cross-worker behavior needs topology-specific validation
File-store or object-store landing zone	Durable stage boundary, replay, or blast-radius isolation	Durability still depends on storage, retention, naming, acknowledgement, and reader behavior
External broker	Broker-backed buffering and partitioning	Guarantees depend on the broker component and its acknowledgement/offset configuration
Message edge	Trigger and coordination signals	Not a bulk-data transport

Choose a durable landing zone when losing an in-memory handoff during worker loss is unacceptable. Use an external broker when its partitioning, retention, and consumer-group behavior is part of the requirement.

Workflow and trigger boundaries

A workflow composes jobs; it does not replace the delivery semantics of their inputs, outputs, and transports. A workflow checkpoint controls supported waiting work but is not a source cursor or distributed transaction.

Trigger invocation has its own bounded behavior:

• an idempotency key deduplicates supported repeated invocation requests;
• dispatch acceptance proves message dispatch, not workflow completion;
• a synchronous caller response requires the typed synchronous-response transport and a terminal response-complete workflow output; and
• callback delivery is best effort with its own retry policy.

See Triggers and Workflow Checkpoints for those contracts.

Validate the path

Test the whole handoff with:

• a normal event and a representative batch;
• a destination timeout after it receives a request;
• a retryable and a non-retryable response;
• a worker restart before and after acknowledgement;
• a duplicate event;
• an out-of-order event; and
• a bounded replay from saved state.

Record source position, job and worker identities, attempts, destination identifiers, and the final receipt. A throughput-only test does not establish recovery or delivery correctness.