Skip to content

Job Inputs

Inputs determine how events enter a job. Every job selects exactly one input and can optionally attach a trigger to control cadence. The visual editor exposes the most common settings; the DSL reference covers the full schema for advanced scenarios.

Input Trigger style Common uses Notes
azure-blob Scheduled poll Ingest blobs from Azure storage accounts. Supports wildcard object selection and incremental checkpoints.
echo Manual or triggerless Seed jobs with inline sample data. Handy for quick tests and template jobs.
exec Scheduled poll Run a command and capture STDOUT as events. Ship logs from legacy tools without building wrappers.
file-store Scheduled poll Read from managed FileStore buckets. Best for hybrid deployments that mirror data into the control plane.
files Continuous Tail directories or batch uploaded files. Detects new files and streams them line by line.
gcs Scheduled poll Pull objects from Google Cloud Storage. Shares batching semantics with S3 and Azure connectors.
http-poll Scheduled poll Call REST APIs on a cadence. Configure headers, query params, and request bodies; supports retries and timeouts.
http-server Event-driven Accept inbound webhooks. Runs an embedded HTTP listener on the worker.
internal-messages Event-driven React to platform events or job-emitted messages. Filter by kind, source, type, job, or tag.
s3 Scheduled poll Read from Amazon S3 buckets. Handles compressed objects and streams large files.
windows-event-log Event-driven Capture Windows system events. Available on Windows workers only.
worker-channel Event-driven Chain jobs together inside a worker. Consumes messages emitted by upstream jobs in the same worker.

If you do not see a specific connector in the UI, check your edition and licensing; some inputs are only available with certain licenses.

Inputs that poll external systems expose a Trigger block. Choose between cron expressions, fixed intervals, or internal message triggers. The scheduler guarantees that each job run receives a distinct window, and you can offset or jitter schedules to avoid thundering herds. Continuous inputs such as files, http-server, and worker-channel ignore the trigger block. They stay active as long as the job runtime is running.

Most inputs share two common switches:

  • JSON: Treat payloads as JSON documents so fields are available without extra parsing. Leave disabled when ingesting free-form text; the runtime will wrap the payload in _raw instead.
  • Ignore line breaks: Combine the entire payload into a single event. Enable when a single response spans multiple lines (for example, pretty-printed JSON from an HTTP API).

Document-oriented connectors (exec, http-poll, the object stores) group the events produced during one fetch into a document. Downstream actions can reference document metadata or preserve the grouping by using output batching in Document mode.

Inputs that reach out to external services include a retry block. Use:

  • retry.timeout: per-attempt timeout (defaults to 30s).
  • retry.retries: number of retry attempts (omit for unlimited).

Most connectors apply a bounded backoff internally between attempts; keep retries low to avoid overwhelming an unhealthy upstream system.

For field-level schemas and advanced options, consult the DSL index and open the linked input pages.