LyftData Events
LyftData Events (lyftdata-events)
Section titled “LyftData Events (lyftdata-events)”Cursor-paged input over LyftData’s indexed logs/events store.
Analytics json
Minimal example
Section titled “Minimal example”input: lyftdata-events: api-key: ~ server-url: ~JSON
{ "input": { "lyftdata-events": { "api-key": null, "server-url": null } }}Contents
Section titled “Contents”A ✓ marks a field that accepts a context variable such as
{{ VARIABLE }}.
Filters
Section titled “Filters”Filters
| Field | Type | Required | Description |
|---|---|---|---|
record-types |
string[] |
Logs/events record types to read; staging validation requires at least one entry. | |
source-jobs |
string[] |
Optional originating job-name filters. | |
source-workers |
string[] |
Optional originating worker-id filters. |
Paging
Section titled “Paging”Paging
| Field | Type | Required | Description |
|---|---|---|---|
page-size |
number (integer) |
Maximum rows fetched per poll. Bounded to protect the server. Examples: 42, 1.2e-10 |
|
state-id |
string |
Optional explicit runtime-artifact state id for cursor persistence. |
Retry
| Field | Type | Required | Description |
|---|---|---|---|
retry |
Retry |
Retry policy for transient server API failures. |
Server
Section titled “Server”Server
| Field | Type | Required | Description |
|---|---|---|---|
server-url ✓ |
string |
✅ | Base URL for the LyftData server API, such as https://lyftdata.example.com. |
api-key ✓ |
string |
✅ | DB Explorer service API key used as the bearer token. |
auth-header |
string |
Header used to carry the API key. | |
auth-prefix |
string |
Prefix prepended to the API key in auth-header. |
|
insecure ✓ |
boolean (bool) |
Disable TLS certificate validation for local development servers. Default: false |
Trigger
Section titled “Trigger”Trigger
| Field | Type | Required | Description |
|---|---|---|---|
trigger |
Trigger |
When to poll for the next page of indexed events. Allowed values: message, cron, interval |
Schema
Section titled “Schema”- Trigger - Cron - Window - Start Options
- Trigger - Interval - Window - Start Options
- Trigger Options
- Retry - Backoff Fields
- Retry Fields
- Trigger - Message Fields
- Trigger - Cron - Window - Start - Start Time Fields
- Trigger - Cron - Window Fields
- Trigger - Cron Fields
- Trigger - Interval - Window - Start - Start Time Fields
- Trigger - Interval - Window Fields
- Trigger - Interval Fields
- Retry - Backoff - Strategy Options
- Trigger - Message - Filter Kind Options
- Trigger - Message - Filter Source Options
- Trigger - Message - Filter Type Options
Trigger - Cron - Window - Start Options
Section titled “Trigger - Cron - Window - Start Options”| Option | Name | Type | Description |
|---|---|---|---|
start-time |
Start Time | object |
|
tracked |
Tracked | string |
Examples: /path/to/file, c:\users\joe\data\file.txt |
Trigger - Interval - Window - Start Options
Section titled “Trigger - Interval - Window - Start Options”| Option | Name | Type | Description |
|---|---|---|---|
start-time |
Start Time | object |
|
tracked |
Tracked | string |
Examples: /path/to/file, c:\users\joe\data\file.txt |
Trigger Options
Section titled “Trigger Options”| Option | Name | Type | Description |
|---|---|---|---|
message |
Message | object |
|
cron |
Cron | object |
|
interval |
Interval | object |
Retry - Backoff Fields
Section titled “Retry - Backoff Fields”| Field | Type | Required | Description |
|---|---|---|---|
strategy |
Strategy |
Backoff strategy to use (default exponential). Allowed values: exponential, linear, fixed |
|
base |
string |
Base delay before retrying (e.g. “200ms”). | |
max |
string |
Maximum delay between retries. | |
jitter ✓ |
boolean (bool) |
Whether to add jitter to retry delays. Default: false |
Retry Fields
Section titled “Retry Fields”| Field | Type | Required | Description |
|---|---|---|---|
max-attempts |
number (integer) |
Maximum attempts before giving up. Examples: 42, 1.2e-10 |
|
forever ✓ |
boolean (bool) |
Retry indefinitely until cancelled. Default: false |
|
backoff |
Backoff |
Backoff strategy configuration. |
Trigger - Message Fields
Section titled “Trigger - Message Fields”| Field | Type | Required | Description |
|---|---|---|---|
limit ✓ |
number (integer) |
The number of times to run the input. Examples: 42, 1.2e-10 |
|
filter-kind |
Filter Kind |
Specifies whether the message originated from the “system” or by the “user”. Allowed values: system, user, runtime-artifact-fetch, runtime-artifact-fetch-error, runtime-artifact-clear, runtime-artifact-clear-ack, runtime-artifact-fetch-reply, runtime-artifact-update, runtime-artifact-update-ack |
|
filter-source |
Filter Source[] |
Specifies what process generated the message. Was it a “server”, “worker” or “job”? Allowed values: job, worker, server |
|
filter-worker |
string |
Specifies what worker to select. | |
filter-job |
string |
Specifies the name of the job that the message came from. | |
filter-type |
Filter Type[] |
Specifies that particular types of message ought to match. Allowed values: worker-licensed, worker-unlicensed, variable, variable-deleted, begin-shutting-down-job, begin-shutting-down-server, begin-shutting-down-worker, broadcast-job-thread-state, broadcast-server-thread-state, broadcast-worker-thread-state, … |
|
filter-tag |
string |
Specifies that messages matched ought to carry a tag with a particular value. This only matches against user-generated messages. |
Trigger - Cron - Window - Start - Start Time Fields
Section titled “Trigger - Cron - Window - Start - Start Time Fields”| Field | Type | Required | Description |
|---|---|---|---|
start-time |
time-format (string) |
✅ | Allows the windowing to start at a specified time. Hint: %Y-%m-%d %H:%M:%S%.3f %zIt should in the following format: 2019-07-10 18:45:00.000 +0200 |
highwatermark-file |
path (string) |
✅ | Specify file where timestamp would be stored in order to resume, for when Job has been restarted. Examples: /path/to/file, c:\users\joe\data\file.txt |
Trigger - Cron - Window Fields
Section titled “Trigger - Cron - Window Fields”| Field | Type | Required | Description |
|---|---|---|---|
size |
duration (string) |
✅ | Window size. |
offset |
duration (string) |
Window offset. Default: 0s |
|
start |
Start |
Specify file where timestamp would be stored in order to resume, for when Job has been restarted. Allowed values: start-time, tracked |
Trigger - Cron Fields
Section titled “Trigger - Cron Fields”| Field | Type | Required | Description |
|---|---|---|---|
cron |
cron-expression (string) |
✅ | The Cron pattern. |
immediate ✓ |
boolean (bool) |
Run as soon as invoked, instead of waiting for the specified cron interval. Default: false |
|
random-offset |
duration (string) |
Sets a random offset to the schedule, then sticks to it. Default: 0s |
|
window |
Window |
Optional window definition when the schedule should only read a bounded range. |
Trigger - Interval - Window - Start - Start Time Fields
Section titled “Trigger - Interval - Window - Start - Start Time Fields”| Field | Type | Required | Description |
|---|---|---|---|
start-time |
time-format (string) |
✅ | Allows the windowing to start at a specified time. Hint: %Y-%m-%d %H:%M:%S%.3f %zIt should in the following format: 2019-07-10 18:45:00.000 +0200 |
highwatermark-file |
path (string) |
✅ | Specify file where timestamp would be stored in order to resume, for when Job has been restarted. Examples: /path/to/file, c:\users\joe\data\file.txt |
Trigger - Interval - Window Fields
Section titled “Trigger - Interval - Window Fields”| Field | Type | Required | Description |
|---|---|---|---|
size |
duration (string) |
✅ | Window size. |
offset |
duration (string) |
Window offset. Default: 0s |
|
start |
Start |
Specify file where timestamp would be stored in order to resume, for when Job has been restarted. Allowed values: start-time, tracked |
Trigger - Interval Fields
Section titled “Trigger - Interval Fields”| Field | Type | Required | Description |
|---|---|---|---|
duration |
duration (string) |
✅ | Duration to wait between events. |
random-offset |
duration (string) |
Sets a random offset to the schedule, then sticks to it. Default: 0s |
|
window |
Window |
Optional window definition when the interval should only cover a bounded range. |
Retry - Backoff - Strategy Options
Section titled “Retry - Backoff - Strategy Options”| Value | Aliases | Name | Description |
|---|---|---|---|
exponential |
exponential | ||
linear |
linear | ||
fixed |
fixed |
Trigger - Message - Filter Kind Options
Section titled “Trigger - Message - Filter Kind Options”| Value | Aliases | Name | Description |
|---|---|---|---|
system |
system | ||
user |
user | ||
runtime-artifact-fetch |
runtime-artifact-fetch | ||
runtime-artifact-fetch-error |
runtime-artifact-fetch-error | ||
runtime-artifact-clear |
runtime-artifact-clear | ||
runtime-artifact-clear-ack |
runtime-artifact-clear-ack | ||
runtime-artifact-fetch-reply |
runtime-artifact-fetch-reply | ||
runtime-artifact-update |
runtime-artifact-update | ||
runtime-artifact-update-ack |
runtime-artifact-update-ack |
Trigger - Message - Filter Source Options
Section titled “Trigger - Message - Filter Source Options”| Value | Aliases | Name | Description |
|---|---|---|---|
job |
job | ||
worker |
worker | ||
server |
server |
Trigger - Message - Filter Type Options
Section titled “Trigger - Message - Filter Type Options”| Value | Aliases | Name | Description |
|---|---|---|---|
worker-licensed |
worker-licensed | ||
worker-unlicensed |
worker-unlicensed | ||
variable |
variable | ||
variable-deleted |
variable-deleted | ||
begin-shutting-down-job |
begin-shutting-down-job | ||
begin-shutting-down-server |
begin-shutting-down-server | ||
begin-shutting-down-worker |
begin-shutting-down-worker | ||
broadcast-job-thread-state |
broadcast-job-thread-state | ||
broadcast-server-thread-state |
broadcast-server-thread-state | ||
broadcast-worker-thread-state |
broadcast-worker-thread-state | ||
check-job-report-time |
check-job-report-time | ||
check-worker-report-time |
check-worker-report-time | ||
de-register-job-thread-dependency |
de-register-job-thread-dependency | ||
de-register-server-thread-dependency |
de-register-server-thread-dependency | ||
de-register-worker-thread-dependency |
de-register-worker-thread-dependency | ||
deployed-job-active |
deployed-job-active | ||
deployed-job-removed |
deployed-job-removed | ||
deployed-job-should-be-running |
deployed-job-should-be-running | ||
deployment-phase |
deployment-phase | ||
durable-transport-effect-ack |
durable-transport-effect-ack | ||
heart-beat |
heart-beat | ||
initialise-internal-state |
initialise-internal-state | ||
initialise-job-states |
initialise-job-states | ||
job-batch-end |
job-batch-end | ||
job-backlog-update |
job-backlog-update | ||
job-analytics-batch |
job-analytics-batch | ||
job-checkpoint-update |
job-checkpoint-update | ||
job-deploy-ready |
job-deploy-ready | ||
job-deployed |
job-deployed | ||
job-document-end |
job-document-end | ||
job-document-start |
job-document-start | ||
job-errors |
job-errors | ||
job-execution-anomaly |
job-execution-anomaly | ||
job-execution-status |
job-execution-status | ||
job-emit-custom |
job-emit-custom | ||
job-finished |
job-finished | ||
job-idle |
job-idle | ||
job-initiated |
job-initiated | ||
job-is-processing |
job-is-processing | ||
job-logs |
job-logs | ||
job-metrics |
job-metrics | ||
job-notifications |
job-notifications | ||
job-removing |
job-removing | ||
job-removed |
job-removed | ||
job-remove-failed |
job-remove-failed | ||
job-remove-ready |
job-remove-ready | ||
job-replaced |
job-replaced | ||
job-required |
job-required | ||
job-run-ended |
job-run-ended | ||
job-runtime-error |
job-runtime-error | ||
job-runtime-settings |
job-runtime-settings | ||
job-run-started |
job-run-started | ||
job-running-docker |
job-running-docker | ||
job-running-script |
job-running-script | ||
job-running-subprocess |
job-running-subprocess | ||
job-running-system-d |
job-running-system-d | ||
job-settings |
job-settings | ||
job-step-statistics |
job-step-statistics | ||
job-started |
job-started | ||
job-staged |
job-staged | ||
job-state-transition |
job-state-transition | ||
job-shutting-down |
job-shutting-down | ||
job-stopping |
job-stopping | ||
job-stopped |
job-stopped | ||
job-timed-out |
job-timed-out | ||
job-suspicious-silence |
job-suspicious-silence | ||
job-thread-state |
job-thread-state | ||
job-trace |
job-trace | ||
job-trace-requires-samples |
job-trace-requires-samples | ||
job-updated |
job-updated | ||
job-worker-comms-error |
job-worker-comms-error | ||
job-unstaged |
job-unstaged | ||
license-state-changed |
license-state-changed | ||
license-validation-failed |
license-validation-failed | ||
license-validation-ok |
license-validation-ok | ||
license-volume-violation |
license-volume-violation | ||
new-license |
new-license | ||
override-job-coordinated-shutdown |
override-job-coordinated-shutdown | ||
override-server-coordinated-shutdown |
override-server-coordinated-shutdown | ||
override-worker-coordinated-shutdown |
override-worker-coordinated-shutdown | ||
register-job-thread-dependency |
register-job-thread-dependency | ||
register-server-thread-dependency |
register-server-thread-dependency | ||
register-worker-thread-dependency |
register-worker-thread-dependency | ||
run-job-failure |
run-job-failure | ||
server-logs |
server-logs | ||
server-metrics-batch |
server-metrics-batch | ||
server-started |
server-started | ||
server-starting |
server-starting | ||
server-stopping |
server-stopping | ||
server-thread-state |
server-thread-state | ||
server-worker-comms-error |
server-worker-comms-error | ||
shutdown-jobs |
shutdown-jobs | ||
shutdown-worker |
shutdown-worker | ||
system-shutdown |
system-shutdown | ||
update-upstream-sync-for-job |
update-upstream-sync-for-job | ||
update-upstream-sync-for-worker |
update-upstream-sync-for-worker | ||
update-variable |
update-variable | ||
user-alert |
user-alert | ||
user-generated |
user-generated | ||
trigger-response |
trigger-response | ||
trigger-proxy-response |
trigger-proxy-response | ||
workflow-step-terminal |
workflow-step-terminal | ||
workflow-step-terminal-ack |
workflow-step-terminal-ack | ||
user-notification |
user-notification | ||
worker-command-for-job |
worker-command-for-job | ||
worker-auth-lease-ack |
worker-auth-lease-ack | ||
worker-connected |
worker-connected | ||
worker-created |
worker-created | ||
worker-debug-heart-beat |
worker-debug-heart-beat | ||
worker-error |
worker-error | ||
worker-first-seen |
worker-first-seen | ||
worker-heart-beat |
worker-heart-beat | ||
worker-logs |
worker-logs | ||
worker-metrics-batch |
worker-metrics-batch | ||
worker-offline |
worker-offline | ||
worker-requests-auth-lease |
worker-requests-auth-lease | ||
worker-server-comms-error |
worker-server-comms-error | ||
worker-settings |
worker-settings | ||
worker-shutdown |
worker-shutdown | ||
worker-shutting-down |
worker-shutting-down | ||
worker-started |
worker-started | ||
worker-state-uuid |
worker-state-uuid | ||
worker-stopping |
worker-stopping | ||
worker-suspicious-silence |
worker-suspicious-silence | ||
worker-system-information |
worker-system-information | ||
worker-thread-state |
worker-thread-state | ||
worker-updated |
worker-updated | ||
worker-modified |
worker-modified | ||
worker-removed |
worker-removed | ||
context-changed |
context-changed | ||
rerender-deployment |
rerender-deployment | ||
job-killed |
job-killed | ||
message-serviced |
message-serviced | ||
failed-to-service-message |
failed-to-service-message | ||
worker-wants-initial-settings |
worker-wants-initial-settings | ||
worker-wants-initial-settings-reply |
worker-wants-initial-settings-reply | ||
worker-wants-deployed-jobs |
worker-wants-deployed-jobs | ||
worker-wants-deployed-jobs-reply |
worker-wants-deployed-jobs-reply | ||
worker-wants-job-configuration |
worker-wants-job-configuration | ||
worker-wants-job-configuration-reply |
worker-wants-job-configuration-reply | ||
job-wants-dslir-key |
job-wants-dslir-key | ||
job-wants-dslir-key-reply |
job-wants-dslir-key-reply | ||
worker-wants-dslir-key |
worker-wants-dslir-key | ||
worker-wants-dslir-key-reply |
worker-wants-dslir-key-reply | ||
job-wants-variables |
job-wants-variables | ||
job-wants-variables-reply |
job-wants-variables-reply | ||
job-wants-credentials |
job-wants-credentials | ||
job-wants-credentials-reply |
job-wants-credentials-reply | ||
job-wants-credentials-error |
job-wants-credentials-error | ||
job-wants-secret-variables-reply |
job-wants-secret-variables-reply | ||
job-credentials-invalidated |
job-credentials-invalidated | ||
aggregator-health |
aggregator-health | ||
worker-aggregator-health |
worker-aggregator-health | ||
worker-verification-token |
worker-verification-token | ||
worker-requests-verification-token |
worker-requests-verification-token | ||
runtime-artifact-update |
runtime-artifact-update | ||
runtime-artifact-update-ack |
runtime-artifact-update-ack | ||
runtime-artifact-clear |
runtime-artifact-clear | ||
runtime-artifact-clear-ack |
runtime-artifact-clear-ack | ||
runtime-artifact-fetch |
runtime-artifact-fetch | ||
runtime-artifact-fetch-reply |
runtime-artifact-fetch-reply | ||
runtime-artifact-fetch-error |
runtime-artifact-fetch-error |