Stream

Stream (stream)

Track per-key changes and emit deltas, elapsed times, and optional aggregates.

Stateful Transform json

Minimal example

actions:
  - stream:
      watch: ""

JSON

{
  "actions": [
    {
      "stream": {
        "watch": ""
      }
    }
  ]
}

Contents

• Minimal example

• Aggregations

• Behaviour

• General

• Input

• Missing Data

• Output

• Recency

• Resources

• Windowing

• Schema

A ✓ marks a field that accepts a context variable such as {{ VARIABLE }}.

Aggregations

Aggregations

Field	Type	Required	Description
aggregations	Aggregations[]		Optional incremental aggregates maintained per key.

Behaviour

Behaviour

Field	Type	Required	Description
delta ✓	boolean (bool)		Emit the difference between the current and last value. Default: false
only-changes ✓	boolean (bool)		Only emit records when the watched field changes. Default: false
reset-on-document-end ✓	boolean (bool)		Reset state when a document end marker is observed. Default: false

General

General

Field	Type	Required	Description
description	string		Describe this step.
condition	lua-expression (string)		Only run this action when the Lua condition evaluates to true. Examples: 2 * count()

Input

Input

Field	Type	Required	Description
watch	field (string)	✅	Field whose value should be tracked. Examples: data_field
group-by	field (string)		Group tracking state by this field’s value. Examples: data_field
input-time	field (string)		Source field providing event time for elapsed calculations. Examples: data_field

Missing Data

Missing Data

Field	Type	Required	Description
fill-strategy	Fill Strategy		Strategy applied when the watched value is missing. Allowed values: forward-fill, default-value

Output

Output

Field	Type	Required	Description
marker	string		Optional marker added to generated events (when not enriching).
output-field	field (string)		Field name used for the emitted delta. Examples: data_field
elapsed-field	field (string)		Field name capturing elapsed milliseconds. Examples: data_field

Recency

Recency

Field	Type	Required	Description
recency	Recency		Emit a boolean indicating whether the key changed within a threshold.

Resources

Resources

Field	Type	Required	Description
max-keys	number (integer)		Maximum number of concurrent keys tracked. Examples: 42, 1.2e-10
eviction	Eviction		Eviction policy applied when max-keys is exceeded. Allowed values: drop-new, lru, ttl

Windowing

Windowing

Field	Type	Required	Description
window-size	number (integer)		Maximum number of recent events to keep per key when computing aggregates. Examples: 42, 1.2e-10

Schema

• Fill Strategy Options
• Eviction Options
• Aggregations Fields
• Fill Strategy - Default Value Fields
• Recency Fields
• Eviction - Ttl Fields
• Aggregations - Op Options

Fill Strategy Options

Option	Name	Type	Description
forward-fill	Forward Fill	map	Carry forward the previous value when the field is missing.
default-value	Default Value	object	Substitute a default JSON value when the field is missing.

Eviction Options

Option	Name	Type	Description
drop-new	Drop New	map	Drop new keys when the limit is reached.
lru	Lru	map	Evict the least-recently updated key.
ttl	Ttl	object	Evict keys that have been idle longer than the configured duration.

Aggregations Fields

Field	Type	Required	Description
field	field (string)	✅	Source field used for the aggregation. Examples: data_field
op	Op		Aggregation operation. Allowed values: avg, mean, min, max, first, last, sum, stddev, range, earliest, …
r-as	field (string)		Optional alias for the output field. Examples: data_field
window-size	number (integer)		Override window size for this aggregation (falls back to stream-level window). Examples: 42, 1.2e-10
percentile	number (integer)		Percentile to compute when op = percentile (0-100). Examples: 42, 1.2e-10

Fill Strategy - Default Value Fields

Field	Type	Required	Description
value	map (object)	✅

Recency Fields

Field	Type	Required	Description
threshold-ms	duration (integer)	✅	Threshold in milliseconds for considering a key “recent”.
output-field	field (string)		Field where the recency flag is written. Examples: data_field

Eviction - Ttl Fields

Field	Type	Required	Description
max-idle	string	✅

Aggregations - Op Options

Value	Aliases	Name	Description
avg		avg
mean		mean
min		min
max		max
first		first
last		last
sum		sum
stddev		stddev
range		range
earliest		earliest
latest		latest
count		count
distinct-count		distinct-count
median		median
percentile		percentile
variance		variance
variance-population		variance-population
stddev-population		stddev-population
sum-squares		sum-squares
mode		mode
list		list
values		values