Skip to content
LyftData Docs

Aggregate

Aggregate (aggregate)

Section titled “Aggregate (aggregate)”

Aggregate events by key and emit summary statistics.

Stateful Transform json

Minimal example

Section titled “Minimal example”
actions:
  - aggregate: {}
JSON 
{
  "actions": [
    {
      "aggregate": {}
    }
  ]
}

Contents

Section titled “Contents”

A ✓ marks a field that accepts a context variable such as {{ VARIABLE }}.

Fields

Section titled “Fields”
Field Type Required Description
aggregations Aggregations[] Aggregations to compute for each group.

Behaviour

Section titled “Behaviour”
Behaviour
Field Type Required Description
reset-on-document ✓ boolean (bool) Reset aggregation state on document boundaries.
Default: false

General

Section titled “General”
General
Field Type Required Description
description string describe this step.
condition string Only run this action if the condition is met.

Grouping

Section titled “Grouping”
Grouping
Field Type Required Description
group-by string[] Fields used to compute the grouping key (optional).

Output

Section titled “Output”
Output
Field Type Required Description
output Output Output configuration (per-window record or expand).
Allowed values: per-window-record, expand-records

Resources

Section titled “Resources”
Resources
Field Type Required Description
max-groups number (integer) Maximum distinct groups tracked in-memory (new groups beyond this limit are dropped).
Examples: 42, 1.2e-10

State

Section titled “State”
State
Field Type Required Description
state State State configuration (in-memory or spill-to-disk).
Allowed values: in-memory, spill-to-disk

Trigger

Section titled “Trigger”
Trigger
Field Type Required Description
aggregate-trigger Aggregate Trigger Trigger policy controlling emission cadence.

Windowing

Section titled “Windowing”
Windowing
Field Type Required Description
time-field field (string) Field containing the event timestamp (ISO 8601).
Examples: data_field
window Window Optional window definition for tumbling windows.
allowed-lateness ✓ string Allow events to arrive up to this duration late (e.g. “30s”).

Schema

Section titled “Schema”

State Options

Section titled “State Options”
Option Name Type Description
in-memory In Memory map
spill-to-disk Spill To Disk object Persist state to disk under the job data directory configured for the runtime.

Aggregations Fields

Section titled “Aggregations Fields”
Field Type Required Description
field string ✅ Source field for aggregation.
op Op Aggregation operation.
Allowed values: count, sum, mean, min, max, first, last, stddev, variance, z-score
r-as string Alias for the output field (defaults to op_field).

State - Spill To Disk Fields

Section titled “State - Spill To Disk Fields”
Field Type Required Description
dir string ✅ Subdirectory (or relative path) within the job data directory to store spill files.

Aggregate Trigger Fields

Section titled “Aggregate Trigger Fields”
Field Type Required Description
count number (integer) Emit after this many events per group.
Examples: 42, 1.2e-10
interval string Emit periodically based on processing time (e.g. “30s”).
on-window-close ✓ boolean (bool) Emit when a window closes.
Default: false

Window Fields

Section titled “Window Fields”
Field Type Required Description
size string ✅ Window size (e.g. “1m”).
offset string Optional offset applied to the window (e.g. “10s”).

Aggregations - Op Options

Section titled “Aggregations - Op Options”
Value Aliases Name Description
count count
sum sum
mean avg, average mean
min min
max max
first first
last last
stddev std, std_dev, stdev stddev
variance var variance
z-score z_score z-score

Output Options

Section titled “Output Options”
Value Aliases Name Description
per-window-record per-window-record Emit a single record summarising the window/group
expand-records expand-records Expand aggregated results into individual events per source event