Aggregate (aggregate)
Aggregate events by key and emit summary statistics.
Stateful Transform json
Minimal example
JSON
Contents
Fields
| Field | Type | Required | Description |
|---|
aggregations | Aggregations[] | | Aggregations to compute for each group. |
Behaviour
Behaviour
| Field | Type | Required | Description |
|---|
reset-on-document ✓ | boolean (bool) | | Reset aggregation state on document boundaries.
Default: false |
General
General
| Field | Type | Required | Description |
|---|
description | string | | describe this step. |
condition | string | | Only run this action if the condition is met. |
Grouping
Grouping
| Field | Type | Required | Description |
|---|
group-by | string[] | | Fields used to compute the grouping key (optional). |
Output
Output
| Field | Type | Required | Description |
|---|
output | Aggregation Output | | Output configuration (per-window record or expand).
Allowed values: per-window-record, expand-records |
Resources
Resources
| Field | Type | Required | Description |
|---|
max-groups | number (integer) | | Maximum distinct groups tracked in-memory (new groups beyond this limit are dropped).
Examples: 42, 1.2e-10 |
State
State
| Field | Type | Required | Description |
|---|
state | Aggregation State | | State configuration (in-memory or spill-to-disk).
Allowed values: in-memory, spill-to-disk |
Trigger
Trigger
| Field | Type | Required | Description |
|---|
aggregate-trigger | Aggregate Trigger | | Trigger policy controlling emission cadence. |
Windowing
Windowing
| Field | Type | Required | Description |
|---|
time-field | field (string) | | Field containing the event timestamp (ISO 8601).
Examples: data_field |
window | Window | | Optional window definition for tumbling windows. |
allowed-lateness ✓ | string | | Allow events to arrive up to this duration late (e.g. ”30s”). |
Schema
Aggregation State Options
| Option | Name | Type | Description |
|---|
in-memory | In Memory | map | |
spill-to-disk | Spill To Disk | object | Persist state to disk under the job data directory configured for the runtime. |
Aggregations Fields
| Field | Type | Required | Description |
|---|
field | string | ✅ | Source field for aggregation. |
op | Aggregation Function | | Aggregation operation.
Allowed values: count, sum, mean, min, max, first, last, stddev, variance, z-score |
r-as | string | | Alias for the output field (defaults to op_field). |
Aggregation State - Spill To Disk Fields
| Field | Type | Required | Description |
|---|
dir | string | ✅ | Subdirectory (or relative path) within the job data directory to store spill files. |
Aggregate Trigger Fields
| Field | Type | Required | Description |
|---|
count | number (integer) | | Emit after this many events per group.
Examples: 42, 1.2e-10 |
interval | string | | Emit periodically based on processing time (e.g. ”30s”). |
on-window-close ✓ | boolean (bool) | | Emit when a window closes.
Default: false |
Window Fields
| Field | Type | Required | Description |
|---|
size | string | ✅ | Window size (e.g. “1m”). |
offset | string | | Optional offset applied to the window (e.g. ”10s”). |
Aggregations - Aggregation Function Options
| Value | Name | Description |
|---|
count | count | Count |
sum | sum | Sum |
mean | mean | Mean |
min | min | Min |
max | max | Max |
first | first | First |
last | last | Last |
stddev | stddev | Stddev |
variance | variance | Variance |
z-score | z-score | Z Score |
Aggregation Output Options
| Value | Name | Description |
|---|
per-window-record | per-window-record | Emit a single record summarising the window/group |
expand-records | expand-records | Expand aggregated results into individual events per source event |