Chunk

Chunk (chunk)

Segment large payloads into smaller chunks for downstream processing.

Text AI & ML Transform json

Minimal example

actions:
  - chunk: {}

JSON

{
  "actions": [
    {
      "chunk": {}
    }
  ]
}

Contents

• Minimal example

• Fields

• Schema

Fields

Field	Type	Required	Description
description	string		Describe this step.
condition	lua-expression (string)		Only run this action if the condition is met. Examples: 2 * count()
input-field	field (string)		Field containing the text or array to chunk. Examples: data_field
output-field	field (string)		Field to write chunk output to when using array mode. Examples: data_field
mode	Mode		Chunking mode (characters, tokens, or sentences). Allowed values: characters, tokens, sentences
size	number (integer)		Target size of each chunk. Examples: 42, 1.2e-10
overlap	number (integer)		Overlap (same unit as size) between consecutive chunks. Examples: 42, 1.2e-10
locale	string		Locale hint used for sentence detection.
output	Output		Output behaviour (array writes chunks to a field, events emits new events). Allowed values: array, events
metadata-field	field (string)		Field used to capture chunk metadata (offsets, identifiers, etc.). Examples: data_field

Schema

• Mode Options
• Output Options

Mode Options

Value	Aliases	Name	Description
characters		characters
tokens		tokens
sentences		sentences

Output Options

Value	Aliases	Name	Description
array		array
events		events