Elasticsearch

Elasticsearch (elasticsearch)

Bulk-index events into Elasticsearch clusters.

Search Enterprise edition binary json raw

Minimal example

output:
  elasticsearch: {}

JSON

{
  "output": {
    "elasticsearch": {}
  }
}

Contents

• Minimal example

• Batching

• Connection

• Headers

• Processing

• Reliability

• Routing

• Security

• Schema

A ✓ marks a field that accepts a context variable such as {{ VARIABLE }}.

Batching

Batching

Field	Type	Required	Description
batch	Batch		Logical batching hints shared with other outputs.

Connection

Connection

Field	Type	Required	Description
endpoint ✓	url (string)		HTTPS endpoint for the cluster (required when cloud-id is unset). Examples: https://example.com/path
cloud-id ✓	string		Elastic Cloud / Serverless Cloud ID (mutually exclusive with endpoint).
region ✓	string		Optional default region used for auth helpers and logging.
query ✓	map (string)		Static query parameters appended to bulk requests.
request-timeout-secs	number (integer)		Request timeout in seconds. Examples: 42, 1.2e-10
connect-timeout-secs	number (integer)		Connect timeout in seconds. Examples: 42, 1.2e-10

Headers

Headers

Field	Type	Required	Description
headers ✓	map (string)		Additional HTTP headers forwarded with each request.

Processing

Processing

Field	Type	Required	Description
pipeline ✓	string		Optional ingest pipeline to apply server-side.
op-type	Op Type		Bulk operation type for each event. Allowed values: index, create, update, delete
retry-on-conflict	number (integer)		Retry on conflict (for update/upsert operations). Examples: 42, 1.2e-10
refresh ✓	boolean (bool)		Force refresh after each batch. Default: false
gzip ✓	boolean (bool)		Compress bulk payloads with gzip. Default: false
input-field	field (string)		Event field containing the JSON document payload. Examples: data_field
input-pointer	json-pointer (string)		JSON pointer for extracting the payload.
payload-template	string		Template expression to render the payload when not using event body.

Reliability

Reliability

Field	Type	Required	Description
retry	Retry		Retry policy for bulk failures.

Routing

Routing

Field	Type	Required	Description
mode	Mode		Indexing lifecycle mode (classic index vs data stream). Allowed values: classic-index, data-stream
index ✓	string		Static index/alias target when mode=classic-index.
index-field	field (string)		Event field that provides the target index/alias. Examples: data_field
index-pointer	json-pointer (string)		JSON pointer resolving to the index/alias.
data-stream	Data Stream		Data stream configuration when mode=data-stream.
document-id-field	field (string)		Event field containing the document _id. Examples: data_field
document-id-pointer	json-pointer (string)		JSON pointer resolving to the document _id.
document-id-expression	string		Expression used to compute the document _id.
routing-field	field (string)		Event field providing custom routing key. Examples: data_field
routing-pointer	json-pointer (string)		JSON pointer providing custom routing key.
routing-expression	string		Expression computing the routing key.

Security

Security

Field	Type	Required	Description
auth	Auth		Authentication strategies (basic, API keys, bearer).
aws	Aws		AWS SigV4 credentials for managed OpenSearch/Elasticsearch.

Schema

• Batch Fields
• Retry - Backoff Fields
• Retry Fields
• Data Stream Fields
• Auth - Basic Fields
• Auth - Api Key Fields
• Auth - Tls Fields
• Auth Fields
• Aws Fields
• Query Table
• Headers Table
• Batch - Mode Options
• Op Type Options
• Retry - Backoff - Strategy Options
• Mode Options

Batch Fields

Field	Type	Required	Description
fixed-size ✓	number (integer)		maximum number of events in an output batch. Examples: 42, 1.2e-10
max-bytes ✓	number (integer)		Close the batch before adding an event that would make the serialized request payload exceed this many bytes. Examples: 42, 1.2e-10
mode	Mode	✅	If ‘document’ send on end of document generated by input. If ‘fixed’, use fixed_size. Allowed values: fixed, document
timeout	time-interval (string)	✅	interval after which the batch is sent, to keep throughput going (default 100ms). Default: 100ms Examples: 500ms, 2h
header	multiline-text (string)		put a header line before the batch.
footer	multiline-text (string)		put a header line after the last line of the batch.
use-document-marker ✓	boolean (bool)		Enrich the job metadata with a document marker (for document handling in batch mode). Default: false
wrap-as-json ✓	boolean (bool)		Format the output batch as a JSON array. Default: false

Retry - Backoff Fields

Field	Type	Required	Description
strategy	Strategy		Backoff strategy to use (default exponential). Allowed values: exponential, linear, fixed
base	string		Base delay before retrying (e.g. “200ms”).
max	string		Maximum delay between retries.
jitter ✓	boolean (bool)		Whether to add jitter to retry delays. Default: false

Retry Fields

Field	Type	Required	Description
max-attempts	number (integer)		Maximum attempts before giving up. Examples: 42, 1.2e-10
forever ✓	boolean (bool)		Retry indefinitely until cancelled. Default: false
backoff	Backoff		Backoff strategy configuration.

Data Stream Fields

Field	Type	Required	Description
r-type ✓	string	✅	Data stream type (logs, metrics, traces).
dataset ✓	string	✅	Data stream dataset (e.g. service.application).
namespace ✓	string	✅	Data stream namespace (e.g. prod, staging).

Auth - Basic Fields

Field	Type	Required	Description
username ✓	string
password ✓	string

Auth - Api Key Fields

Field	Type	Required	Description
id ✓	string
key ✓	string
base64 ✓	string

Auth - Tls Fields

Field	Type	Required	Description
ca-certificate ✓	string		PEM encoded CA certificate bundle.
client-certificate ✓	string		PEM encoded client certificate for mutual TLS.
client-key ✓	string		PEM encoded private key for mutual TLS.
insecure-skip-verify ✓	boolean (bool)		Skip TLS certificate verification (development only). Default: false

Auth Fields

Field	Type	Required	Description
basic	Basic
api-key	Api Key
bearer-token ✓	string
tls	Tls

Aws Fields

Field	Type	Required	Description
region ✓	string	✅	AWS region used for request signing.
service ✓	string		Override AWS service name (default es).
profile ✓	string		Optional shared credentials profile.
role-arn ✓	string		AssumeRole ARN for federated access.
session-name ✓	string		Session name when assuming roles.
access-key-id ✓	string		Access key used for signing requests.
secret-access-key ✓	string		Secret key used for signing requests.
session-token ✓	string		Optional session token for temporary credentials.

Query Table

Parameter	Value
param	value

Value format: templated-text.

Headers Table

Header	Value
Header-Name	value

Value format: templated-text.

Batch - Mode Options

Value	Aliases	Name	Description
fixed		fixed
document		document

Op Type Options

Value	Aliases	Name	Description
index		index	Index (create or replace) each document
create		create	Only create new documents; fail if _id exists
update		update	Update existing documents (requires partial document or script)
delete		delete	Delete documents (requires _id)

Retry - Backoff - Strategy Options

Value	Aliases	Name	Description
exponential		exponential
linear		linear
fixed		fixed

Mode Options

Value	Aliases	Name	Description
classic-index		classic-index	Standard index or alias writes
data-stream		data-stream	Data stream aware writes (auto-selects stream based on dataset/namespace)