Infer

Infer (infer)

Execute inference workloads (LLM, embeddings, anomaly detection).

AI & ML Enterprise edition json

Minimal example

actions:
  - infer:
      anomaly-detect: {}
      embedding: {}
      llm-completion: {}

JSON

{
  "actions": [
    {
      "infer": {
        "anomaly-detect": {},
        "embedding": {},
        "llm-completion": {}
      }
    }
  ]
}

Contents

• Minimal example

• Fields

• Schema

A ✓ marks a field that accepts a context variable such as {{ VARIABLE }}.

Fields

Field	Type	Required	Description
description	string		describe this step.
condition	lua-expression (string)		Only run this action if the condition is met. Examples: 2 * count()
llm-completion	LLM Completion	✅	Large language model completion.
embedding	Embedding	✅	Text or document embeddings.
anomaly-detect	Anomaly Detect	✅	Anomaly detection scoring.
timeout-ms	number (integer)		Maximum time to wait under infer backpressure (milliseconds, 0 uses the built-in default). Examples: 42, 1.2e-10
cache	Cache		Optional cache configuration.
rate-limit	Rate Limit		Optional rate limiting configuration.
concurrency	number (integer)		Maximum concurrent in-flight requests (overrides provider defaults). Examples: 42, 1.2e-10
streaming	boolean (bool)		Enable streaming responses when supported (LLM mode only).
on-error	string		Error handling strategy (e.g. fail, skip, dlq:name).

Schema

• LLM Completion - Llm - Prompt Fields
• LLM Completion - Llm - Field Map Fields
• LLM Completion - Llm Fields
• LLM Completion Fields
• Embedding - Embedding Fields
• Embedding Fields
• Anomaly Detect - Anomaly - Training Fields
• Anomaly Detect - Anomaly Fields
• Anomaly Detect Fields
• Cache Fields
• Rate Limit Fields
• Anomaly Detect - Anomaly - Training - Params Table
• LLM Completion - Llm - Provider Options
• LLM Completion - Llm - Response Format Options
• Embedding - Embedding - Provider Options
• Anomaly Detect - Anomaly - Algo Options
• Anomaly Detect - Anomaly - Training - Mode Options

LLM Completion - Llm - Prompt Fields

Field	Type	Required	Description
system	string		System message supplied to the model.
template	string		Prompt template (may include {{json record}} style placeholders).
schema	string		Optional JSON schema or tool definitions for structured output.

LLM Completion - Llm - Field Map Fields

Field	Type	Required	Description
input	field (string)		Field containing the prompt input payload. Examples: data_field
output	field (string)		Field to write model response into. Examples: data_field
usage	field (string)		Optional field to capture token usage metadata. Examples: data_field

LLM Completion - Llm Fields

Field	Type	Required	Description
provider	Provider	✅	Provider selection (llama-server or openai-compat). Allowed values: llama-server, openai-compat
model	string	✅	Model identifier or path (provider specific).
endpoint	url (string)		Optional HTTP endpoint for remote providers (OpenAI-compatible / llama-server). Examples: https://example.com/path
api-key ✓	string		Secret or variable reference for the API key / bearer token (use `${dyn
prompt	Prompt		Prompt template configuration.
field-map	Field Map		Field mapping for input/output projection.
input-field	field (string)		Field containing the prompt input payload. Examples: data_field
response-field	field (string)	✅	Field to write model response into (required). Examples: data_field
usage-field	field (string)		Optional field to capture token usage metadata. Examples: data_field
response-format	Response Format		Desired response format emitted by the provider. Allowed values: str, json
concurrency	number (integer)		Maximum concurrent requests for this provider (overrides action-level concurrency). Examples: 42, 1.2e-10
streaming	boolean (bool)		Enable streaming token responses when supported.
temperature	number (integer)		Temperature parameter for text generation. Examples: 42, 1.2e-10
top-p	number (integer)		Top-p parameter for nucleus sampling. Examples: 42, 1.2e-10
max-tokens	number (integer)		Maximum tokens to generate (remote providers only). Examples: 42, 1.2e-10

LLM Completion Fields

Field	Type	Required	Description
llm	Llm		LLM configuration (required when mode = llm-completion).

Embedding - Embedding Fields

Field	Type	Required	Description
provider	Provider	✅	Embedding provider selection. Allowed values: llama-server, openai-compat
model	string	✅	Model identifier or path (provider specific).
endpoint	url (string)		HTTP endpoint for remote embedding providers. Examples: https://example.com/path
api-key ✓	string		Secret or variable reference for the API key / bearer token (use `${dyn
input-field	field (string)		Field containing the text payload to embed. Examples: data_field
response-field	field (string)		Field to write the embedding vector to (required). Examples: data_field
additional-response-fields	string[]		Additional fields to mirror the embedding vector into.
normalize	boolean (bool)		Normalize embedding vectors before emitting.
concurrency	number (integer)		Maximum concurrent embedding requests. Examples: 42, 1.2e-10

Embedding Fields

Field	Type	Required	Description
embedding	Embedding		Embedding configuration (required when mode = embedding).

Anomaly Detect - Anomaly - Training Fields

Field	Type	Required	Description
mode	Mode		Training mode (online or offline). Allowed values: online, offline
window	number (integer)		Sliding window size for online training. Examples: 42, 1.2e-10
params	map (string)		Algorithm-specific parameters.
model-variable ✓	string		Serialized model payload reference (use `${dyn

Anomaly Detect - Anomaly Fields

Field	Type	Required	Description
algo	Algo	✅	Algorithm selection. Allowed values: zscore, isolation-forest, one-class-svm
fields	string[]		Numeric fields to monitor for anomalies.
output-field	field (string)	✅	Field to write anomaly score to. Examples: data_field
flag-field	field (string)		Optional field to emit boolean anomaly flag. Examples: data_field
input-field	field (string)		Field containing the anomaly input payload. Examples: data_field
response-field	field (string)		Field to write the anomaly response into (defaults to output_field). Examples: data_field
score-threshold	number (integer)		Score threshold to mark anomalies (optional, algorithm specific). Examples: 42, 1.2e-10
training	Training		Training configuration (online/offline).

Anomaly Detect Fields

Field	Type	Required	Description
anomaly	Anomaly		Anomaly configuration (required when mode = anomaly-detect).

Cache Fields

Field	Type	Required	Description
namespace	string		Cache namespace identifier.
max-entries	number (integer)		Maximum cache entries. Examples: 42, 1.2e-10
ttl	duration (string)		Cache entry TTL (e.g. “5m”).

Rate Limit Fields

Field	Type	Required	Description
requests-per-second	number (integer)		Maximum requests per second. Examples: 42, 1.2e-10
tokens-per-minute	number (integer)		Maximum tokens per minute (LLM providers). Examples: 42, 1.2e-10
max-concurrency	number (integer)		Maximum concurrent requests. Examples: 42, 1.2e-10

Anomaly Detect - Anomaly - Training - Params Table

Parameter	Value
param	value

Value format: templated-text.

LLM Completion - Llm - Provider Options

Value	Aliases	Name	Description
llama-server		llama-server
openai-compat		openai-compat

LLM Completion - Llm - Response Format Options

Value	Aliases	Name	Description
str		str	Emit text/string output (default)
json		json	Request JSON formatted responses via provider-specific hinting

Embedding - Embedding - Provider Options

Value	Aliases	Name	Description
llama-server		llama-server
openai-compat		openai-compat

Anomaly Detect - Anomaly - Algo Options

Value	Aliases	Name	Description
zscore		zscore
isolation-forest		isolation-forest
one-class-svm		one-class-svm

Anomaly Detect - Anomaly - Training - Mode Options

Value	Aliases	Name	Description
online		online
offline		offline