Infer

Infer (infer)

Execute inference workloads (LLM, embeddings, anomaly detection).

AI & ML json

Minimal example

actions:
  - infer: {}

JSON

{
  "actions": [
    {
      "infer": {}
    }
  ]
}

Contents

• Minimal example

• Fields

• Schema

Fields

Field	Type	Required	Description
description	string		describe this step.
condition	lua-expression (string)		Only run this action if the condition is met. Examples: 2 * count()
workload	Workload		Mode-specific configuration (llm, embedding, or anomaly).
timeout-ms	duration (integer)		Maximum time to wait under infer backpressure (milliseconds, 0 uses the built-in default).
cache	Cache		Optional cache configuration.
rate-limit	Rate Limit		Optional rate limiting configuration.
concurrency	number (integer)		Maximum concurrent in-flight requests (overrides provider defaults). Examples: 42, 1.2e-10
streaming	boolean (bool)		Enable streaming responses when supported (LLM mode only).
on-error	string		Error handling strategy (e.g. fail, skip, dlq:name).

Schema

• Workload Options
• Workload - Llm Completion - Llm - Prompt Fields
• Workload - Llm Completion - Llm - Field Map Fields
• Workload - Llm Completion - Llm Fields
• Workload - Llm Completion Fields
• Workload - Embedding - Embedding Fields
• Workload - Embedding Fields
• Workload - Anomaly Detect - Anomaly - Training Fields
• Workload - Anomaly Detect - Anomaly Fields
• Workload - Anomaly Detect Fields
• Cache Fields
• Rate Limit Fields
• Workload - Anomaly Detect - Anomaly - Training - Params Table
• Workload - Llm Completion - Llm - Provider Options
• Workload - Llm Completion - Llm - Response Format Options
• Workload - Embedding - Embedding - Provider Options
• Workload - Anomaly Detect - Anomaly - Algo Options
• Workload - Anomaly Detect - Anomaly - Training - Mode Options

Workload Options

Option	Name	Type	Description
llm-completion	Llm Completion	object	Large language model completion.
embedding	Embedding	object	Text or document embeddings.
anomaly-detect	Anomaly Detect	object	Anomaly detection scoring.

Workload - Llm Completion - Llm - Prompt Fields

Field	Type	Required	Description
system	string		System message supplied to the model.
template	string		Prompt template (may include {{json record}} style placeholders).
schema	string		Optional JSON schema or tool definitions for structured output.

Workload - Llm Completion - Llm - Field Map Fields

Field	Type	Required	Description
input	field (string)		Field containing the prompt input payload. Examples: data_field
output	field (string)		Field to write model response into. Examples: data_field
usage	field (string)		Optional field to capture token usage metadata. Examples: data_field

Workload - Llm Completion - Llm Fields

Field	Type	Required	Description
provider	Provider	✅	Provider selection (llama-server or openai-compat). Allowed values: llama-server, openai-compat
model	string	✅	Model identifier or path (provider specific).
endpoint	url (string)		Optional HTTP endpoint for remote providers (OpenAI-compatible / llama-server). Examples: https://example.com/path
api-key	string		Secret or variable reference for the API key / bearer token (use `${dyn
prompt	Prompt		Prompt template configuration.
field-map	Field Map		Field mapping for input/output projection.
input-field	field (string)		Field containing the prompt input payload. Examples: data_field
response-field	field (string)	✅	Field to write model response into (required). Examples: data_field
usage-field	field (string)		Optional field to capture token usage metadata. Examples: data_field
response-format	Response Format		Desired response format emitted by the provider. Allowed values: str, json
concurrency	number (integer)		Maximum concurrent requests for this provider (overrides action-level concurrency). Examples: 42, 1.2e-10
streaming	boolean (bool)		Enable streaming token responses when supported.
temperature	number (integer)		Temperature parameter for text generation. Examples: 42, 1.2e-10
top-p	number (integer)		Top-p parameter for nucleus sampling. Examples: 42, 1.2e-10
max-tokens	number (integer)		Maximum tokens to generate (remote providers only). Examples: 42, 1.2e-10

Workload - Llm Completion Fields

Field	Type	Required	Description
llm	Llm		LLM configuration (required when mode = llm-completion).

Workload - Embedding - Embedding Fields

Field	Type	Required	Description
provider	Provider	✅	Embedding provider selection. Allowed values: llama-server, openai-compat
model	string	✅	Model identifier or path (provider specific).
endpoint	url (string)		HTTP endpoint for remote embedding providers. Examples: https://example.com/path
api-key	string		Secret or variable reference for the API key / bearer token (use `${dyn
input-field	field (string)		Field containing the text payload to embed. Examples: data_field
response-field	field (string)		Field to write the embedding vector to (required). Examples: data_field
additional-response-fields	string[]		Additional fields to mirror the embedding vector into.
normalize	boolean (bool)		Normalize embedding vectors before emitting.
concurrency	number (integer)		Maximum concurrent embedding requests. Examples: 42, 1.2e-10

Workload - Embedding Fields

Field	Type	Required	Description
embedding	Embedding		Embedding configuration (required when mode = embedding).

Workload - Anomaly Detect - Anomaly - Training Fields

Field	Type	Required	Description
mode	Mode		Training mode (online or offline). Allowed values: online, offline
window	number (integer)		Sliding window size for online training. Examples: 42, 1.2e-10
params	map (string)		Algorithm-specific parameters.
model-variable	string		Serialized model payload reference (use `${dyn

Workload - Anomaly Detect - Anomaly Fields

Field	Type	Required	Description
algo	Algo	✅	Algorithm selection. Allowed values: zscore, isolation-forest, one-class-svm
fields	string[]		Numeric fields to monitor for anomalies.
output-field	field (string)	✅	Field to write anomaly score to. Examples: data_field
flag-field	field (string)		Optional field to emit boolean anomaly flag. Examples: data_field
input-field	field (string)		Field containing the anomaly input payload. Examples: data_field
response-field	field (string)		Field to write the anomaly response into (defaults to output_field). Examples: data_field
score-threshold	number (integer)		Score threshold to mark anomalies (optional, algorithm specific). Examples: 42, 1.2e-10
training	Training		Training configuration (online/offline).

Workload - Anomaly Detect Fields

Field	Type	Required	Description
anomaly	Anomaly		Anomaly configuration (required when mode = anomaly-detect).

Cache Fields

Field	Type	Required	Description
namespace	string		Cache namespace identifier.
max-entries	number (integer)		Maximum cache entries. Examples: 42, 1.2e-10
ttl	duration (string)		Cache entry TTL (e.g. “5m”).

Rate Limit Fields

Field	Type	Required	Description
requests-per-second	number (integer)		Maximum requests per second. Examples: 42, 1.2e-10
tokens-per-minute	number (integer)		Maximum tokens per minute (LLM providers). Examples: 42, 1.2e-10
max-concurrency	number (integer)		Maximum concurrent requests. Examples: 42, 1.2e-10

Workload - Anomaly Detect - Anomaly - Training - Params Table

Parameter	Value
param	value

Value format: templated-text.

Workload - Llm Completion - Llm - Provider Options

Value	Description
llama-server	Llama Server
openai-compat	Openai Compat

Workload - Llm Completion - Llm - Response Format Options

Value	Description
str	Emit text/string output (default)
json	Request JSON formatted responses via provider-specific hinting

Workload - Embedding - Embedding - Provider Options

Value	Description
llama-server	Llama Server
openai-compat	Openai Compat

Workload - Anomaly Detect - Anomaly - Algo Options

Value	Description
zscore	Zscore
isolation-forest	Isolation Forest
one-class-svm	One Class Svm

Workload - Anomaly Detect - Anomaly - Training - Mode Options

Value	Description
online	Online
offline	Offline