Skip to content

Web Dav Store

Web Dav Store (web-dav-store)

Write to a WebDAV-compatible object store.

Object Store WebDAV binary json raw

Minimal example

output:
web-dav-store:
base-url: ~
object-name:
name: ~
JSON
{
"output": {
"web-dav-store": {
"base-url": null,
"object-name": {
"name": null
}
}
}
}

Contents

Fields

FieldTypeRequiredDescription
base-url LocationstringBase URL for the WebDAV endpoint (e.g., https://host/webdav).
object-name LocationObject NameFile name (may contain slashes).
retry ReliabilityRetryRetry on failure.
batch ProcessingBatchWrite events as batches.
bucket-name LocationstringOptional bucket or root path that will be prepended to object names.
mode BehaviorModeSpecify whether we’re putting or deleting.
Allowed values: put, delete
input-field Processingfield (string)Use the specified field as the content for the file line.
Examples: data_field
disable-object-name-guid Object Propertiesboolean (bool)Disable the GUID prefix if you want object name to be treated literally (off for deletes).
guid-prefix Object PropertiesstringGUID Prefix, will be prepended to the GUID, the default value is ”/”.
guid-suffix Object PropertiesstringGUID Suffix, will be appended to the GUID if specified.
preprocessors ProcessingPreprocessors[]Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified.
Allowed values: gzip, parquet, base64
track-schema Reliabilityboolean (bool)Check the schema of the written data and update __SCHEMA_NUMBER (written data must be JSON).
config Advancedmap (string)Additional configuration passed directly to the HTTP client (see object_store http configs).

Reliability

Show fields
FieldTypeRequiredDescription
retryRetryRetry on failure.
track-schemaboolean (bool)Check the schema of the written data and update __SCHEMA_NUMBER (written data must be JSON).

Processing

Show fields
FieldTypeRequiredDescription
batchBatchWrite events as batches.
input-fieldfield (string)Use the specified field as the content for the file line.
Examples: data_field
preprocessorsPreprocessors[]Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified.
Allowed values: gzip, parquet, base64

Location

Show fields
FieldTypeRequiredDescription
base-urlstringBase URL for the WebDAV endpoint (e.g., https://host/webdav).
object-nameObject NameFile name (may contain slashes).
bucket-namestringOptional bucket or root path that will be prepended to object names.

Behavior

Show fields
FieldTypeRequiredDescription
modeModeSpecify whether we’re putting or deleting.
Allowed values: put, delete

Object Properties

Show fields
FieldTypeRequiredDescription
disable-object-name-guidboolean (bool)Disable the GUID prefix if you want object name to be treated literally (off for deletes).
guid-prefixstringGUID Prefix, will be prepended to the GUID, the default value is ”/”.
guid-suffixstringGUID Suffix, will be appended to the GUID if specified.

Advanced

Show fields
FieldTypeRequiredDescription
configmap (string)Additional configuration passed directly to the HTTP client (see object_store http configs).

Schema

Object Name Options

OptionNameTypeDescription
nameNamestringObject Name.
fieldFieldstringField containing the Object Name.

Retry Fields

FieldTypeRequiredDescription
timeouttime-interval (string)timeout (e.g. 500ms, 2s etc. - default is 30).
Examples: 500ms, 2h
retriesnumber (integer)number of retries.
Examples: 42, 1.2e-10

Batch Fields

FieldTypeRequiredDescription
fixed-sizenumber (integer)maximum number of events in an output batch.
Examples: 42, 1.2e-10
modeModeIf ‘document’ send on end of document generated by input. If ‘fixed’, use fixed_size.
Allowed values: fixed, document
timeouttime-interval (string)interval after which the batch is sent, to keep throughput going (default 100ms).
Examples: 500ms, 2h
headermultiline-text (string)put a header line before the batch.
footermultiline-text (string)put a header line after the last line of the batch.
use-document-markerboolean (bool)Enrich the job metadata with a document marker (for document handling in batch mode).
wrap-as-jsonboolean (bool)Format the output batch as a JSON array.

Config Table

SettingValue
setting.namevalue

Value format: templated-text.

Batch - Mode Options

ValueDescription
fixedFixed
documentDocument

Mode Options

ValueDescription
putPut Objects
deleteDelete Objects

Preprocessors Options

ValueDescription
gzipGzip the output data
parquetExtract the received data as JSON rows from a parquet file
base64Decode base64 as binary