Google Cloud Storage (gcs)
Write events to a Google Cloud bucket.
Contents
Authentication
Authentication
| Field | Type | Required | Description |
|---|
credentials | gcs_output:credentials | ✅ | Credentials for GCP. Supports service account JSON, application-default credentials, or context-provided material via PossibleContextString. |
Behavior
Behavior
| Field | Type | Required | Description |
|---|
mode | Mode | | Put or delete the object? Put uploads content (default), while Delete removes the named object. |
Location
Location
| Field | Type | Required | Description |
|---|
bucket-name | string | ✅ | Bucket Name. |
object-name | gcs_output:object-name | ✅ | Remote object name or field reference (via ObjectDestination). Supports folder-style paths using / separators. |
Object properties
Object Properties
| Field | Type | Required | Description |
|---|
disable-object-name-guid | boolean (bool) | | Disable the autogenerated GUID prefix so the provided object name is used verbatim. This is forced off for delete operations. |
guid-prefix | string | | Optional GUID prefix appended ahead of the generated identifier. Defaults to /. |
guid-suffix | string | | Optional GUID suffix appended after the generated identifier. |
Processing
Processing
| Field | Type | Required | Description |
|---|
batch | Batch | | Configure batching before uploads; batching is recommended when writing many small events to reduce API calls. |
input-field | event-field (string) | | Use the specified field as the object payload. When omitted the entire event body is serialized according to the selected preprocessors. |
preprocessors | Preprocessors | | Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified. |
Reliability
Reliability
| Field | Type | Required | Description |
|---|
retry | Retry | | How to retry operation if it fails. |
track-schema | boolean (bool) | | Validate JSON payloads against the stored schema and maintain __SCHEMA_NUMBER metadata. Only available when the payload serializes to JSON. |
Retry Fields
| Field | Type | Required | Description |
|---|
count | integer | | The number of retry attempts. If unspecified, retries will continue indefinitely. |
pause | string | | How long to pause before re-trying. |
Batch Fields
| Field | Type | Required | Description |
|---|
fixed-size | integer | | maximum number of events in an output batch. |
mode | symbol | ✅ | If ‘document’ send on end of document generated by input. If ‘fixed’, use fixed_size. |
timeout | string | ✅ | interval after which the batch is sent, to keep throughput going (default 100ms). |
header | string | | put a header line before the batch. |
footer | string | | put a header line after the last line of the batch. |
use-document-marker | bool | | Enrich the job metadata with a document marker (for document handling in batch mode). |
wrap-as-json | bool | | Format the output batch as a JSON array. |
Mode Options
| Value | Name | Description |
|---|
put | put | Put Objects |
delete | delete | Delete Objects |
Preprocessors Options
| Value | Name | Description |
|---|
gzip | gzip | Gzip the output data |
parquet | parquet | Extract the received data as JSON rows from a parquet file |
base64 | base64 | Decode base64 as binary |