Google Cloud Storage

Google Cloud Storage (gcs)

Write events to a Google Cloud bucket.

Block Store Cloud binary json raw

Minimal example

output:
  gcs:
    bucket-name: ~
    credentials:
      storage-url: ~
    object-name:
      name: ~

JSON

{
  "output": {
    "gcs": {
      "bucket-name": null,
      "credentials": {
        "storage-url": null
      },
      "object-name": {
        "name": null
      }
    }
  }
}

Contents

• Minimal example

• Fields

• Reliability

• Processing

• Location

• Behavior

• Object Properties

• Authentication

• Schema

Fields

Field	Type	Required	Description
bucket-name Location	string	✅	Bucket Name.
object-name Location	Object Name	✅	Remote object name or field reference (via ObjectDestination). Supports folder-style paths using / separators.
credentials Authentication	Credentials	✅	Credentials for GCP. Supports service account JSON, application-default credentials, or context-provided material via PossibleContextString.
retry Reliability	Retry		How to retry operation if it fails.
batch Processing	Batch		Configure batching before uploads; batching is recommended when writing many small events to reduce API calls.
input-field Processing	field (string)		Use the specified field as the object payload. When omitted the entire event body is serialized according to the selected preprocessors. Examples: data_field
mode Behavior	Mode		Put or delete the object? Put uploads content (default), while Delete removes the named object. Allowed values: put, delete
disable-object-name-guid Object Properties	boolean (bool)		Disable the autogenerated GUID prefix so the provided object name is used verbatim. This is forced off for delete operations.
guid-prefix Object Properties	string		Optional GUID prefix appended ahead of the generated identifier. Defaults to /.
guid-suffix Object Properties	string		Optional GUID suffix appended after the generated identifier.
preprocessors Processing	Preprocessors[]		Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified. Allowed values: gzip, parquet, base64
track-schema Reliability	boolean (bool)		Validate JSON payloads against the stored schema and maintain __SCHEMA_NUMBER metadata. Only available when the payload serializes to JSON.

Reliability

Show fields

Field	Type	Required	Description
retry	Retry		How to retry operation if it fails.
track-schema	boolean (bool)		Validate JSON payloads against the stored schema and maintain __SCHEMA_NUMBER metadata. Only available when the payload serializes to JSON.

Processing

Show fields

Field	Type	Required	Description
batch	Batch		Configure batching before uploads; batching is recommended when writing many small events to reduce API calls.
input-field	field (string)		Use the specified field as the object payload. When omitted the entire event body is serialized according to the selected preprocessors. Examples: data_field
preprocessors	Preprocessors[]		Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified. Allowed values: gzip, parquet, base64

Location

Show fields

Field	Type	Required	Description
bucket-name	string	✅	Bucket Name.
object-name	Object Name	✅	Remote object name or field reference (via ObjectDestination). Supports folder-style paths using / separators.

Behavior

Show fields

Field	Type	Required	Description
mode	Mode		Put or delete the object? Put uploads content (default), while Delete removes the named object. Allowed values: put, delete

Object Properties

Show fields

Field	Type	Required	Description
disable-object-name-guid	boolean (bool)		Disable the autogenerated GUID prefix so the provided object name is used verbatim. This is forced off for delete operations.
guid-prefix	string		Optional GUID prefix appended ahead of the generated identifier. Defaults to /.
guid-suffix	string		Optional GUID suffix appended after the generated identifier.

Authentication

Show fields

Field	Type	Required	Description
credentials	Credentials	✅	Credentials for GCP. Supports service account JSON, application-default credentials, or context-provided material via PossibleContextString.

Schema

• Object Name Options
• Credentials - Service Account Options
• Credentials - Application Credentials Options
• Credentials Options
• Retry Fields
• Batch Fields
• Batch - Mode Options
• Mode Options
• Preprocessors Options

Object Name Options

Option	Name	Type	Description
name	Name	string	Object Name.
field	Field	string	Field containing the Object Name.

Credentials - Service Account Options

Option	Name	Type	Description
path	Path	string	Service Account File location. Examples: /path/to/file, c:\users\joe\data\file.txt
key	Key	string

Credentials - Application Credentials Options

Option	Name	Type	Description
path	Path	string	Application Credential File Location. Examples: /path/to/file, c:\users\joe\data\file.txt
key	Key	string	Application Credential in JSON format.

Credentials Options

Option	Name	Type	Description
storage-url	Storage Url	string	URL in the form gs:///path.
service-account	Service Account	one_of	Service Account credentials.
application-credentials	Application Credentials	one_of	Application credentials.

Retry Fields

Field	Type	Required	Description
timeout	time-interval (string)	✅	timeout (e.g. 500ms, 2s etc. - default is 30). Examples: 500ms, 2h
retries	number (integer)		number of retries. Examples: 42, 1.2e-10

Batch Fields

Field	Type	Required	Description
fixed-size	number (integer)		maximum number of events in an output batch. Examples: 42, 1.2e-10
mode	Mode	✅	If ‘document’ send on end of document generated by input. If ‘fixed’, use fixed_size. Allowed values: fixed, document
timeout	time-interval (string)	✅	interval after which the batch is sent, to keep throughput going (default 100ms). Examples: 500ms, 2h
header	multiline-text (string)		put a header line before the batch.
footer	multiline-text (string)		put a header line after the last line of the batch.
use-document-marker	boolean (bool)		Enrich the job metadata with a document marker (for document handling in batch mode).
wrap-as-json	boolean (bool)		Format the output batch as a JSON array.

Batch - Mode Options

Value	Description
fixed	Fixed
document	Document

Mode Options

Value	Description
put	Put Objects
delete	Delete Objects

Preprocessors Options

Value	Description
gzip	Gzip the output data
parquet	Extract the received data as JSON rows from a parquet file
base64	Decode base64 as binary