GCS Put Blobs

Tools

Description

Use this tool to put local files into Google Cloud Storage, as blobs.

This tool sends single or multiple files, using filters to determine which ones to send. Each blob is checked with an MD5 hash to make sure there was no data corruption during the transfer.

To use this tool, define the following a local file or folder source from which you want to send, and a Google Cloud Storage target to send your files to. You can define the source path manually using tool parameters.

Usage

Add the process action TOOL GCS Put Blobs from the Process Palette, under the Tools section.
Select a source and target:
- Optionally, drag and drop one of the following File metadata nodes onto the <SOURCE> field of the tool:
  - Folder
  - File
- Drag and drop one of the following Google Cloud Storage metadata nodes onto the <TARGET> field of the tool:
  - Storage
  - Bucket
  - Folder
If you did not drag and drop a file or folder node as a source, define the source in the tool’s parameters.
Set the blob metadata in other tool parameters as needed.

The tool inherits parameters from the metadata node you drag onto it.

For information about metadata in Google Cloud Storage, see the official documentation.

Parameters

Name Default Description

Name	Default	Description
File Flatten Mode	true	Sets whether or not the remote blobs should have the same directory structure as their local versions, relative to the source directory. `true` Sent files are saved in the target bucket with no subdirectories. `false` The tool recreates the source’s subdirectory stucture. Sent files inherit the same relative directory structure and location on the remote end.
XPath Expression For Source	$SOURCE	A valid XPath expression referencing a directory to use as a source location. The expression can return a file or folder node from a File metadata object. This parameter is ignored if you set the Source Folder Path
Source Folder Path		Manual entry of the source folder. You can omit this parameter if XPath Expression For Source returns a valid reference to a folder.
Source File Includes		A list of file to include in the operation, as a semicolon-separated list of file masks. An empty value matches all files. When the source is a folder, or if you set the Source Folder Path, the file mask evaluates inside this folder. The following wildcard characters are supported: ? Matches one character in a segment of the file’s path. * Matches zero or more characters in a segment of the file’s path. ** Matches zero or more segments of the file’s path of the file. Examples: to retrieve XML and JSON files in the current directory: `.xml;.json` to retrieve XML files in any test subdirectory: `*/test/.xml`
Source File Excludes		A list of files to exclude from the operation, as a semicolon-separated list of file masks. An empty value matches all files. When the source is a folder, or if you set the Source Folder Path, the file mask evaluates inside this folder. The following wildcard characters are supported: ? Matches one character in a segment of the file’s path. * Matches zero or more characters in a segment of the file’s path. ** Matches zero or more segments of the file’s path of the file. Examples: to ignore XML and JSON files in the current directory: `.xml;.json` to ignore XML files in any test subdirectory: `*/test/.xml`
Source File Name		Full path of a file to use as the source. Use this parameter when you want to perform an operation on a single file. When this parameter is set, the tool ignores the Source Folder Path, Source File Includes, and Source File Excludes parameters.
XPath Expression For Target	$TARGET	A valid XPath expression referencing a bucket or directory to use as a target location. The expression can return a storage, bucket, or folder node from a Google Cloud Storage metadata object.
Target Bucket Name		Manual entry of the target bucket name. You can omit this parameter if XPath Expression For Target returns a valid reference to a bucket or one of its children.
Target Directory Path		Manual entry of the target bucket directory. You can omit this parameter if XPath Expression For Target returns a valid reference to a directory or one of its children, or if the bucket itself is the root directory.
Target Storage Class		Specifies the storage class of the target blob. When empty, the blob uses the bucket’s default value.
Target Content-Type (RFC 2616)	`text/plain`	Specifies the media type of the target Blob.
Target Content-Language		Specifies the intended languages of the target Blob.
Target Content-Encoding		Specifies content compression headers for the target Blob.
Target Content-Disposition		Specifies presentation information headers for the target Blob.
Target Cache-Control		Specifies browser cache headers for the target Blob.
Target Metadata		One or more key-value pairs to set on the remote target blob. You can set this parameter in the form of Java properties. For instance: `metadata1=value1 metadata2=value2 #comment metadata3=value3`
Target Blob Name		Full path of a blob to use as the target. Use this parameter when you want to work with a single blob. This parameter only works if you also set the Source File Name.
Existing Blobs Behavior	overwrite	Controls the tool’s behavior if target blobs already exist. Possible options are: overwrite The tool overwrites target blobs. ignore The tool leaves target blobs alone, with no error. throwError The tool leaves target blobs alone, and also throws an error.

File Flatten Mode

true

Sets whether or not the remote blobs should have the same directory structure as their local versions, relative to the source directory.

true: Sent files are saved in the target bucket with no subdirectories.
false: The tool recreates the source’s subdirectory stucture. Sent files inherit the same relative directory structure and location on the remote end.

XPath Expression For Source

$SOURCE

A valid XPath expression referencing a directory to use as a source location. The expression can return a file or folder node from a File metadata object.

This parameter is ignored if you set the Source Folder Path

Source Folder Path

Manual entry of the source folder. You can omit this parameter if XPath Expression For Source returns a valid reference to a folder.

Source File Includes

A list of file to include in the operation, as a semicolon-separated list of file masks. An empty value matches all files.

When the source is a folder, or if you set the Source Folder Path, the file mask evaluates inside this folder.

The following wildcard characters are supported:

?: Matches one character in a segment of the file’s path.
*: Matches zero or more characters in a segment of the file’s path.
**: Matches zero or more segments of the file’s path of the file.

Examples:

to retrieve XML and JSON files in the current directory: *.xml;*.json
to retrieve XML files in any test subdirectory: **/test/*.xml

Source File Excludes

A list of files to exclude from the operation, as a semicolon-separated list of file masks. An empty value matches all files.

When the source is a folder, or if you set the Source Folder Path, the file mask evaluates inside this folder.

The following wildcard characters are supported:

?: Matches one character in a segment of the file’s path.
*: Matches zero or more characters in a segment of the file’s path.
**: Matches zero or more segments of the file’s path of the file.

Examples:

to ignore XML and JSON files in the current directory: *.xml;*.json
to ignore XML files in any test subdirectory: **/test/*.xml

Source File Name

Full path of a file to use as the source. Use this parameter when you want to perform an operation on a single file.

When this parameter is set, the tool ignores the Source Folder Path, Source File Includes, and Source File Excludes parameters.

XPath Expression For Target

$TARGET

A valid XPath expression referencing a bucket or directory to use as a target location. The expression can return a storage, bucket, or folder node from a Google Cloud Storage metadata object.

Target Bucket Name

Manual entry of the target bucket name.

You can omit this parameter if XPath Expression For Target returns a valid reference to a bucket or one of its children.

Target Directory Path

Manual entry of the target bucket directory.

You can omit this parameter if XPath Expression For Target returns a valid reference to a directory or one of its children, or if the bucket itself is the root directory.

Target Storage Class

Specifies the storage class of the target blob. When empty, the blob uses the bucket’s default value.

Target Content-Type (RFC 2616)

text/plain

Specifies the media type of the target Blob.

Target Content-Language

Specifies the intended languages of the target Blob.

Target Content-Encoding

Specifies content compression headers for the target Blob.

Target Content-Disposition

Specifies presentation information headers for the target Blob.

Target Cache-Control

Specifies browser cache headers for the target Blob.

Target Metadata

One or more key-value pairs to set on the remote target blob. You can set this parameter in the form of Java properties.

For instance:

metadata1=value1
metadata2=value2
#comment
metadata3=value3

Target Blob Name

Full path of a blob to use as the target. Use this parameter when you want to work with a single blob.

This parameter only works if you also set the Source File Name.

Existing Blobs Behavior

overwrite

Controls the tool’s behavior if target blobs already exist. Possible options are:

overwrite: The tool overwrites target blobs.
ignore: The tool leaves target blobs alone, with no error.
throwError: The tool leaves target blobs alone, and also throws an error.