.png){kind=logo}
S3 Manifest Builder
-
DarkLight
S3 Manifest Builder
-
DarkLight
This article is specific to the following platforms - Redshift.
S3 Manifest Builder
Build and upload a manifest file which describes a set of files (S3 objects) to load based upon a pattern using a regular expression.
If the pattern is simply a prefix, this is not required as the S3 Load can use a prefix directly. However, for more complicated file-naming schemes, use this component to build a manifest file first, and then use an S3 Load with the manifest option and a path pointing to the generated manifest file.
Properties
| Property | Setting | Description |
|---|---|---|
| Name | Text | A human-readable name for the component. |
| Source S3 Path | S3 Tree | The S3 bucket (and optionally the path within it) to search. |
| Manifest Path | S3 Tree | The S3 location to write the manifest file to. |
| Manifest File name | Text | The file name for the manifest. |
| Filter Regular Expression | Regular Expression |
This is a regular expression. All names found from Source S3 Path are tested
against this expression, and if they match they are included in the manifest.
This requires an exact match, so if you are searching for text anywhere within the name you will need to include wildcards before and after the text. |
| Mandatory | Select | Yes writes the Mandatory option into the manifest, meaning the subsequent S3 load will fail if the file is not present. No makes the file optional, and no error will be raised if the file is missing. |
Variable Exports
This component makes the following values available to export into variables:
| Source | Description |
|---|---|
| Matched Objects | The number of objects matching the filter pattern and written to the Manifest. |