Connecting to Google Cloud Storage
Matillion CDC can load data from any of your pipelines into Google Cloud Storage. Follow the steps on this page to configure Google Cloud Storage as a destination.
Please refer to the following topics to help you set up CDC for Google Cloud Storage:
Google Cloud Storage prerequisites
- You need a Google Cloud Platform account with administrative permissions to create resources. Signing up is free.
- Permissions to create and manage IAM Roles on Google Cloud Storage buckets. Users must be able to create a bucket (if one doesn't already exist), add/modify bucket policies, and upload files to the bucket.
- The Google Service account used by the Agent container should have the following permissions for the Google Cloud Storage bucket.
- Provide a unique prefix name for each pipeline.
Read Google's Cloud Storage documentation to learn more about the service.
Connect to Google Cloud Storage
- After you configure the source during a CDC pipeline creation, you will be directed to choose a destination you would like to load your data into.
- In the Choose destination page, select Google Cloud Storage.
Configure Google Cloud Storage Connection Settings
Specify the following settings in the Connect to Google Cloud Storage destination page:
|Bucket||This refers to the name of the Google Cloud Storage bucket you want to use as a destination. Find your bucket name in the Google Cloud Management Console.|
|Prefix||Prefix is the name of the 'folder' or a location within the storage bucket that all CDC data for this pipeline should be saved to. You could have multiple agents using the same bucket with different prefixes.|
A pipeline must have one unique prefix.
You can test your connection by using the Test connection button at the bottom; if the test is successful, click Test and Continue.