Connecting to Google Cloud Storage
  • Dark

Connecting to Google Cloud Storage

  • Dark


Matillion CDC can load data from any of your pipelines into Google Cloud Storage. Follow the steps on this page to configure Google Cloud Storage as a destination.


Please refer to the following topics to help you set up CDC for Google Cloud Storage:

Google Cloud Storage prerequisites

  • You need a Google Cloud Platform account with administrative permissions to create resources. Signing up is free.
  • Permissions to create and manage IAM Roles on Google Cloud Storage buckets. Users must be able to create a bucket (if one doesn't already exist), add/modify bucket policies, and upload files to the bucket.
  • The Google Service account used by the Agent container should have the following permissions for the Google Cloud Storage bucket.
  • storage.buckets.get
  • storage.objects.get
  • storage.multipartUploads.abort
  • storage.multipartUploads.create
  • storage.multipartUploads.list
  • storage.multipartUploads.listParts
  • storage.objects.create
  • storage.objects.delete
  • storage.objects.get
  • storage.objects.list
  • storage.objects.update
  • Provide a unique prefix name for each pipeline.

Read Google's Cloud Storage documentation to learn more about the service.

Connect to Google Cloud Storage

Select destination

  • After you configure the source during a CDC pipeline creation, you will be directed to choose a destination you would like to load your data into.
  • In the Choose destination page, select Google Cloud Storage.

Configure Google Cloud Storage Connection Settings

Specify the following settings in the Connect to Google Cloud Storage destination page:

Field Name Description
Bucket This refers to the name of the Google Cloud Storage bucket you want to use as a destination. Find your bucket name in the Google Cloud Management Console.
Prefix Prefix is the name of the 'folder' or a location within the storage bucket that all CDC data for this pipeline should be saved to. You could have multiple agents using the same bucket with different prefixes.

A pipeline must have one unique prefix.

Test Connection

You can test your connection by using the Test connection button at the bottom; if the test is successful, click Test and Continue.