Manage Stages

Manage Stages


This article is specific to the following platforms - Snowflake.

Overview

When configuring the Create External Table component, users must manually configure the "Stages" property. This can be done via the Manage Stages feature in Matillion ETL.



Getting Started

In the Environments panel (bottom left of the Matillion ETL interface), right-click on one of your Environments, and then click Manage Stages. The Manage Stages dialog will then appear.

In the Manage Stages dialog, click to add a new stage.



Creating a Stage

1. Create Stage

  • Method: In creating your new stage, choose one of: "Create", "Create if not exists", or "Create or replace".
  • Schema: Select an environment schema for your stage.
  • Name: Give your schema a descriptive name.

Then, click Next.

2. Create Stage Type

  • Type: for the stage to correctly interact with an external table, set the type as External.
  • Platform: the cloud platform (AWS, Azure, or GCP) that hosts the data.
  • S3 URL: (AWS only) select the S3 URL you wish to point the created stage at.
  • Blog Storage URL: (Azure only) select the Azure Blog Storage Container URL you wish to point the created stage at.
  • URL: (GCP only) select the Google Cloud Storage (GCS) URL you wish to point the created stage at.
  • Integration: (GCP only) select a Google Cloud Storage integration. For help setting up a GCS integration, see our Cloud Storage integration setup guide.
  • Credentials: Set your credentials. For help with the "Manage Credentials" feature, click here.
  • Encryption: Decide how the files are encrypted inside the S3 Bucket.
    • Client Side Encryption: Encrypt the data within the Matillion ETL client. Users will also need to provide a Master Key in the following field.
    • SSE S3: Encrypt the data according to a key stored on an S3 bucket
    • SSE KMS: Encrypt the data according to a key stored on KMS. Users will also need to provide a KMS Key ID in the next field.
    • None: No encryption.

Click Finish to create or replace the stage.

Next, click OK to return to the Matillion ETL canvas.

To finish, navigate to the "Stage" property within the Create External Table component, and find your new stage.

Please Note

Staging areas can also be created within Snowflake using the CREATE STAGE command.