Manage Stages
  • Dark
    Light

Manage Stages

  • Dark
    Light

This article is specific to the following platforms - Snowflake.

Overview

When configuring the Create External Table component, users must manually configure the Stages property. This can be done via the Manage Stages feature in Matillion ETL.



Getting Started

1. In your Matillion ETL instance, click the Environments panel, situated in the lower-left of the UI, then right-click on one of your environments. Click Manage Stages. The Manage Stages dialog will open.

2. Click to create a new stage. The Create Stage dialog will open.



Creating a Stage

1. Page 1 of Create Stage requests metadata about the stage.

  • Method: Use the drop-down menu to choose one of the following methods: Create, Create if not exists, or Create or replace.
  • Schema: Use the drop-down menu to select an environment schema for your new stage.
  • Name: Give your schema a descriptive name.

Then, click Next.

2. To create a Stage Type, use the drop-down menus to select the following options. Certain options are specific to each platform.

  • Type: Set the type as External for the stage to correctly interact with an external table.
  • Platform: the cloud platform (AWS, Azure, or GCP) that hosts the data.
  • Authentication: (AWS and Azure only) select the method you want to use to access your chosen external table. If you select Credentials, refer to Credentials. if you select Storage Integration, refer to Integration.
  • S3 URL: (AWS only) select the S3 Container URL you wish to point the created stage at.
  • Blob Storage URL: (Azure only) select the Azure Blob Storage Container URL you wish to point the created stage at.
  • URL: (GCP only) select the Google Cloud Storage (GCS) Container URL you wish to point the created stage at.
  • Integration: Select a platform-specific Cloud Storage integration. For more information, read Cloud Storage Integration Setup Guide.
  • Credentials: Select your credentials. Click Manage to open the Manage Credentials dialog, where you can set roles and permissions on your hosting platform of your Matillion ETL instance.
  • Encryption: (AWS and Azure only) decide how the files are encrypted inside the S3 and Blob Storage bucket, respectively.
    • Client Side Encryption: Encrypt the data within the Matillion ETL client. Users will also need to provide a Master Key in the following field.
    • S3 Encryption: Encrypt the data according to a key stored on an S3 bucket.
    • KMS Encryption: Encrypt the data according to a key stored on KMS. Users will also need to provide a KMS Key ID in the next field.
    • None: No encryption.

3. Click Finish to create or replace the stage.

4. The new stage will be listed in the Manage Stages dialog. Click OK to return to the Matillion ETL canvas.

5. To finish, navigate to the Stage property within the Create External Table component, and find the new stage.

Please Note

Staging areas can also be created within Snowflake using the CREATE STAGE command.



Deleting a Stage

In the Manage Stages dialog you can remove a stage you have previously created. To do this:

  1. Click for the corresponding stage.
  2. Click Yes in the Confirm Deletion dialog.


What's Next