Environments

Environments


Overview

An Environment in Matillion describes a single connection to a database, on which Matillion ETL jobs can be run. Numerous Environments can be set up at once, meaning different users can use different Environments on a single Matillion ETL instance. When a user runs a job, the job runs within the Environment currently in use.

Environments can be set up at user-level.

Before starting any job with Matillion ETL, the user must have set up an Environmennt. To begin, click the Project button in the top left of the Matillion ETL window, and then click Add Environment.

Alternatively, in the lower left corner of the Matillion ETL window, right-click on one of your Environments to navigate to the Add Environment button.

Configuring Environments

Upon clicking Add Environment a wizard will open, prompting the user to input vital information regarding the database to be connected to.

  • Environment Name: Describes the Environment. e.g. Test/Live etc
  • AWS Credentials: Choose a set of AWS Credentials Credentials to use with this Environment.
  • GCP Credentials: Choose a set of GCP Credentials to use with this Environment.
  • Azure Credentials: Choose a set of Azure Credentials to use with this Environment.

 

Matillion ETL requires access to different platform-specific resources to complete certain functions and each environment is set up to access these resources. By default, Matillion ETL will attempt to use the instance credentials to discover available resources. However, users can also define their own credentials from within the Matillion ETL client by clicking the 'Manage Credentials' button to open a new window. See Managing Credentials for more information.
 

Next, you are required to give information about your Redshift cluster:

  • Redshift Cluster: If you have created an IAM Role, this box will list your Redshift clusters that are visible to that role in the same region as the Instance is running. Selecting that cluster will populate many of the connection settings fields on the screen with sensible defaults. If the box is empty these fields will need to be set manually.
  • Endpoint: The physical address of the Leader Node. It will be either a name or an I.P. address.
  • Port:  Usually 5439 or 5432, but can be configured differently when setting up your Redshift cluster.
  • Username: Username for the Environment connection.
  • Password: Password for the Environment connection. Users have the option to store their password inside the component; however, we highly recommend using the Password Manager feature instead.
  • Enable SSL: Do this to encrypt communications between Matillion and Redshift. Some Redshift clusters may be configured to require this.

Next, we connect to a database on that cluster:

  • Database:  However you named it when setting up your Redshift cluster. You may run with multiple database names, in which case, choose the one you would like to use for this project here.
  • Default Schema: 'public' by default, but if you have configured multiple schemas within your Redshift database, specify the schema you would like to use here.
  • Default Bucket: Specific the default S3 bucket to use in many Matillion ETL components and features. This does not prevent a bucket being specified in such cases.
  • Concurrent Connections: (Enterprise Mode only.) State the maximum number of database connections to allow concurrently. This can potentially speed up large ETL jobs. The maximum number of connections possible is equal to the number of threads the instance has at its disposal. See our short Video Tutorial for more details.

Click Test to ensure connection is successful. Lastly, click Finish.

 

A setup wizard will appear, prompting the user to input vital information regarding the database to be connected to.

  • Environment Name: Describes the Environment. e.g. Test/Live etc
  • AWS Credentials: Choose a set of AWS Credentials to use with this Environment.
  • GCP Credentials: Choose a set of GCP Credentials to use with this Environment.
  • Azure Credentials: Choose a set of Azure Credentials to use with this Environment.

 

Matillion ETL requires access to different platform-specific resources to complete certain functions and each environment is set up to access these resources. By default, Matillion ETL will attempt to use the instance credentials to discover available resources. However, users can also define their own credentials from within the Matillion ETL client by clicking the 'Manage Credentials' button to open a new window. See Managing Credentials for more information.

  • Account: The name of your Snowflake account. More information about gathering Snowflake account details can be found here.
  • Username: Username for the Environment connection.
  • Password Type: Choose between using a password or a Private Key to authenticate your Snowflake connection.
  • Password: Password or Private Key (as chosen in Password Type) for the Environment connection. Users have the option to store their password or private key inside the component; however, we highly recommend using the Password Manager feature instead.

 

Next, the user creating the Environment will need to set the Snowflake default settings:

  • Default Role: Select a Role. For more information, please consult the Snowflake documentation.
  • Default Warehouse  Select the Environment's Default Warehouse. More...
  • Default Database:  Select the Environment's Default Database. More...
  • Default Schema:  Select the Environment's Default Schema.

Click Test to ensure connection is successful. Lastly, click Finish.

 

A setup wizard will appear, prompting the user to input vital information regarding the database to be connected to.

  • Environment Name: Describes the Environment. e.g. Test/Live etc
  • GCP Credentials: Choose a set of GCP credentials to use with this Environment.
  • AWS Credentials: Choose a set of AWS credentials to use with this Environment.
  • Azure Credentials: Choose a set of Azure credentials to use with this Environment.
  • Default Project: Select the Environment's Default Project.
  • Default Dataset: Select the Environment's Default Dataset.

Click Test to ensure connection is successful. Lastly, click Finish.