Launching Matillion ETL for Snowflake - AWS
Matillion ETL is an AMI-based ETL/ELT tool built specifically for Snowflake. Modern, browser-based UI. Powerful, push-down ETL/ELT. This page describes how to launch and connect to Matillion ETL from the AWS Marketplace.
If you have already completed these steps and wish to connect Matillion ETL to Snowflake, please refer to the documentation here...
Selecting Matillion ETL from the AWS Marketplace
To launch and configure Matillion ETL, you should select it from the AWS Marketplace and start the launch/setup procedure. To do this, use the following steps:
- Locate 'Matillion ETL for Snowflake' on the AWS Marketplace
- From the Matillion ETL product page on the AWS Marketplace, press the yellow Continue to Subscribe button.
- Depending on your account, you may also need to select Continue to Configuration on the next screen.
- On the Configure this software page, select Amazon Machine Image with a 64-bit Amazon Machine Image (AMI) as your Fulfillment Option.
- It is highly recommended that users always select the most up-to-date product version available and in a region which they (or their current AWS services) reside in.
- Click Continue to Launch to continue the setup. Note that in spite of the name, clicking this will not yet launch your instance or charge your account.
Launch this software
On the Launch from software screen, simply select Launch through EC2 on the Choose Action dropdown and then click Launch.
Again note that this simply continues with the launch setup and will not yet launch the instance or charge your account.
Choosing the Instance Type
Important: The instance type you select affects how many users can use Matillion ETL concurrently. For more information see Choosing an instance size.
Choose one of the supported Instance Types. Each Instance Type is appropriately sized to support a given number of users and the software itself recognises the Instance Type it is running on and restricts maximum concurrent users on this basis.
- For teams of 1-2 data professionals using Matillion ETL concurrently, choose t3.medium
- For teams of 3-5 data professionals using Matillion ETL concurrently, choose m5.large
- For teams of 6-12 data professionals using Matillion ETL concurrently, choose m5.xlarge
- For production instances that run jobs but need few users, choose an r5 type instance.
You can read more about Instance Types and pricing at here.
Choose the instance size you require, then click Next: Configure Instance Details.
Important: DO NOT click the blue “Review and Launch” button yet, as there are options you will want to configure on later screens.
Configure Instance DetailsThis screen is why we’ve used Manual Setup instead of 1-click setup. There are some settings you can specify on this screen, unavailable through 1-click, that improve the functionality of your Matillion Instance.
We advise you configure this screen as follows:
- Number of instances – leave as default (unless you want multiple instances)
- Purchasing option – leave as default i.e. unchecked
- Network – Choose a VPC. The zone which this VPC is located should ideally be the same as your Snowflake account (European Snowflake accounts should be paired with a EU-based AWS region, for example).
- Auto-assign public IP – This depends very much on the setup, by default a new VPC will not have VPN connections or NAT Gateways available, so in order for Matillion ETL to connect to the internet and for you to access Matillion this will normally need to be set to Enable.
- IAM Role – Follow instructions below:
IAM Roles are used to allow your Instances to use the Amazon API's securely without manual management of security keys. To use all the features of Matillion ETL, you should configure an IAM Role for your instance to use. This procedure assumes you do not already have an appropriate IAM role setup (if you do, simply select it)
- Click Create New IAM Role
- Click the blue Create New Role button
- Enter a suitable name into the Role Name field e.g. Matillion_ETL_IAM_Role
- Choose Amazon EC2 as the role type, by pressing the select button on this row
- From the list select
- Click Next Step. For more information on the policies see Managing Credentials.
- Finally click the Create Role button.
Returning to the Configure Instance Details screen, you can now select the IAM role you have just created.
To finish configuring the Configure Instance Details, choose settings as below for Shutdown behaviour, Enable termination protection and Monitoring:
- Shutdown behaviour – leave as default i.e. Stop
- Enable termination protection – we suggest setting to, ‘Enabled’
- Monitoring – we suggest setting to, ‘Enabled’
- Tenancy - Run a shared hardware instance
- Click, Next – Add Storage
- You can choose the default root volume size here however for production implementations we would recommend increasing this by a factor of 5
- Click, Next - Tag Instance
- Add any instance tags your require
- Click, Next – Configure Security Group
- A default security group will be created with the minimum set of ports. If required, name and adjust as to your security requirements.
- The default recommended security group uses SSH (port 22) and HTTP(S) (port 80 and 443) access to the instance. The range of allowed IPs should be tailored to your needs.
You are now ready to Review and Launch your Matillion ETL for Snowflake AMI.
Once the AMI has initialised, which normally takes a few minutes, you can access Matillion ETL by entering the hostname or IP of the instance into a web browser.
Log in to your copy of Matillion with ec2-user and the instance ID i-xxxxxxxx (e.g. i-88ed92c6)