Contents x
Mandrill Extract
  • Dark
    Light
Contents

Mandrill Extract

  • Dark
    Light
This article is specific to the following platforms - Snowflake - Redshift - BigQuery.

Mandrill Extract

The Mandrill Extract component calls the Mandrill API to retrieve and store data to be either referenced by an external table or loaded into a table, depending on the user's cloud data warehouse. Users can then transform their data with the Matillion ETL library of transformation components.

Using this component may return structured data that requires flattening. For help with flattening such data, we recommend using the Nested Data Load Component for Amazon Redshift and the Extract Nested Data Component for Snowflake or Google BigQuery.


Properties

Snowflake Properties

Property Setting Description
Name String A human-readable name for the component.
Data Source Select Select a data source from a list. Returned columns and Component Properties are dependent on the data source chosen.
Key Password Enter an API Key to connect your Mandrill App. See the 3rd Party OAuth documentation for a guide on how to acquire an API Key. We recommend storing API Keys in the Password Manager.
Query String (Data Source: Messages Search) Enter a string to match to queried messages. Only rows that include the string will be returned.
Date From String (Data Source: Messages Search) A date to query messages from. Format of YYYY-MM-DD
Date To String (Data Source: Messages Search) A date to query messages up to. Format of YYYY-MM-DD
Tags Text Multiple (Data Source: Messages Search) Enter tags for the query to be filtered by.
Senders Text Multiple (Data Source: Messages Search) Enter the names of Senders for the query to be filtered by.
API Keys Text Multiple (Data Source: Messages Search) Enter API Keys for the query to be filtered by.
Limit Integer (Data Source: Messages Search) The maximum number of messages to return from a query.
Note: A maximum of 1000 results can be returned at a time due to a limitation of the Mandrill API. To load larger amounts of data, we recommend iterating the component using "Date From" and "Date To" to page results.
Location Storage Location Provide an S3 bucket path, GCS bucket path, or Azure Blob Storage path that will be used to store the data. Once on an S3 bucket, GCS bucket or Azure Blob, the data can be referenced by an external table. A folder will be created at this location with the same name as the Target Table.
Integration Select (GCP only) Choose your Google Cloud Storage Integration. Integrations are required to permit Snowflake to read data from and write to a Google Cloud Storage bucket. Integrations must be set up in advance of selecting them in Matillion ETL. To learn more about setting up a storage integration, read our Storage Integration Setup Guide.
Warehouse Select Choose a Snowflake warehouse that will run the load.
Database Select Choose a database to create the new table in.
Schema Select Select the table schema. The special value, [Environment Default], will use the schema defined in the environment. For more information on using multiple schemas, see this article.
Target Table Text Provide a new table name.
Warning: This table will be recreated and will drop any existing table of the same name.

Redshift Properties

Property Setting Description
Name String A human-readable name for the component.
Data Source Select Select a data source from a list. Returned columns and Component Properties are dependent on the data source chosen.
Key Password Enter an API Key to connect your Mandrill App. See the 3rd Party OAuth documentation for a guide on how to acquire an API Key. We recommend storing API Keys in the Password Manager.
Query String (Data Source: Messages Search) Enter a string to match to queried messages. Only rows that include the string will be returned.
Date From String (Data Source: Messages Search) A date to query messages from. Format of YYYY-MM-DD
Date To String (Data Source: Messages Search) A date to query messages up to. Format of YYYY-MM-DD
Tags Text Multiple (Data Source: Messages Search) Enter tags for the query to be filtered by.
Senders Text Multiple (Data Source: Messages Search) Enter the names of Senders for the query to be filtered by.
API Keys Text Multiple (Data Source: Messages Search) Enter API Keys for the query to be filtered by.
Limit Integer (Data Source: Messages Search) The maximum number of messages to return from a query.
Note: A maximum of 1000 results can be returned at a time due to a limitation of the Mandrill API. To load larger amounts of data, we recommend iterating the component using "Date From" and "Date To" to page results.
Location Text/Select Provide an S3 Bucket path that will be used to store the data. Once on an S3 bucket, the data can be referenced by an external table. A folder will be created at this location with the same name as the Target Table.
Type Dropdown Select between a standard table and an external table.
Standard Schema Dropdown Select the Redshift schema. The special value, [Environment Default], will use the schema defined in the Matillion ETL environment.
External Schema Select Select an External Schema that will hold your new External Table.
Target Table Text Provide a name for the External Table to be used.
Warning: This table will be recreated and will drop any existing table of the same name.

BigQuery Properties

Property Setting Description
Name String A human-readable name for the component.
Data Source Select Select a data source from a list. Returned columns and Component Properties are dependent on the data source chosen.
Key Password Enter an API Key to connect your Mandrill App. See the 3rd Party OAuth documentation for a guide on how to acquire an API Key. We recommend storing API Keys in the Password Manager.
Query String (Data Source: Messages Search) Enter a string to match to queried messages. Only rows that include the string will be returned.
Date From String (Data Source: Messages Search) A date to query messages from. Format of YYYY-MM-DD
Date To String (Data Source: Messages Search) A date to query messages up to. Format of YYYY-MM-DD
Tags Text Multiple (Data Source: Messages Search) Enter tags for the query to be filtered by.
Senders Text Multiple (Data Source: Messages Search) Enter the names of Senders for the query to be filtered by.
API Keys Text Multiple (Data Source: Messages Search) Enter API Keys for the query to be filtered by.
Limit Integer (Data Source: Messages Search) The maximum number of messages to return from a query.
Note: A maximum of 1000 results can be returned at a time due to a limitation of the Mandrill API. To load larger amounts of data, we recommend iterating the component using "Date From" and "Date To" to page results.
Table Type Select Select whether the table is Native (by default in BigQuery) or an external table.
Project Select Select the Google Bigquery project. The special value, [Environment Default], will use the project defined in the environment.
For more information, refer to the BigQuery documentation.
Dataset Select Select the Google Bigquery dataset to load data into. The special value, [Environment Default], will use the dataset defined in the environment.
For more information, refer to the BigQuery documentation.
Target Table String A name for the table.
Warning: This table will be recreated and will drop any existing table of the same name.
Only available when the table type is Native.
New Target Table String A name for the new external table.
Only available when the table type is External.
Cloud Storage Staging Area Cloud Storage Bucket Specify the target Google Cloud Storage bucket to be used for staging the queried data. Users can either:
  1. Input the URL string of the Cloud Storage bucket following the template provided: gs://<bucket>/<path>
  2. Navigate through the file structure to select the target bucket.

Only available when the table type is Native.
Location Cloud Storage Bucket Specify the target Google Cloud Storage bucket to be used for staging the queried data. Users can either:
  1. Input the URL string of the Cloud Storage bucket following the template provided: gs://<bucket>/<path>
  2. Navigate through the file structure to select the target bucket.
Only available when the table type is External.
Load Options Multiple Select Clean Cloud Storage Files: Destroy staged files on Cloud Storage after loading data. Default is On.
Cloud Storage File Prefix: Give staged file names a prefix of your choice. The default setting is an empty field.
Recreate Target Table: Choose whether the component recreates its target table before the data load. If Off, the component will use an existing table or create one if it does not exist. Default is On.
Use Grid Variable: Check this checkbox to use a grid variable. This box is unchecked by default.
What's Next

Cookie consent

We use our own and third-party cookies to understand how you interact with our knowledge base. This helps us show you more relevant content based on your browsing and navigation history.