Manage Extract Profiles

Manage Extract Profiles


This article is specific to the following platforms - Redshift - Snowflake - BigQuery - Synapse.

Overview

This page explains how to create a new extract profile for use with the API Extract component.



Getting Started

Click Project → Manage Extract Profiles.

Now in the Manage Extract Profiles dialog, click to add a new extract profile.

In the Add Extract Profile dialog, specify the name of the new extract profile. Then, click Setup.

The next window is the Configure Extract Profiles dialog. Currently there are no endpoints set up. Click New Endpoint. This action opens the Configure Extract Connector Wizard.



1. Source Details

The first page of the wizard requires basic source information:

Profile Name

This setting has already been set, it is the extract profile name.


Endpoint Name

Specify a name for the endpoint to be created—this is the data source.


Description

Provide a human-readable description of the endpoint for reference. This setting is optional.

Click Next.



2. Endpoint Configuration

The second page of the wizard requires information about the API call you wish to make.

Specify the method. Users can select between GET and POST.

Input the endpoint URI.


Params

Specify any parameters to validate the API call. Users can pass a Parameter Name, Value, Type, and specify whether the parameter is a Constant or not.


Auth

Tick

Then, select Username and Password or API Key from the first dropdown. Input the credentials into the correct fields.

Select between Basic and Bearer in the second dropdown.


Body

Specify, if required, the body of the request.


Response

Here the API call response is displayed. When a user clicks Send, the Response tab is automatically brought into view. The following message is displayed for validated API calls:

JSON File validated successfully with no errors.


Log

A log of activity.


After specifying the method, the URI, and any required elements, click Send. As mentioned above, the Response tab will be displayed. Upon validation, click Next.



3. View Fields

On the third page of the wizard, users can view the structure of the data returned from the API. Matillion ETL will attempt to identify the Repeat Element from the data. However, users can also specify this themselves.

Users can also configure paging by selecting one of the supported paging schemes and providing the required parameters.

Click Next.



4. Review

The final page of the wizard is for reviewing the the configuration of the new endpoint. Publishing the configuration will make the data source immediately available for your API Extract profile.

Click Back to make changes. Otherwise, click Finish.


API Extract

Bring the API Extract component onto the canvas.

Within the component's properties, click into the API property and select your new API extract profile. Then, click into the Data Source property and select the data source (the endpoint name).