Add Pipeline
  • Dark
    Light

Add Pipeline

  • Dark
    Light

Add a Pipeline

A Pipeline is a user-defined configuration within Matillion Data Loader that describes how a set of data is taken from a Data Source and loaded into a target Destination. Pipelines are saved after creation and run according to a schedule. Matillion Data Loader will collect metrics based on these scheduled runs so users can easily monitor how their Pipelines are performing.

Creating a pipeline is a simple process and consists of the following stages:

To begin adding a new Pipeline, select the Add Pipeline button while in the All Pipelines tab.

This will launch the Add Pipeline wizard.

Add Pipeline Wizard

1. Select a Data Source and click Next. Each Pipeline supports only a single Data Source at a time.

data-loader-warehouse

2. Choose a Destination Warehouse from the list.

3. Next, select a Destination Config from the drop-down menu. If you have not already configured a destination, you will need to create one now. Click Manage to open the Manage Destinations dialog and click Add to create a new configuration. The process for creating a configuration for each destination is covered in the following topics:

4. Enter a Pipeline Name, which should be unique and descriptive.

5. Finally, click Next when you are happy to proceed. The next stages of the wizard are specific your chosen data source. Locate your chosen data source from the list of Data Sources and follow the configuration instructions. When you have completed the configuration for your data source, the pipeline is active and created.


What's Next