Data Lineage
  • Dark
    Light

Data Lineage

  • Dark
    Light

 

This article version is deprecated

Warning

As of version 1.63.4, the visual lineage tab from Matillion ETL has been deprecated.

For more information, and to find out what the future of lineage in Matillion ETL looks like, read Matillion ETL Supports Data Lineage Tools with Metadata API Integrations from the Matillion blog.

Overview

Data Lineage is an Enterprise Mode feature that lets users to track columns across transformations to understand how that column has been affected by components up to the point of inspection.

Data Lineage can be accessed via the Data Lineage tab, available after selecting a transformation component on the job canvas.


Example

We could sample a component and see the current values that each columns holds. However, identifying where that column has come from, and the journey it has taken, can be difficult to determine, especially if you are not the creator of the job.

By selecting a component and browsing to the Lineage tab, we can attain the lineage for the data as per the conclusion of this component.

A user can check the lineage of a Calculator component, which yields a number of columns. Clicking one of the columns, projectid, we can see the lineage of this column on the right-hand side. Using the arrows to expand the tree, we can follow the PROJECTID column through different components. Each entry is given in the format:

<ComponentName or TableName>.<ColumnName>

We see that this column originated in the table DOC_TBL and has been brought in by a Table Input component, passed through a Convert Type component, and finally ended up in the Calculator component that we are inspecting.


Video