UI And Basic Functions

UI And Basic Functions

UI and Basic Functions

It is quick and easy to start using Matillion ETL. In spite of many functions and options being available, new users can be running simple tasks in minutes. Here we cover the basics of Matillion ETL for new users while providing links for further reading on each topics.

Below you will find an explanation of the UI and its elements along with explanations of some common terminology you will come across in other documents and the tool itself. You will need a working instance of Matillion ETL to follow the instructions on this page.
Please refer to the installation guide if you require help setting up Matillion ETL.
Please refer to the installation guide if you require help setting up Matillion ETL.

UI Layout

1. Navigation Panel.
2. Canvas.
3. Components Menu.
4. Component Tabs.
5. Task Tabs.
6. Drop down menu for Project options.
7. Admin.
8. Help.


1. Navigation Panel

Matillion ETL runs as a 'job' created by the user. Each job is a series of tasks that are each represented by a 'component'. Many components can be linked together to tackle more sophisticated problems and such a collection of components can be thought of as a single 'job'. There are two kinds of job - Orchestration and Transformation.

An Orchestration job is generally one that deals with loading and unloading data to tables and file storage. This is also known as staging the data. 
A Transformation job is generally one that manipulates data in the form of tables.

Each Matillion ETL project can have as many jobs as desired and they can be navigated to through this panel. Double clicking a job will open it into the Canvas. Right clicking the Navigation Panel will allow the user to create a new job while right clicking on an existing job will allow the user to edit or delete that job.

Each type of job has its own set of components (Orchestration and Transformation components) that can only be used within their own respective job types. It is also possible to link Transformation jobs from Orchestration jobs, allowing large pipelines to be planned and executed using just a single job.

2. Canvas

The Canvas shows the contents of a selected job. Each opened job will be represented by a tab at the top of the Canvas and can be switched between by clicking on the tabs, without disruption to other open jobs.  The Canvas will be empty when creating a new job and eventually filled with components by the user.

The Canvas can be populated by selecting the desired component from the Components Menu and dragging them into the Canvas. Each component will need to be linked to others to indicate the order in which each is executed. By selecting a single component in the Canvas, its connection points will become visible, displayed as small circles ('Nubs') beside the component.

These Nubs generally come in 4 different colours:

Green (Success): Continue if this component ran successfully.
Red (Failed): Continue if this component failed during its run.
Grey (Unconditional): Continue regardless of success/failure.
Blue (Iterate): Connect to another component (this is with regard to iterator components).

Clicking and dragging on these circles will extend a link that can be clipped into another component. Green output links indicate the origin component has executed successfully while red output links indicate the origin component has unsuccessfully executed. Thus, the user is able to direct the job according to whether the current run is succeeding or not.

Note that all Orchestration jobs must begin with a Start Component and thus, the Start Component cannot receive links from other components.

Since a job can become large and complicated with many components, users can create Notes on the Canvas. Each note consists of a box that can be resized - usually to encompass multiple components that the note refers to - and be edited to contain text. Right clicking anywhere on the Canvas will allow users to add a note. Notes are a good way of grouping components together to make sense of large workflows.

3. Components Menu

The Components Menu is a collapsible list of all components available to the currently-selected job type. Each component embodies a particular function from moving data into file storage to performing custom calculations on table columns. Moving to a different type of job will automatically update the available components such that it is impossible to place a Transformation component into an Orchestration job or vice versa. A search field at the top of the Components Menu allows the user to make a search for specific components (matches component name only).

Clicking on a component and dragging it into the Canvas will create a component of that type to be used in the current job. There is no limit to the number of components, total or of a specific type, that the user can apply to a job.

Components are named according to their function, although it is common for a single component to have many functions that are not strictly related to one another. Thus, it is recommended that users acquaint themselves with the available components such that they can make informed decisions on which components to use to best achieve a given goal.

A list of Data Stagers (Query Components) can be found Data Staging Components.

4. Component Tabs

Clicking on a component in the Canvas will make available that component's details in the Component Tabs. Each component generally shared the same set of tabs but with different contents in each. Common tabs are as follows:
  • Properties: A set of user-defined properties that effect the function of this component. Every component has a unique set of properties and changing one component's properties will not change another, even if the two components are of the same type. The properties for a given component are explained in full in each component's documentation.
  • Export: The user can choose to export certain values from this component as a variable to be used elsewhere. To learn more, see the Variables documentation.
  • Sample: The Sample tab allows the user to take sample output from many components up to a specified number of rows. Highly useful for confirming a component is set up correctly before using it in a live job.
  • Metadata: Shows data for each column featured in the sampled data (Sample tab). This is only available in Transformation Components. Data includes column names, types, sizes and scales.
  • SQL: Shows the user what the SQL equivalent of this component and its current setup is.
  • Plan: Allows the user to get the plan from the SQL query generated by the job up to (and including) the selected component.
  • Help: Shows a brief Help file for the selected component, explaining its function and properties. At the bottom of each help file is a link to the full documentation, including examples of how to use the component.

5. Task Tabs

The Task Tabs show information about tasks that have been run during the current session. Right clicking anywhere in the Canvas, the user has the option to run the current job as it currently stands. Right clicking on a component will also allow the user to run just that single component. In both cases, the run takes the form of a task that is documented in the Task Tabs. For more information on tasks, see documentation for Task Management. Task Tabs are as follows:
  • Tasks: Shows up to the last 20 tasks attempted by the user on this session of the project. Queued Tasks are included, as are tasks that fail or are cancelled. Clicking on a task will expand the selection with further information including error messages where appropriate. See Task Management.
  • Search: Searches the current project for matching components. See Search Tab.
  • Console: Displays the instance console.
  • Command Log: An archive of interactions between your current browser session and the Matillion ETL server.
  • Notices: Alerts for the user concerning their Matillion ETL project. This includes notifications for updates to Matillion ETL.
  • Chat: A simple chat client shared between you and other users collaborating on your project.

6. Project Menu

  A list of options regarding the current Project. Options are listed as follows:

  • Switch Project: Browse to or create the desired project. Jobs inside projects can also be opened from here, automatically switching to their parent project.
  • Manage Project: Set name, description and permissions for sharing the current project. Also see Project Sharing.
  • Rename Project Group: The project group is simply a name that is associated with multiple projects and can be renamed as the user pleases.
  • Delete Project: Delete the current project.
  • Input Data Report: Generates a brief report on all jobs that have been run in this session within the current project.
  • Create Version: Creates a 'version' of the current project, preserving the current state of its jobs. See Version Control for more...
  • Switch Version: Switch between created versions of the project. See Version Control for more...
  • Manage Version: Allows the user to change the current version's details. Users can only rename the current version if more than one version is available. The default version and locked versions cannot be renamed. See Version Control for more...
  • Delete Version: Delete the current version. The default version and locked versions cannot be deleted. See Version Control for more...
  • Add Environment: Add an environment. See Managing Environments for more...
  • Manage Environment Variables: Opens a window that allows the user to define variables for the current environment. See Using Variables for more...
  • Export: Export selected jobs, environments and variables that can later be imported into other projects. Exports as a .json file.
  • Import: Reads a .json file from which the user can select jobs, environments and variables to import into the current project.
  • Manage Credentials: Define credentials that Matillion ETL uses to integrate with the instance environment. See Managing Credentials for more...
  • Manage Schedules: Allows the user to schedule Matillion ETL jobs. See Manage Schedules for more.
  • Task History: Shows a list of completed (successful or failed) tasks that can be selected and expanded for further details. See Task Management for more...
  • Load Status: Shows the status of any data being currently loaded into tables on the current Matillion ETL instance.
  • Manage OAuth: Define OAuth for various APIs from which Matillion ETL can source data. See Manage OAuth for more.
  • Manage Passwords: Allows the user to define username-password pairs that are stored for use in different parts of Matillion ETL.
  • SQS Configuration: Defines Matillion ETL's configuration with Amazon's Simple Queue Service (SQS). See Integration with Amazon SQS for more...
  • Manage Database Drivers: Allows the user to upload database drivers and tell Matillion ETL to use said drivers. See Database Driver Management for more...
  • Manage API Profiles: Allows the user to define profiles that describe JSON or XML API for importing data into tables. For more information, see Matillion ETL API and Writing API Profiles.
  • Manage Backups: Allows the user to schedule automatic backups of the instance. See Automated Backups for more...

7. Admin Menu

A list of options that allow any user with Admin privileges to administrate the Matillion ETL instance. More details on the administrative options below can be found here.

  • Download Server Log: Download Tomcat Catalina logs in plain text.
  • Matillion ETL Updates: Opens a dialog box that allows you to check for updates and install any found.
  • User Configuration: Manage user privileges and access rights.
  • SSL: Configure the Matillion ETL instance to listen for HTTP and HTTPS requests.
  • Restart Server: Restart the Matillion ETL instance.
  • Manage Backups: Configure automatic backups and test backup schedules.
  • Audit: Views a log of all activity on the Matillion ETL instance.

8. Help Menu

The Help drop-down menu lists general useful information about Matillion ETL.

  • Get Support: Opens a new browser window with contact details and information on Matillion ETL support.
  • Support Information: Provides the user with a template to send advanced enquiries to Matillion ETL support. See Getting Support.
  • License: Allows the user to manage Matillion ETL licenses.
  • About: Displays information on the current Matillion ETL version.
Beside the help menu button, current collaborators (users in the same project on the same instance) are listed in square boxes. Hovering one of these boxes will display that user's current job status.


Keyboard Shortcuts

Common windows keyboard shortcuts can be used inside Matillion ETL. A reminder of these can be found by right-clicking the canvas and consulting the menu (shown below).

CTRL + C - (Copy) : Copy a selected component, job or text to be pasted elsewhere.

CTRL + V - (Paste) : Paste whatever has been last stored through Copy or Cut.

CTRL + Z - (Undo) : Revert the job layout to the state before its latest change. Can done multiple times to revert back to any state within the current session. Note this only applies to the canvas and will not undo changes such as job deletions.

CTRL+ Y - (Redo) : Reapplies the last undone change. Thus,can only be used after Undo. Can be used multiple times until the latest version is reached.

CTRL+ A - (Select All) : Instantly selects all components, notes and links on the Canvas.

CTRL+RETURN (Run Job/Close Dialog) : If on the canvas, this will attempt to run the current job. If a dialog box is open, this will confirm and close the box.

CTRL+SHIFT+R: Open a searchable dialog listing all jobs in the current project.

CTRL+SHIFT+T: Open the Search Tab panel.

Finalising your job

Jobs are automatically saved with each change so there is no need to manually save the progress. Jobs can be closed and reopened at any time to continue editing and there is no 'final' state for a job where it cannot be further iterated upon. A job can be run by simply right-clicking on the job and selecting 'Run'. Jobs can also be scheduled to run at a future time to queued behind other jobs. For more information, see documentation on Manage Schedules .