Job Variables

Job Variables


This article is part of a series on Variables.


Overview

Variables can be defined within the scope of a single job. These variables can still be Copied and Shared with regard to their behaviour in Fixed Flow component, and Matillion ETL's various iterator components.

  • Job variables will override any environment variables of the same name within that specific job.
  • Job variables are always included in jobs that are imported or exported and are not available for optional inclusion like environment variables are.



Creating Job Variables

To create a job-scoped variable, right click on the desired Orchestration or Transformation Job within the explorer panel and select Manage Variables. This will bring up the Manage Job Variables dialog.

Manage Job Variables

Users can also reach this dialog by right-clicking a component, such as the Start component, and clicking Manage Job Variables.

Manage Job Variables

The Manage Job Variables dialog gives a complete list of all Job Variables with their properties:

  • Name: Specify the name of the variable.
  • Type: Specify the Matillion ETl Data Type of the variable. More info on variables.
  • Behaviour: Set the branch behaviour inside a job. This determines how the variable is updated when more than a single job branch is making use of it. More info on the behaviour of variables.
  • Visibility: Select either Public or Private. When a variable is private, it cannot be discovered or overwritten when this job is called from a Run Orchestration or Run Transformation component.
  • Value: Set the default value for this variable in this job.
 

Creating A Job Variable

Clicking the button lets you edit that variable's description.

This description has no consequence or functionality beyond being present in the dialog for editing the variable and when the job is being called from a Run Orchestration or Run Transformation component.

Variables can also be edited via Text Mode by clicking the checkbox.



Setting Job Variable Values

When a job begins, all variables are initialised with their default value that was set in the Manage Job Variables dialog. The real power of a variable is that its value can be updated. You can update a variable via the following methods:

Iteration Components

Iteration components work by setting variables to a new value for each iteration. This means you must define variables you wish to iterate in advance, and then use them when configuring iteration components. Iterations can be run in parallel, too, and in which case the variable scope must be set to "Copied" to ensure each parallel component sees its own copied copy of the variable.

Python Scripts

Python scripts can push new values into variables using their built-in context object. See the Python component documentation for an example of this.

Warning: Variables become first class variables in Python and Bash scripts. Please avoid naming them in a manner that clashes with key words in either language. We recommend using a prefix (for example, v_) to ensure no such conflicts occur.



Contact Support

If you have any questions about Matillion ETL's job variables, or require assistance with any other aspect of Matillion ETL, please contact our support team.