Matillion ETL has versioning features to help you manage versions of your ETL Transformation and Orchestration Jobs. Versioning supports many use cases, but perhaps the most common use is to capture and freeze your development at a point in time to designate as "live" or "production" versions, and to then continue working on your default version. This article discusses the basic versioning concepts and steps.
- We recommend that you should keep no more than 10 versions per project and only one version per developer on a project.
- If you also use Git integration on a project, you can store a full history of your job changes. Please see this video for more information.
Matillion ETL's versioning system creates a copy of all of the jobs in your current project at that point in time.
All projects have a Default Version, which is the main working version. It is possible to create multiple new versions that branch from your default version at different points in your project's development. Those versions can, in turn, have new versions branched from them. Changes can then be made to either your default version or to any non-default version you have created, as long as the version is in an unlocked state.
- A new version does not include the undo history for the project (although that history is maintained in the version it branches from), but any subsequent changes made in the new version will be tracked.
- Subsequent new versions are based on the currently selected version (see Switching versions), which may be the previous version, the default version, or any other version.
Creating a new version
1. From the Project menu, click Manage Versions to open the Manage Versions dialog.
If you have more than one version listed in the Manage Versions dialog, ensure that the version that you wish to base the new version on is selected before continuing. See Switching versions for details.
2. As you can see in the above example, we currently only have one version of our project, which is the Default Version. To create a new version, click the + button in the lower-left of the dialog. This will open the Create Version dialog.
3. Give your new version a Version Name and, optionally, a Description. The Version Name can only include the following characters: alphanumerics, underscores, single spaces, parentheses, and hyphens. You cannot use a space as the final character in your Version Name.
4. You can choose to lock or unlock your version (see Version locking) by selecting or deselecting the Lock Version checkbox. When the padlock icon is red, the version is locked. When the padlock is green, the version is unlocked. By default, the new version will be locked.
Version locking allows you to protect a version of your project, This feature can prevent accidental changes to important versions such as a "production" or "live" version. The version can be unlocked if you later decide you need to make changes to the job.
When the padlock icon in the Status column of the Manage Versions dialog is red, the version is locked. When the padlock is green, the version is unlocked. See Managing versions for details of how to lock and unlock versions.
When you have created a new version, you will see all versions of the project listed in the Manage Versions dialog.
To switch to the version named Test-Version in the above example, click on the square icon next to Test-Version in the Switch Version column, and then click OK to confirm the switch.
Deleting a version
To delete a version, click the icon next to the version you want to delete in the Manage Versions dialog. Click OK to confirm the deletion.
It is not possible to delete the Default Version.
Use the icon in the Manage Versions dialog to edit a version's properties. You can edit the version name and description, and lock or unlock the version.
- It is not possible to rename the Default Version
- You can edit a version's name and description regardless of whether the version is locked or unlocked.
Using and running a version
Once versions are set up there are a number of places in Matillion ETL where they can be referenced and used:
- Using Queues: When launching from an SQS Queue, specify the version name in the JSON message (see here).
- In the Scheduler: When scheduling jobs (see here), ensure the version is selected.
- Task History: In the Task History (see here), the version is displayed so that you can easily see which version of the project has been run.
- By URL: If you wish to "deep-link" or bookmark a project version, the version can be referenced in the URL:
https://<server name>/#<Group Name>/<Project Name>/<Version Name>