Git Integration: When To Choose Git

Git Integration: When To Choose Git


Overview

This guide is designed to help users determine whether the Git integration feature for Matillion ETL is suitable for their use case.

Git is a distributed version control system that lets software development teams track a project's changes (both small and large) with speed and efficiency.

Important Information and Links

This article is part of a series of technical documentation covering the Git integration feature within Matillion ETL. Additional documentation includes:



When To Choose Git

Note: Git is an "enterprise" feature within Matillion ETL. To learn more about Matillion's Enterprise Mode and how to upgrade your licence, please click here.

Matillion ETL lets users convert their Matillion Project into a Git project, and use Matillion's Versions management system as a network of independent working areas, centralised via the Git feature, where users can perform actions including Commit, Create Branch, Configure Remote Repository, Fetch, and Push. Each Matillion Version within a Git project points at a specific Git commit.

Choosing Matillion ETL's Git support enables users to:

  • Perform branch-focused software development with Matillion jobs - typically adhering to common software engineering workflows.
  • Move Matillion ETL jobs between projects - for example, between Development, Testing, and Production projects.
  • Commit code to a remote Git repository, such as GitHub, AWS CodeCommit, or Bitbucket.



Suggested Requirements

If the below-listed situations match a user experience of Matillion ETL, we believe the Git feature may be appropriate:

  • Where the user's organisation already mandates the use of Git in software engineering.
  • Where an organisation's Matillion ETL user(s) already have experience with a version control system such as Git.
  • Where an organisation wishes to use an off-site or remote Git repository to externally backup their code.



Considerations

Git is a flexible and complex tool, and teams opting to use Matillion ETL's Git support should establish a workflow that satisfies their specific use case. We have designed an example of a typical workflow for using Git support in Matillion ETL.

Users should also ensure they have access to an external repository, as mentioned above, for example, GitHub. Users should also be aware that actions involving managing the workflow, managing the external repository, and managing user permissions will be required.

Note: Matillion ETL's Git support only allows storage of versionable items, not environmental items.

Versionable items (can be stored in Git):

  • Orchestration Jobs
  • Transformation Jobs

Environmental items (cannot be stored in Git):



Alternatives to using Git Support in Matillion ETL

For many users, integrating Git into their Matillion ETL projects may be offset by simpler alternatives.

Matillion has a built-in versioning system, which enables users to conduct basic version control such as tagging releases and tracking changes. Read more about our Manage Versions feature.

Matillion also has a server migration tool, which lets users move their ETL jobs and other collateral, such as OAuths, between Matillion projects without having to adopt an external Git service. Please see our Migrate feature documentation for further information.

Users can interact with the Matillion API to automate moving jobs between projects, and perform additional environment configuration. Users may find this feature suitable if: they have a pre-existing external scheduler for their jobs; if they need to integrate the deployment of Matillion ETL within a wider automated progress; or if the user wishes to design a bespoke process. Please see our Matillion API documentation for more information.

This support article is the first in a series of technical documentation focusing on Matillion ETL's Git support. For more information on using Git within Matillion ETL, including the underlying architecture and technical terms, read this article next.