Contents x
API v1 - Git Integration
  • Dark
    Light
Contents

API v1 - Git Integration

  • Dark
    Light

Overview

This article is a guide to using the source control management (SCM) API in Matillion ETL. Matillion ETL uses Git for its version control, allowing teams to track changes (both small and large) to a project or a shared job with speed and efficiency.

Note

  • Consider whether Git integration is appropriate for your Matillion ETL practices with our When to Choose Git article.
  • Git integration
  • Git FAQ
  • If you are new to Matillion ETL's API, read our Matillion API introduction.
  • Explore the v1 API map.
  • To jump to the v1 API, just run {host:port}/rest/v1 in your browser.
  • Git integration in Matillion ETL is an Enterprise Mode feature. Read about the benefits of upgrading your license to Enterprise Mode here.
  • All information in <angle brackets> should be replaced with information based on your own configuration.
  • The API has a different set of endpoints for working with (a) projects and (b) shared jobs. The examples in this article are for use with projects, with the endpoints needed for use with shared jobs listed in the table at the end of the article.

Initialize A Local Repository

To initialize a local repo using the Matillion v1 API, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/scm/init

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Project/project/name/Docs/version/name/default/scm/init

The variables in the above example are listed immediately below for added clarity. For security reasons, the instance address remains hidden.

  • groupName = Docs_Project
  • projectName = Docs
  • versionName = default

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), a description of the endpoint, and any pertinent notes that users should consider.

Method Body (JSON) Description
POST 
{
"commitMessage": "<Your Commit Message>",
"userIdentifier": {
"username": "<username>",
"emailAddress": "<emailaddress>"
}
}
Initializes a new repository in this Matillion ETL version.

This request creates an initial Git commit within Matillion ETL.
  • commitMessage: A message to include with your commit. This will be visible on the commit inside the Matillion ETL client
  • username: The username to attach to the commit. Note this is not necessarily your username for the remote repository nor your Matillion ETL instance username and does not affect anything else.
  • emailAddress: The email address to attach to the commit (xxx@xxx.xxx). Note this is not necessarily your email for the remote repository nor your Matillion ETL instance email and does not affect anything else.

Clone Remote Repository

To clone a remote repository, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/scm/clone

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/version/name/default/scm/clone

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), a description of the endpoint, and any pertinent notes that users should consider.

Method Body (JSON) Description
POST 
{
"remoteURI":"<URL>",
"auth": {
"authType": "HTTPS",
"username": "<username>",
"password": "<password>"
}
}
Clone an existing remote repository in this Matillion ETL version.

Successful requests will initialise from or clone your remote repository.
  • remoteURI: The URL of your remote Git repository
  • authType: Matillion ETL currently supports "HTTPS" only
  • username: The username to authenticate with the remote repository
  • password: The corresponding password for the above username

Commit to Local Repository

To commit changes to a local repo using the Matillion v1 API, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/scm/commit

Here is an example of this request with example variables added (for security, the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Project/project/name/Docs/version/name/default/scm/commit

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

SCM commit endpoint

The table below documents the type of API endpoint, the body (in a JSON format), a description of the endpoint, and any pertinent notes that users should consider.

Method Body (JSON) Description
POST 
{
  "branchName":"<branch>",
  "changes":[{"type":"ADDED","filePath":"<path>"}],
  "commitMetadata":{"commitMessage":"<Your Commit Message>","userIdentifier":{"username":"<username>","emailAddress":"<email address>"}}
}

Commit changes to a shared repository.

  • branchName: The repository branch that the changes will be committed to. E.g. "default".
  • filePath: A valid filepath to the changed objects being committed.
  • commitMessage: A message to include with your commit. This will be visible on the commit inside the Matillion ETL client.
  • username: The username to attach to the commit. Note this is not necessarily your username for the remote repository, nor your Matillion ETL instance username, and does not affect anything else.
  • emailAddress: The email address to attach to the commit (xxx@xxx.xxx). Note this is not necessarily your email for the remote repository, nor your Matillion ETL instance email, and does not affect anything else.


Switch Commit

To switch to another Git commit using the API, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/scm/switchCommit

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/version/name/default/scm/switchCommit

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), and a description of the endpoint.

Method Body (JSON) Description
POST 
{
"commitID": "<ID>"
}
Switch to an existing commit by passing the commit ID. You can find commit IDs by using the 'getState' endpoint detailed in the 'Get Commits' section of this article.
 
  • commitID: The commit ID of the commit to switch to


Fetch from Remote Repository

To perform a fetch from a remote repository, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/scm/fetch

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/scm/fetch

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), a description of the endpoint, and any pertinent notes that users should consider.

Method Body (JSON) Description
POST 
{
"auth": {
"authType": "HTTPS",
"username": "<username>",
"password": "<password>"
},
"fetchOptions": {
"removeDeletedRefs": "<true/false>",
"thinFetch": "<true/false>"
}
}
Fetch from a remote repository.

A remote repository must be configured before fetch requests can be made.
 
  • authType: Matillion ETL currently supports "HTTPS" only
  • username: The username to authenticate with the remote repository
  • password: The corresponding password for the above username
  • removeDeletedRefs: Choose to remove deleted references ("true" or "false")
  • thinFetch: Reduces the data sent when the sender and receiver share many of the objects ("true" or "false")


Push to a Remote Repository

To push to a remote repository, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/scm/push

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/scm/push

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), and a description of the endpoint.

Method Body (JSON) Description
POST 
{
"auth": {
"authType": "HTTPS",
"username": "<username>",
"password": "<password>"
},
"pushOptions": {
"atomic": "<true/false>",
"forcePush": "<true/false>",
"thinPush": "<true/false>"
}
}
Push to a remote repository.
 
  • authType: Matillion ETL currently supports "HTTPS" only
  • username: The username to authenticate with the remote repository
  • password: The corresponding password for the above username
  • atomic: Guarantees that either all references will be pushed on the remote, or none of them will; this option avoids partial pushes. ("true" or "false"),
  • forcePush": Forces the local revision to be pushed into the remote repository. This action can cause the remote repository to lose commits, and should be used with caution.("true" or "false")
  • thinPush": Reduces the data sent when the sender and receiver share many of the objects. ("true" or "false")


Configure a Remote Repository

To configure a remote repository, make the following POST call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/scm/remote

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/scm/remote

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), and a description of the endpoint.

Method Body (JSON) Description
POST 
{
   "url": "{your_remote_uri}"
}
Configure a remote repository.
  • url: The URL of your remote Git repository.


Get Remote (returns URL)

To return the URL of your remote repository, make the following GET call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/scm/remote

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/scm/remote

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:


Get Commits

To retrieve a list of each Git commit on the local repository, make the following GET call:

<instance address>/rest/v1/group/name/<groupName>/project/name/<projectName>/scm/getState

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/scm/getState

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), and a description of the endpoint.

Method Body (JSON) Description
GET N/A Provides a list of commits on the local repository.


Get Differences

To get differences between the current Matillion version and the branch head, make the following GET call:

<instanceaddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>scm/differences

Here is an example of this request with variables added (the instance address has not been added):

{host:port}/rest/v1/group/name/Docs_Proj/project/name/Docs/version/name/default/scm/differences

Here is the endpoint as seen when running the {host:port}/rest/v1 in a browser:

The table below documents the type of API endpoint, the body (in a JSON format), and a description of the endpoint.

Method Body (JSON) Description
GET N/A Returns the difference between the current version (including uncommitted changes) and the head.


Shared Jobs

Many of the functions listed above can also be used with shared jobs, by making a POST call with the form:

<instanceAddress>/rest/v1/git/sharedJobs/<endpoint>

The endpoints which can be used for shared jobs are listed in the following tables.

Function Method Endpoint POST Body
Initialize shared jobs repository POST 
<instanceAddress>/rest/v1/git/sharedJobs/init
 See Initialize A Local Repository
Clone remote repository for shared jobs POST 
<instanceAddress>/rest/v1/git/sharedJobs/clone
 See Clone Remote Repository
Commit shared job changes to local repository POST 
<instanceAddress>/rest/v1/git/sharedJobs/commit
 See Commit to Local Repository
Switch to another shared job commit POST 
<instanceAddress>/rest/v1/git/sharedJobs/switchCommit
 See Switch Commit
Fetch shared job from remote repository POST 
<instanceAddress>/rest/v1/git/sharedJobs/fetch
 See Fetch from Local Repository
Push shared job to remote repository POST 
<instanceAddress>/rest/v1/git/sharedJobs/push
 See Push to a Remote Repository
Get differences between the current Shared Job version and the branch head GET 
<instanceAddress>/rest/v1/git/sharedJobs/differences
 N/A
Set remote url for git shared jobs repository POST 
<instanceAddress>/rest/v1/git/sharedJobs/setRemote
 
{
"url": "<URL>"
}
Create a new branch on current commit POST 
<instanceAddress>/rest/v1/git/sharedJobs/createBranch
 
{
    "branchName": "<name of new branch>"
}
Return the history of commits in all branches of the repository GET 
<instanceaddress>/rest/v1/git/sharedJobs/state/<max_count>/<skip>

  • max_count: the number of commits to be returned.
  • skip: the number of commits to skip before returning the <max_count> commits.
 N/A
Delete a branch DELETE 
<instanceaddress>/rest/v1/git/sharedJobs/branch/<branchName>
  • branchName: the name of the repository branch to be deleted.
 N/A
Delete the repository DELETE 
<instanceaddress>/rest/v1/git/sharedJobs/delete
 N/A


Contact Us

If any help is needed navigating the Git API in Matillion ETL, please read Getting Support.

What's Next

Cookie consent

We use our own and third-party cookies to understand how you interact with our knowledge base. This helps us show you more relevant content based on your browsing and navigation history.