API v1 - Shared Jobs

API v1 - Shared Jobs


Overview

This is a guide to providing details on the Shared Job endpoints. The Shared Job endpoint allows users to explore what Shared Jobs are available on the instance as well as importing, exporting and deleting them. Users wanting to make use of this endpoint will require an account with the API Role and Shared Jobs permissions.

You can manage your Shared Jobs, via Project Menu Manage Shared Jobs within the Matillion ETL instance.

Important Information

  • This document is part of a series on Shared Jobs and the Matillion ETL API - v1.
  • This process requires the Matillion ETL instance URL, the username and password of a user with appropriate permissions, and an account with the API Role and Shared Jobs permissions.
  • Users responsible for experimenting with Matillion ETL API services require access to the Matillion ETL instance and ought to know how to make REST API calls either employing a REST API GUI client such as Postman or employing a command-line interface like cURL.



Shared Job API Endpoints

API Base URL

http(s)://<InstanceAddress>/rest/v1/<sharedjob>

API Endpoints and Function

Shared Job endpoint API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET, POST, and DELETE data. The Shared Job endpoint API service is accessed through the Uniform Resource Identifier (URI). The available API endpoints are listed below:

Method Path URI Function
GET /export http://<instance address>/rest/v1/sharedjob/export To export the metadata of the Shared Jobs within the instance.
POST /import http://<instance address>/rest/v1/sharedjob/import To import the data of the Shared Jobs within the instance.
GET /package http://<instance address>/rest/v1/sharedjob/package To get the package available within the Shared Job of the instance.
PATH/package/name/>packageName>
GET /packageName/export http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/export To export the specific package from the Shared Job within the instance.
GET /packageName/job http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job To get the list of Jobs available within the current package of the instance.
POST /packageName/delete http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/delete To delete the selected package from the list using HTTP POST request.
DELETE /packageName http://<instance address>/rest/v1/sharedjob/package/name/<packageName> Delete all jobs within a package.
PATH/job/name/<jobName>
GET /jobName/export http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobName>/export To export the specific job available within the package.
GET /jobName/revision http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobName>/revision To get the revisions available within the selected job.
POST /jobName/delete http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobname>/delete Delete all revisions of a single job.
DELETE /jobName http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobName> Delete all revisions of a single job.
PATH/revision/id/<revisionId>
GET /revisionId/export http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobName>/revision/id/<revisionId>/export To export the specific job available within the package.
POST /revisionId/delete http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobname>/revision/id/<revisionId>/delete To delete the selected job from the list using HTTP POST request.
DELETE /revisionId http://<instance address>/rest/v1/sharedjob/package/name/<packageName>/job/name/<jobName>revision/id/<revisionId>/ Delete a single revision.

Graphical Represenation

To illustrate the Shared Job API v1, endpoints and methods to the further, below is the graphical flow of the /sharedJob endpoint showing possible PATH, GET , POST, and DELETE options.

Shared Job endpoint Flow

Shared Job endpoint Flow



URL Parameters and Description

Below is the list of endpoint parameters and their brief description:

Parameters Name Description
<instance address> This is the server IP address or domain name.
<SharedJobs> The Sharedjobs in the instance.
<packageName> The name of the package within the Sharedjobs.
<jobName> The name of the job within the package.
<revision> The revision listed within the job package of Shared Jobs.
<export> This allows to export the metadata of the resource.
<import> This allows to import the metadata to the resource.
<delete> Delete the selected resource.



Endpoints and Server Response

This chapter describes the Shared Job API endpoints with examples. These APIs offers REST-based web service, offering ease of use and a flexible choice of programming language. These APIs can be used to access and analyse the packages, jobs and revisions within the Matillion ETL instance.

All the APIs listed in this chapter are available to use with GET/POST/DELETE methods to retrieve the data used to get, update, import/export, delete the resource within the Shared Jobs.

List of endpoints for the /sharedJob:

Below is the detailed description of these endpoints with example response from the server.

GET/export

To export the Shared Jobs metadata from within the Matillion instance, use the /export endpoint after /sharedjob. This example using GET method REST API call to export the Shared Jobs metadata available within the instance.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/export
  • Server Response
    {
        "objects": [
            {
                "metadata": {
                    "identifier": {... }
                        ]
                    },...
                    "orchestrationJobs": [
                        "Shared Jobs - Unicode"
                    ],...
                    "transformationJobs": [],       
                        },
                        "variables": {
                            "oauth": {....}
                        },  ....      
        ],
        "version": "master",
        "environment": "redshift"
    }

POST/import

Now you have already export the Shared Job metadata in the example before. This time is to import the data via ExportContainer. Supply the data as already encoded since Shared Job will be created as is. This will be a POST method API call as we will have to attach the Imported data, in the body as a JSON file to run into the Matillion ETL instance.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/export
  • Server Response
    {
      "name": "sharedjobs",
      "statusList": [
        {
          "success": true,
          "name": "matillion.regression.Unicode_1.1"
        },
    	 {
          "success": true,
          "name": "Matillion.Incremental.Gmail Incremental Load.1",
        }
      ],....
      "success": true
    }

GET/package

To get the packges available within the shared job , use the /package endpoint after /sharedjob. This example using GET method REST API call to fetch the packages detail available within the Shared Jobs.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/export
  • Server Response
    [
        "TestPackage-1",
        "TestPackage-2",
        "TestPackage-3"
    ]

PATH/package/name/<packageName>


List of endpoints for the /package/name/<packageName>:

Below is the detailed description of these endpoints with example response from the server.

GET/export

To export a specific package data from within the Shared Jobs, provide /packageName and just use the /export endpoint after that . This example using GET method REST API call to export a package available within the instance.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/export
  • Server Response
    {
        "objects": [
            {
                "metadata": {...
    			},
                    "orchestrationJobs": {...}
                       },
                    "transformationJobs": {...},       
                        },
        ],
        "version": "master",
        "environment": "redshift"
    }

GET/job

This is an example of GET method REST API call, to get the jobs available within the selected package .

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job
  • Server Response
    [
        "SharedJob"
    ]

POST/delete

This will be a POST method API call. The /delete endpoint will allow to delete the selected package from the Shared Job indicating revision id.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/delete
  • Server Response
    {
      "success": true,
      "msg": "Successfully deleted [3] Shared Job(s).",
      "id": -1
    }

DELETE/packageName

To remove any resource from the list, we will use DELETE API request. In this example, we will delete a Package from the Shared Job.

Please Note

There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>
  • Server Response
    {
      "success": true,
      "msg": "Successfully deleted [4] Shared Job(s).",
      "id": -1
    }

PATH/job/name/<jobName>

List of endpoints for the /job/name/<jobName>:

Below is the detailed description of these endpoints with example response from the server.

GET/export

To export a specific job metadata from within the Shared Job Package, provide /jobName and just use the /export endpoint after that . This example using GET method REST API call to export a job available within the instance.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/export
  • Server Response
    {
        "objects": [
            {
                "metadata": {...
    			},
                    "orchestrationJobs": {...}
                       },
                    "transformationJobs": {...},       
                        },
        ],
        "version": "master",
        "environment": "redshift"
    }

GET/revision

This is an example of GET method REST API call, to get the revision id associated with the current job.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/revision
  • Server Response
    [
        1
    ]

POST/delete

This will be a POST method API call. The /delete endpoint will allow to delete the selected package from the Shared Job .

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/delete
  • Server Response
    {
      "success": true,
      "msg": "Successfully deleted [1] Shared Job(s).",
      "id": -1
    }

DELETE/packageName

To remove any resource from the list, we will use DELETE API request. In this example, we will delete a job .

Please Note

There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/delete
  • Server Response
    {
      "success": true,
      "msg": "Successfully deleted [1] Shared Job(s).",
      "id": -1
    }

PATH/revision/id/<revision>

List of endpoints for the /revision/id/<revisionId>:

Below is the detailed description of these endpoints with example response from the server.

GET/export

To export the job metadata using revision id from within the Shared Job Package, provide /revisionId and just use the /export endpoint after that . This example using GET method REST API call to export a job metada including all its PATHS and datatypes within the instance.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/revision/id/<revisionId>/export
  • Server Response
    {
        "endpoints": [
            {...
    		}
    		]'
          "dataTypes": [...
    	     ]
    }

POST/delete

This will be a POST method API call. The /delete endpoint will allow to delete the selected job using its revision id from the Shared Job.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/revision/id/<revisionId>/delete
  • Server Response
    {
      "success": true,
      "msg": "Successfully deleted [1] Shared Job(s).",
      "id": -1
    }

DELETE/revisionId

To remove any resource from the list, we will use DELETE API request. In this example, we will delete a job .

Please Note

There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap.

  • Base URL
    http://<instance address>/rest/v1/sharedjob/package/name/<packagename>/job/name/<jobName>/revision/id/<revisionId>
  • Server Response
    {
      "success": true,
      "msg": "Successfully deleted [1] Shared Job(s).",
      "id": -1
    }