API v1 - Tasks

API v1 - Tasks


Overview

This is a guide to providing details on the Task API from the familiy of the Group API . The Task API endpoint allows users to explore and manages the Tasks created whenever an Orchestration or Transformation operation is performed in the Matillion ETL instance.

This includes:

  • Tasks created by user operations - This includes almost all operations that the user performs that generate database queries such as running a job, retrieving a sample or a row count.
  • Tasks created by the Scheduler.
  • Tasks run via the Matillion ETL API (such tasks will display as "API" in the Tasks tab).

As an overall Orchestration or Transformation task is performed (such as executing an Orchestration job) the overall Task is broken down into more granular tasks as the execution continues. Task information is immediately available in the Task panel at the bottom-right of the client interface, giving concise information on recently-run tasks and how they have progressed. Whereas, Task History is available within Matillion through Project Menu Task History

Task

Task

The Task API provides the resource details to run the job within the project group in the Matillion ETL instance. By resources here, we refer to run jobs, list running tasks, task history and cancel the task. Task endpoint is a part of the PATH/project family with some endpoints that combined with HTTP methods (GET, POST, and DELETE) for unique combinations to get the "resource" information for the Matillion ETL instance.

Important Information

  • This document is part of a series on API v1 - Group and the Matillion ETL API - v1.
  • Users responsible for experimenting with Matillion ETL API services require access to the Matillion ETL instance and ought to know how to make REST API calls either employing a REST API GUI client such as Postman or employing a command-line interface like cURL.
  • Matillion ETL API endpoints require authorisation to make any REST API call, so ensure a username and password for the Matillion ETL instance is configured before making any API call.

Task API Endpoints

API Base URL

http(s)://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task

API Endpoints and Function

Task endpoint API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET, POST, and DELETE data. The Task endpoint API service is accessed through the Uniform Resource Identifier (URI). All following references in this document will assume the API Base URL has been specified. The available API endpoints are listed below:

Method Path URI Function
GET /running http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/running Get the detail of the task running for the selected project.
GET /history http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/history To export the metadata of the current version within the project.
PATH /id/{taskID:\\d+} http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/task/id/<taskID> To retrieve the data of the task using taskID.
PATH /instance http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/task/instance?taskID=<taskID> To get the instance detail via taskID.



Graphical Represenation

To illustrate the Task API v1, endpoints and methods to the further, below is the graphical flow of the /task endpoint showing possible PATH, GET , POST, and DELETE options.

Task endpoint Flow

Task endpoint Flow


URL Parameters and Description

Below is the list of endpoint parameters and their brief description:

Parameters Name Description
<InstanceAddress> This is the server IP address or domain name.
<groupName> the name of the group.
<projectName> The name of the project within the group.
<task> To get the task details, created whenever an Orchestration or Transformation operation performed within the Matillion ETL instance.
<running> To get the detail of tasks running, when an Orchestration or Transformation operation performed within the project in the Matillion ETL instance.
<history> Retrieve the history of tasks within selected project of the Matillion ETL instance.
<instance> To get the instance detail where an Orchestration or Transformation operation performed.



Endpoints and Server Response

This chapter describes the Task API endpoints along with some server response examples for reference. These APIs offers REST-based web service, offering ease of use and a flexible choice of programming language. Task API endpoints can be used to access and analyse the tasks, whenever an Orchestration or Transformation operation performed within the Matillion ETL instance.

List of endpoints for the /task:

Please Note

For the illustrative view of the flow of PATH/task endpoints, you can refer the section Graphical Representation of this guide.

Below is the detailed description of these endpoints with some server response examples for reference.

GET/running

This is a GET method REST API request that will provide the list of task running in the project when Orchestration or Transformation operation preformed within the instance.

  • Base URL
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/running
  • Server Response
    [
      {
        "id": 63059,
        "type": "RUN_ORCHESTRATION",
        "customerID": 2052,
        "groupName": "API GROUP",
        "projectID": 4394,
        "projectName": "API PROJECT",
        "versionID": 4395,
        "versionName": "default",
        "jobID": 4461,
        "jobName": "API JOB TEST",
        "environmentID": 4398,
        "environmentName": "API PROJECT_full",
        "state": "RUNNING",
        "enqueuedTime": 1594729286893,
        "startTime": 1594729286894,
        "endTime": 0,
        "message": null,
        "originatorID": "ws_61498_7",
        "rowCount": 0,
        "tasks": [
          {
            "taskID": 1,
            "parentID": -1,
            "type": "VALIDATE_ORCHESTRATION",
            "jobID": 4461,
            "jobName": "API JOB TEST",
            "jobRevision": 2,
            "jobTimestamp": 1594714513373,
            "componentID": 4462,
            "componentName": "Start 0",
            "state": "SUCCESS",
            "rowCount": -1,
            "startTime": 1594729286896,
            "endTime": 1594729286900,
            "message": ""
          },
    ]

GET/history

This endpoint is to fetch the history of the task performed within the instance for a specific period of time.

  • Base URL
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/history?since=<since>
  • Where <since> takes the form yyyy-MM-dd

  • Server Response
    [
      {
        "id": 63018,
        "type": "RUN_ORCHESTRATION",
        "customerID": 793,
        "groupName": "API GROUP",
        "projectID": 795,
        "projectName": "API PROJECT",
        "versionID": 796,
        "versionName": "default",
        "jobID": 827,
        "jobName": "Sleep",
        "environmentID": 799,
        "environmentName": "test",
        "state": "SUCCESS",
        "enqueuedTime": 1594729190954,
        "startTime": 1594729190955,
        "endTime": 1594729201024,
        "message": null,
        "originatorID": "ws_61498_7",
        "rowCount": 0,
        "tasks": [
          {
            "taskID": 1,
            "parentID": -1,
            "type": "VALIDATE_ORCHESTRATION",
            "jobID": 827,
            "jobName": "Sleep",
            "jobRevision": 2,
            "jobTimestamp": 1594714500345,
            "componentID": 828,
            "componentName": "Start 0",
            "state": "SUCCESS",
            "rowCount": -1,
            "startTime": 1594729190956,
            "endTime": 1594729190958,
            "message": ""
          },
          {
            "taskID": 2,...
          }
        ],
        "hasHistoricJobs": true,
        "jobNames": [
          "API JOB TEST"
        ]

PATH/id/{taskID}

To get the task detail of the operation performed by the specific Orchestration or Transfrmation job via Id number of the job. This is a GET API call request to retrieve the task details.

  • Base URL
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/id/<taskId>
  • Server Response
    {
      "id": 12345,
      "type": "RUN_ORCHESTRATION",
      "customerID": 793,
      "groupName": "API GROUP",
      "projectID": 795,
      "projectName": "API PROJECT",
      "versionID": 796,
      "versionName": "default",
      "jobID": 827,
      "jobName": "API JOB TEST",
      "environmentID": 799,
      "environmentName": "test",
      "state": "SUCCESS",
      "enqueuedTime": 1594729190954,
      "startTime": 1594729190955,
      "endTime": 1594729201024,
      "message": null,
      "originatorID": "ws_61498_7",
      "rowCount": 0,
      "tasks": [
        {
          "taskID": 1,
          "parentID": -1,
          "type": "VALIDATE_ORCHESTRATION",
          "jobID": 827,
          "jobName": "API JOB TEST",
          "jobRevision": 2,
          "jobTimestamp": 1594714500345,
          "componentID": 828,
          "componentName": "Start 0",
          "state": "SUCCESS",
          "rowCount": -1,
          "startTime": 1594729190956,
          "endTime": 1594729190958,
          "message": ""
        },
      ],
      "hasHistoricJobs": true,
      "jobNames": [
        "API JOB TEST"
      ]
    }

PATH/instance?taskID=<taskID>

To get the task detail of the operation performed by the specific Orchestration or Transfrmation job via Id number of the job. This is a GET API call request to retrieve the task details.

  • Base URL
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/instance?taskID=<taskID>
  • Server Response
    {
      "id": 12345,
      "type": "RUN_ORCHESTRATION",
      "customerID": 793,
      "groupName": "API GROUP",
      "projectID": 795,
      "projectName": "API PROJECT",
      "versionID": 796,
      "versionName": "default",
      "jobID": 827,
      "jobName": "API JOB TEST",
      "environmentID": 799,
      "environmentName": "test",
      "state": "SUCCESS",
      "enqueuedTime": 1594729190954,
      "startTime": 1594729190955,
      "endTime": 1594729201024,
      "message": null,
      "originatorID": "ws_61498_7",
      "rowCount": 0,
      "tasks": [
        {
          "taskID": 1,
          "parentID": -1,
          "type": "VALIDATE_ORCHESTRATION",
          "jobID": 827,
          "jobName": "API JOB TEST",
          "jobRevision": 2,
          "jobTimestamp": 1594714500345,
          "componentID": 828,
          "componentName": "Start 0",
          "state": "SUCCESS",
          "rowCount": -1,
          "startTime": 1594729190956,
          "endTime": 1594729190958,
          "message": ""
        },
      ],
      "hasHistoricJobs": true,
      "jobNames": [
        "API JOB TEST"
      ]
    }