Roles & Permissions (Azure)
  • Dark
    Light

Roles & Permissions (Azure)

  • Dark
    Light

Overview

For Matillion ETL to detect Azure Blob Storage containers, additional credentials may be required. Matillion ETL can either use Instance Credentials or User Defined Credentials, the latter of which will require you to gather credentials from your Azure account and enter them into Matillion ETL.


Using Identities (Instance Credentials)

To use Instance Credentials, your Matillion ETL Virtual Machine (VM) must already be set up. If you wish to use a User Identity (as opposed to a System Assigned Identity, which is unique to the VM) then you will need to search for the Managed Identities blade on the Azure Portal and set one up.

If you have not already done so, please follow the steps below:

1. From the Azure Portal, click Virtual Machines and select the virtual machine containing your instance.

Azure Portal-Virtual Machines

Azure Portal Virtual Machines

2. On the Virtual machines page, click Identity from the left-hand vertical menu, and then click User assigned and then click Add.

Identity

Identity

Please Note

  • If you wish to use a System Assigned Identity, click the System Assigned tab and set the Status to On. Make note of the Object ID.
  • If you wish to use a User Assigned Identity, click User assigned and then Add a User Identity of your choice.

3. Next, on the Add user assigned managed identity dialog, select one or more user assigned managed identities you want to assign to the resource. Then, click Add.

Add User Assigned managed identity

Add User Assigned managed identity

4. Next, browse to Storage accounts on the Azure Portal and select the account(s) that contains Blob Storage that you wish for Matillion ETL to have access to.

Storage Accounts

Storage Accounts

5. Click Access control (IAM) from the left-hand vertical menu and then click Add, followed by Add role assignment.

add role assignment

Add role assignment

6. In the Add role assignment dialog, set the Role as Storage Account contributor, and then select the desired user. Finally, click Save.

storage account contributor

Storage account contributor

Please Note

  • For System Assigned Identities, set the Assign access to dropdown to Virtual Machine and select/search for the VM that you turned on System Assigned Identity..
  • For User Assigned Identities, set the Assign access to dropdown to Azure AD user, group, or application and select/search for the User Identity you assigned to your VM.

The image below is the example while creating a new environment in the Matillion ETL instance.

Create Environment

Create Environment


Using Apps (User Defined Credentials)

Creating an App and Owning Storage Accounts To add Storage Accounts to Matillion ETL,we must first create an App. This requires a user with the 'Application administrator' directory role.

  1. Navigate to the Azure Portal. The Microsoft Azure login screen will appear immediately. Enter valid login credentials to continue. The browser will then redirect to the Microsoft Azure dashboard. Click App registrations on the Azure services menu at the top of the screen.

    Please Note

    If App registrations is not available on the Azure services menu, simply click More services, on the right of the menu, for a longer list of options.

    Microsoft Azure dashboard

    Microsoft Azure dashboard

  2. On the App registrations page, click + New registrations on the menu at the top of the screen.

    App registrations

    App registrations

  3. Now, in the Register an application window, provide details for the following fields:

    • Name – provide a name for the app
    • Supported account types – tick the checkbox next to Accounts in any organizational directory (Any Azure AD directory – Multi-tenant) and personal Microsoft accounts
    • Redirect URI (optional) – paste the Callback URL (copied from the Manage OAuth window in Matillion ETL), then click Register
    Register App

    Register App

  4. The browser will then redirect to the Overview window on the app's newly created dashboard. From here, copy the codes to the right of Application (client) ID and Directory (tenant) ID as they will be required later.

    App Overview

    App Overview

  5. Important Information

    If you haven't already, add a Blob Storage resource to this storage account.

  6. Return to the storage account and select the blob account from the list of storage accounts and click Overview and then, click Containers.
  7. Blob Account

    Blob Account

  8. To add a new container, click the + Container button, give it a name and click OK.
  9. Add Container

    Add Container

Now, when you import details from your App into Matillion ETL, your client will be able to discover those buckets the App has ownership of. To use this App in your Matillion ETL client, see the next section.


Gathering Azure credentials 

For a Matillion ETL instance to take advantage of Azure resources, you are required to provide credentials in the form of a Tenant ID, which is unique to your Azure account, then a Client ID and Secret Key which are taken from a Registered App.
 

Tenant ID

From the Azure Portal, browse to Azure Active DirectoryProperties and take the Tenant ID . Tenant ID

Tenant ID

Client ID

Browse from the Azure Portal to Azure Active DirectoryApp Registrations, and select an App that is associated with your desired Storage Accounts. Take the Application ID as your Client ID. Client ID

Client ID

Secret Key

  1. Browse from the Azure Portal to Azure Active DirectoryApp Registrations. Select the App associated with your desired Storage Accounts
  2. App Registration

    App Registration

  3. Next, click Certificates & secrets on the sidebar on the left. Then, in the Certificates & secrets window, click + New client secret.
  4. New client secret

    New client secret

  5. The Add a client secret pop-up window will then appear. Provide details for the following fields:

    • Description – provide a description of the client secret.
    • Expires – tick the checkbox next to when the client secret should expire, then click Add.
    Add client secret

    Add client secret

  6. Returning to the Certificates & secrets window, the new client secret will appear on the list in the Client secrets section. Copy the Value of the relevant client secret, as it will be required in Authorising for use in Matillion ETL.

    Please Note

    • Make sure to copy the client secret value right away as it may appear only once.
    • Additionally, when copying the client secret value, some browsers may add a space to the end of the string. Watch out for this as it will cause the credentials to fail.
    Copy client secret

    Copy client secret


Test Credentials

You can test credentials using ProjectManage Credentials.

If instance credentials are available, you can Test them by clicking the Test button at the top of the dialog. This will check access to any services that Matillion ETL uses. You may continue even if the tests fail, however some parts of the product may be impaired or non-functional without appropriate credentials.

User Defined CredentialsAzure are listed, by name, under their respective tabs. New User Defined Credentials can be added by using the button, edited using the Icon or Deleted using the Icon by each entry. When Creating or Editing credentials, a Test button is made available in the new dialog to check the details before finalising your credentials.

For detailed information on Manage Credentials, please visit here.

Manage Credentials

Manage Credentials