Roles and Permissions (Azure)

Overview

For Matillion ETL to detect Azure Blob Storage containers, additional credentials may be required. Matillion ETL can either use Instance Credentials or User Defined Credentials, the latter of which requires you to gather credentials from your Azure account, and enter them into Matillion ETL.

Using identities (instance credentials)

To use Instance Credentials, your Matillion ETL Virtual Machine (VM) must already be set up. If you wish to use a User Identity (as opposed to a System Assigned Identity, which is unique to the VM) then you will need to search for the Managed Identities blade on the Azure Portal and set one up.

If you haven't already done so, please follow the steps below:

1. From the Azure Portal, click Virtual Machines and select the virtual machine containing your instance.
2. On the Virtual machines page, click Identity from the left-hand vertical menu, then click the User assigned tab at the top, and click + Add.

3. Next, on the Add user assigned managed identity dialog, you will see a list of user assigned managed identities. Select one or more identities that you want to assign to the resource, then click the Add button.

4. Next, in the Azure portal, browse the list of Storage accounts, and select the Blob Storage account that you want Matillion ETL to access.

5. Once you've selected your Blob Storage account, click Access control (IAM) from the left-hand vertical menu, and click + Add at the top of the page, followed by Add role assignment, situated underneath the aforementioned Add button.

6. The Add role assignment dialog will be displayed. Use the Role drop-down menu to select Storage Account contributor, and select the desired user. When you've made your selections, click Save.

The image below is the example while creating a new environment in the Matillion ETL instance.

Using apps (user defined credentials)

Creating an App and Owning Storage Accounts To add Storage Accounts to Matillion ETL,we must first create an App. This requires a user with the 'Application administrator' directory role.

1. Navigate to the Azure Portal. The Microsoft Azure login screen will appear immediately. Enter valid login credentials to continue. The browser will then redirect to the Microsoft Azure dashboard. Click App registrations on the Azure services menu at the top of the screen.

2. The App registrations page will be displayed. Click + New registration at the top.
3. Now, in the Register an application> window, provide details for the following fields:
   • Name – provide a name for the app.
   • Supported account types – tick the checkbox next to Accounts in any organizational directory (Any Azure AD directory – Multi-tenant) and personal Microsoft accounts.
   • Redirect URI (optional) – paste the Callback URL (copied from the Manage OAuth window in Matillion ETL), then click Register.
4. The browser will redirect to the Overview page on the app's newly created dashboard. From here, copy the codes to the right of Application (client) ID and Directory (tenant) ID as they will be required later.

5. Return to the storage account and select the blob account from the list of storage accounts, and click Overview, then click Containers.

6. To add a new container, click the + Container button at the top. Give it a name, and click Create.

Now, when you import details from your App into Matillion ETL, your client will be able to discover those buckets the App has ownership of. To use this App in your Matillion ETL client, see the next section.

Gathering Azure credentials

For a Matillion ETL instance to take advantage of Azure resources, you are required to provide credentials in the form of a Tenant ID, which is unique to your Azure account, then a Client ID and Secret Key which are taken from a Registered App.

Tenant ID

From the Azure Portal, browse to Azure Active Directory, then click Properties from the sidebar on the left, and copy the Tenant ID.

Client ID

Browse from the Azure Portal to Azure Active Directory, then click App Registrations, and select an App that's associated with your desired Storage Accounts. Copy the Application ID, for your Client ID.

Secret Key

1. Browse from the Azure Portal to Azure Active Directory, then click App Registrations. Select the App associated with your desired Storage Accounts

2. Next, click Certificates & secrets on the sidebar on the left. Then, in the Certificates & secrets window, click + New client secret, situated underneath the Client Secret section.

3. The Add a client secret pop-up window will appear. Provide details for the following fields:

   • Description – provide a description of the client secret.
   • Expires – tick the checkbox next to when the client secret should expire, then click Add.

4. Returning to the Certificates & secrets window, the new client secret will appear on the list in the Client secrets section. Copy the Value of the relevant client secret, as you will need to refer to this later in the process.

Test credentials

You can test credentials using Project, then Manage Credentials.

If instance credentials are available, you can test them by clicking the Test button at the top of the dialog. This will check access to any services that Matillion ETL uses. You may continue even if the tests fail, however some parts of the product may be impaired or non-functional without appropriate credentials.

Azure User Defined Credentials are listed by name under their respective tabs. New User Defined Credentials can be added by using the + button, edited using the pencil icon or deleted using the X icon by each entry. When creating or editing credentials, a Test button is made available in the new dialog to check the details before finalizing your credentials.

For detailed information on Manage Credentials, click here.

If you need help with any issues connecting to Azure Blob Storage from Matillion ETL, please refer to the Troubleshooting connection to Azure Blob Storage guide for more information.