Secrets Manager

Overview

CDC in Matillion Data Loader requires access to secrets stored on your AWS Secrets Manager Service for:

Identification between installed agents and your Hub account (see Adding Platform Keys for information).
Storing database credentials to be used when configuring a pipeline's source.

Log into your AWS account. This should be the same account that you will be using the Matillion CDC agent in.
Browse to the AWS Secrets Manager service.
Click Store a new secret.
Select Other type of secret.
In the plaintext field, remove any formatting and enter your secret.

Note

A secret can be used for either hosting a password for the connection to a source database and/or hosting the Private Key.
Although you may need to store multiple passwords and keys, these should each be in a separate secret.
Password/Secret needs to be in the plaintext value. Make sure you clear the {"":""} json format first; the JSON parser returns a token error if the key is stored in json format and parsed out in the code.

For example, a database password appears as below:

A private key secret would appear as follows in plaintext value:

In the Encryption key field, we advise leaving the field blank so Secrets Manager automatically provisions the KMS key. If you opt to use a customer-managed KMS key, you will need to provide your agent access to a custom key if used.
Click Next
Give your secret a Secret name
- For database passwords, the secret name can be arbitrary and is referred to in Matillion Data Loader
It is not required to give individual Resource permissions if this key is being used by services within the same account so, within our best-practice guidelines, this can be ignored
- If you are expected to access this key from another AWS account, consult your administrator for access
- This is not the same as granting permission to other resources to access the key. See the Permissions section after creating your secret for more information
Click Next and then Next again on the Configure rotation page
Review your new secret and click Store when satisfied
Click back into your new Secret and note down the Secret ARN. You might need ARN to be provided when you create an agent.

Your CDC agent will require the following AWS Secrets Manager permissions:

For more information on permissions, see the article on IAM Roles.

Table of contents