This article is designed to help you troubleshoot some of the typical errors that you may encounter while configuring and working with CDC UI Agent installation and configuration process.
We ask that if you encounter problems with these tasks, you make a reasonable attempt to troubleshoot the issue before raising a ticket with Matillion Support.
This article has been categorized into following sections:
- Prerequisite for Customer Infrastructure setup.
- Prerequisite for AWS environment setup.
- Template Usage - Troubleshooting.
- Accessing Agent Logs.
Matillion will continue to add and update the information in this article, to continually help you investigate, diagnose and fix any issues. If you are not able to resolve your issue with this information, please contact Matillion Support.
Prerequisite for customer infrastructure setup
- Matillion recommends you to setup your database infrastructure for CDC before proceeding with Agent installation instructions.
- Please ensure you have required database authority and privileges to manage datastores.
- If you are not a database administrator, contact your system administrator to give access to your account to create a stack in your infrastructure environment.
Prerequisite for AWS environment setup
- An active AWS account is available.
- The Amazon instance is running.
- SELECT privileges are granted to the database user.
- Secret Key needs to be uploaded to the AWS Secret Manager.
- Subnets within a customer private cloud with outbound access to the relevant data sources and destinations, as well as to ECR and to the Matillion CDC websocket endpoint.
- Security group controlling ingress/egress within the subnet(s).
- CloudWatch log group for container log ingestion.
- S3 bucket for data ingestion, with permissions granted to the IAM role.
- IAM task execution role with permissions to pull from ECR Public repositories.
- A valid log group needs to be created, where Matillion pushes the logs. You should also have valid AWS credentials (access and secret keys) of an IAM user who has
PutLogEventspermissions on the log group created.
Template usage troubleshooting
The Matillion CDC Agent is a container image that's required to be deployed on any supporting containerised infrastructure and as part of this installation process, the agent requires some specific environment variables to be configured for platform authentication and functionality.
Some of these environment variables are automatically generated by the Matillion CDC platform and stated on the Installation Instructions page in CDC UI.
Please ensure you take note of the environment variables listed on the Installation Instructions page as they will be required when you install and configure agent in your database environment.
There are templates available for every database to install agent setup configurations. You can refer required templates in Deploying CDC Agent article available on Matillion CDC documentation site.
AWS Template - Basic/Advanced
Basic Template: Basic templates should be used by users with more stringent requirements around networking and permissions. In some cases such users may need to set up manually in a way that meets their company policy.
Advanced Templates: Advanced templates contains required resources for the users that will automatically update and fill required fields while configuring the stack in AWS. This option is suitable for a user looking to try out the software.
Accessing agent logs
Agent logs provide verbose descriptions of agent information, warnings, and errors to help you diagnose and fix any installation and platform communication issues. If you are not able to resolve your issue with this information, please contact Matillion Support.
Agents are automatically configured to write logs to the platform's native logging facility, and therefore the process for accessing the logs will depend on the platform used.
Agents deployed to AWS use the AWS CloudWatch service. To view the logs, log into your AWS account and browse to the CloudWatch service. The logs will be located under the Log group you created when you installed the agent.
Agents deployed to Azure use the Azure Log Analytics service. To view the logs, log into your Microsoft Azure portal, select the resource group that hosts your agent, and select the Log Analytics workspace you created when you installed the agent.
Google Cloud Platform
Agents deployed to GCP store logs on the virtual machine instance that hosts the agent. To view the logs, log into your Google Console, select the VM that the agent is deployed to, and browse to the Observability information.