Matillion ETL for Redshift Release Notes

Matillion ETL for Redshift Release Notes


WARNING: We highly recommend updating your Matillion ETL instance following the recent Redshift update. Please refer to this technote for more information.

From v1.46 onward, users can request only a maximum of 500,000 rows of Task History. This is due to a change in how task history is recorded will not be a limitation in future versions. If you wish to gather more than 500,000 records, this guide makes retrieving older task history quick and simple.


Attention: If you started your Matillion ETL instance at 1.42.x or earlier and have manually upgraded the Tomcat package to a newer version than 8.5.42 please be aware that upgrading may cause problems.If your Tomcat version has not been changed this should not affect you, although we would still recommend starting a new instance from the Marketplace to ensure no future upgrade issues.

We highly recommend reading the Updating Matillion ETL article for the safest way to update your instance. Do not hesitate to contact Matillion support for assistance in this matter.


Matillion ETL for Redshift v1.47

New Features

  • New ‘Create your own connector’ feature. The new API Extract wizard and API Extract component allows you to connect you to any service that offers a REST API, allowing you to create your own connectors to load data.
  • New Intercom Extract component lets users connect to their Intercom account and load data into external tables.

Improvements

  • Assert components now auto-populate their properties when connected to another component. For example: Connecting Assert Table to an S3 Load component will automatically configure it to use the same table as the S3 Load component


Matillion ETL for Redshift v1.46.10

  • Fixed an issue where External Table Output could not populate the list of existing tables.




Matillion ETL for Redshift v1.46.8

  • Fixed an issue in the Matillion ETL v1 API where the flag "onConflict=OVERWRITE" during job import was not working.
  • Fixed an issue where running a Transformation job more than once concurrently can cause it to interfere with itself.
  • Multiple bug fixes for "Assert" Components
    • Multi-line data types now supported
    • Fixed an issue where mismatched value types pass validation
    • No longer validates on size and precision fields where not relevant
    • Multiple fixes to error messages


Matillion ETL for Amazon Redshift v1.46.7, v1.44.17 and v1.42.14

  • Fixed an issue where a recent Redshift update changing the Redshift cluster schema prevented Matillion ETL from using External Tables via Amazon Spectrum.
For 1.46.7 please follow the recommended update route. For earlier versions, please see Updating to a Specific Release.

 

Matillion ETL for Amazon Redshift v1.46

New Components

  • New Zendesk Chat Extract component allows users to bring their Zendesk Chat data into an external table.
  • New Zendesk Talk Query component allows users to bring their Zendesk Talk data into a table.
  • New Salesforce Output component allows users to push data from their data warehouse to a Salesforce Object of their choice.
  • New enterprise-only “Assert” components can be used to test an existing object (Table, External Table, Scalar Variable, or View) against user-defined criteria. This allows users to prevent jobs continuing if resources are not in their expected state.

New Features

  • New Recycle Bin feature gives users the opportunity to recover any deleted jobs before committing to delete them completely.
  • New Amazon Redshift Management Shared Jobs allow users to perform common Redshift management tasks from within Matillion ETL jobs. This includes creating, pausing, and resizing clusters, creating and deleting snapshots, and modifying usage limits.

Improvements

  • Reworked Component panel organization gives an alphabetized list of all connectors with Load and Unload components getting their own folders.
  • Connectors that only import data into external tables have been renamed as “Extract” components.
  • Updated Salesforce Query driver

Matillion ETL for Redshift v1.44.18

  • Fixed an issue where External Table Output could not populate the list of existing tables.



Matillion ETL for Redshift 1.44.16

  • Fixed an issue where a certain Matillion ETL functions such as Python Script or failing a Migrate may cause instance credentials to be stored in log files.

Matillion ETL for Redshift 1.44.14

  • Updated Google Ad Manager Query component from API v201905 to API v201908, anticipating the deprecation of v201905 in June 2020. Users can override this version change using the "Version" connection option with a value of "v201905".

Matillion ETL for Redshift 1.44.13

  • Fixed an issue where Shared Jobs are not correctly running concurrent connections and transformation jobs.


Matillion ETL for Redshift 1.44.11


Notes

  • Users experiencing issues pulling Ticket_Metrics columns in the Zendesk Query component while using a custom RSD config can see this article for potential solutions.


New Features

  • Project Admin assignment option added to External Security users.
  • Added new data sources for Zendesk Query component: group_memberships, groups, satisfaction_ratings, sla_policies, tags, ticket_audits, ticket_comments, ticket_fields, ticket_forms, ticket_metrics_events_incremental.


Fixes

  • Fixed an issue where Data Transfer component would fail to maintain a connection when loading from Sharepoint.
  • Fixed an issue where Zendesk Query component was bringing in erroneously-empty date fields.
  • Fixed an issue where Twitter Query component was attempting to use an outdated version of the AdStats data source.
  • Fixed "Error parsing int value [null]" error from Google Ad Manager Query component.
  • Non-functional 'FromFriendlyName' columns have been removed from Gmail Query component data sources.
 



Matillion ETL for Redshift 1.44.9


Fixes

  • Fixed 'black screen' error caused by XSS filter being enabled by default.







Matillion ETL for Redshift 1.44.8


New Features

  • Job Start Control allows you to ‘prevent duplicate job’ runs by selecting the option in the scheduler. When selected this allows users to specify that a scheduled job run should not be queued if that same job from a previous schedule is still running, preventing duplicates from stacking up in the queue.
  • Box and Dropbox source options added to Data Transfer Component, allowing users to move their stored files between cloud services.


New Components

  • Amplitude connector allows users to connect to their Amplitude account and load their data into external tables.
  • Box connector allows users to collect data regarding their stored files and load it into external tables.
  • Dropbox connector allows users to collect data regarding their stored files and load it into external tables.
  • MindSphere connector (Amazon Redshift only) allows users to connect to their MindSphere account and load data into external tables.
 

Improvements

  • Add New Data Sources added to the Google Analytics Query component allows users a way of adding new tables to the "Data Sources" property without the need to deal with custom schemas.
  • Git Credentials can now be stored so users don’t need to re-enter credentials every time they want to fetch or push a branch.
  • Project Admin option now added to users defined in Internal Security that allows these users to see and access all projects regardless of the project privacy settings.
 

Matillion ETL for Redshift 1.42.11

  • Upgrade: Added OAuth 2.0 support for Xero component (OAuth 1.0 now deprecated)
  • Upgrade: Updated Xero Driver
  • Fix: Customer seeing Database Query validation errors after updating from 1.39.8 to 1.42.x,
  • Fix: Create Table displaying errors when given a Snowflake Database Name containing a hyphen.







Matillion ETL for Redshift 1.42.10

  • Upgraded BigQuery driver to v19
  • Fix: SQS Queue erroneously disconnecting
  • Fix: postgreS error: "could not open relation with OID"





Matillion ETL for Redshift 1.42.8

  • Fixed various issues and bugs including:
    • Google OAuth creation to failing.
    • Query To Grid component would sometimes use the wrong database name in advance mode.
    • Taking metadata from new tables with no encoding set could cause the Metadata To Grid component to fail validation.


Matillion ETL for Redshift 1.42



New Features

 New Components   Improvements  

Matillion ETL for Redshift 1.41.8

  • Fixed an issue where Git Push to remote repository fails citing an 'Invalid Private Key'.
 

Matillion ETL for Redshift 1.41.5

  • Fixed error when upgrading to a t3.xlarge instance that would cause schedules to run and unable to be stopped.
  • Increased possible Column Name lengths for Grid Variable mappings.




Matillion ETL for Redshift 1.41



New Components

Improvements  
  • Components
  • Additional Improvements
    • The Shared Jobs now allows creators to mark parameters as ‘required’ so users of the job must provide input for these parameters.
    • LDAP Configuration (External Security) now can be tested before confirming config. 
    • Recursive Validation means that jobs linked to others via Run Orchestration and Run Transformation will be validated alongside their parent job.



Matillion ETL for Redshift 1.40.11

  • Twitter driver upgraded to support Ads API v5
  • DoubleClick For Publishers (Google Ads Manager) driver upgraded to support API v201811 and later.
  • Windows Fileshare SMB2/3 Support for Data Transfer component and File Iterator component.
  • SAP Netweaver Query - update to the location of additional files including a "JCo" connection library and the Java wrapper "sapjco3.jar".
  • Fixed an issue where SAP Netweaver Query component was not correctly finding associated libraries.
  • Fixed an issue where Hubspot OAuth registration link would fail.




Matillion ETL for Redshift 1.40 (Features Video)

New Features

  • Git functionality allows users to work with local and remote repositories, managing Matillion Projects and Versions, with commits and branches seamlessly inside the Matillion ETL client. (Enterprise Only)
  • Go beyond a single connection to a target database with the new Multiple Connections option for faster concurrent loading. (Enterprise Only)

Improvements

  • User Interface
    • Changes to error and validation messaging in product.
    • Improved plus(+) sign and grid visibility.
    • UI Refresh - Tabs with no component selected.
    • Keyboard Shortcut Ctrl+A for single-line text input fields.
    • Order the Component Tree.
    • Add Manage Variables button in Code Editors.
    • Filter added to Sample Tab.
    • Simple JSON or XML parser for Configure API Profile Dialog.
    • All sorting of Project versions in Manage Versions.
  • Components
    • Dynamics CRM Query component now supports CRM2011accesible from CRM Version dropdown.
    • JDBC Incremental Load - Prepopulate URL Field with Connection Prefix.
    • Additional JDBC Table Metadata To Grid fields give allows users to specify more information about their data source.
  • Additional Improvements
    • Remove row count from Result.java.
    • AMIs upgraded to Python 3.6.
    • The API Profile Generator component is now able to find files in the current directory the user is in, rather than specifying the whole path of the file.