Matillion ETL for Snowflake Release Notes

Matillion ETL for Snowflake Release Notes


Important: Users with instances hosted on Microsoft Azure or Amazon Web Services should take notice of a security advisory relating to Matillion ETL v1.44 instances.

Attention: If you started your Matillion ETL instance at 1.42.x or earlier and have manually upgraded the Tomcat package to a newer version than 8.5.42 please be aware that upgrading may cause problems.If your Tomcat version has not been changed this should not affect you, although we would still recommend starting a new instance from the Marketplace to ensure no future upgrade issues.

We highly recommend reading the Updating Matillion ETL article for the safest way to update your instance. Do not hesitate to contact Matillion support for assistance in this matter.



Matillion ETL for Snowflake v1.47.6

  • Fixed an issue where the Matillion ETL client would get stuck loading projects.
  • Fixed an issue with the Postgres driver: "The SSLSocketFactory class provided org.postgresql.ssl.DefaultJavaFactory could not be instantiated".
  • Fixed an issue where the JDBC properties file could not longer be edited in 1.47.



Matillion ETL for Snowflake v1.47

Note: Due to some technical issues with this release, we do not recommend customers manually update to this version. We highly recommend users on 1.47 to update to 1.47.6 immediately.

New Features


Improvements

  • New ‘Create your own connector’ feature. The new API Extract wizard and API Extract component allows you to connect you to any service that offers a REST API, allowing you to create your own connectors to load data.
  • Create External Table can now load files based on user-defined regular expressions.
  • Added option to set DDL Collation in the Create Table component.
  • Load Components now have a “Match by Column Name” property that allows loading of semi-structured data directly from a file to a table.
  • Individual Jobs are now selectable in the Migration wizard, allowing greater control over what is migrated.
  • Table data can now be copied to external Parquet files using LZO compression
  • Users can now specify a fully qualified name for a stage and format in Snowflake from a different database than that which the user has currently selected stage name in the format of database.schema.stage and the format name in the format of database.schema.format. 




Matillion ETL for Snowflake v1.46.10

  • Fixed an issue that caused intermittent hanging of transfgormation jobs following the 1.46.8 update.




Matillion ETL for Snowflake v1.46.8

  • Added support for Azure Blob Storage when running Azure Government Cloud
  • Fixed an issue in the Matillion ETL v1 API where the flag "onConflict=OVERWRITE" during job import was not working.
  • Fixed an issue where running a Transformation job more than once concurrently can cause it to interfere with itself.
  • Multiple bug fixes for "Assert" Components
    • Multi-line data types now supported
    • Fixed an issue where mismatched value types pass validation
    • No longer validates on size and precision fields where not relevant
    • Multiple fixes to error messages
    • Fixed an error when asserting values on a view containing a datetime column

Matillion ETL for Snowflake v1.46

New Components

  • New Zendesk Chat Extract component allows users to bring their Zendesk Chat data into an external table.
  • New Zendesk Talk Query component allows users to bring their Zendesk Talk data into a table.
  • New Salesforce Output component allows users to push data from their data warehouse to a Salesforce Object of their choice.
  • New Unpivot component can be used to convert table columns into additional table rows, the opposite of our existing Pivot component.
  • New MindSphere Extract component allows users to pull data from their MindSphere account into a table.
  • New “Assert” components can be used to test an existing object (Table, External Table, Scalar Variable or View) against user-defined criteria. This allows users to prevent jobs continuing if resources are not in their expected state.

New Features

  • New Recycle Bin feature gives users the opportunity to recover any deleted jobs before committing to delete them completely.
  • Additional Data Sources added to the Zuora Query component
    • Revenue Schedule
    • Revenue Event
    • Revenue Event Item
    • Revenue Event Item Invoice Item
    • Revenue Event Item Invoice Item Adjustment
    • Revenue Charge Summary Item
    • Revenue Charge Summary

Improvements

  • Reworked Component panel organization gives an alphabetized list of all connectors.
  • Connectors that import raw data directly into external tables have been renamed as “Extract” components.
  • A number of components have had property updates to better align with Snowflake.
  • Updated Salesforce Query driver

Matillion ETL for Snowflake 1.44.16

Fixed an issue where a certain Matillion ETL functions such as Migrate, Python Script and COPY INTO commands may cause instance credentials to be stored in log files.



Matillion ETL for Snowflake 1.44.14

  • Updated Google Ad Manager Query component from API v201905 to API v201908, anticipating the deprecation of v201905 in June 2020. Users can override this version change using the "Version" connection option with a value of "v201905".
 
 

Matillion ETL for Snowflake 1.44.13

  • Fixed an issue where Shared Jobs are not correctly running concurrent connections and transformation jobs.





Matillion ETL for Snowflake 1.44.11

 

Notes

  • Users experiencing issues pulling Ticket_Metrics columns in the Zendesk Query component while using a custom RSD config can see this article for potential solutions.


New Features

  • Project Admin assignment option added to External Security users.
  • Added new data sources for Zendesk Query component: group_memberships, groups, satisfaction_ratings, sla_policies, tags, ticket_audits, ticket_comments, ticket_fields, ticket_forms, ticket_metrics_events_incremental.


Fixes

  • Fixed an issue where Data Transfer component would fail to maintain a connection when loading from Sharepoint.
  • Fixed an issue where Zendesk Query component was bringing in erroneously-empty date fields.
  • Fixed an issue where Twitter Query component was attempting to use an outdated version of the AdStats data source.
  • Fixed "Error parsing int value [null]" error from Google Ad Manager Query component.
  • Non-functional 'FromFriendlyName' columns have been removed from Gmail Query component data sources.

 

Matillion ETL for Snowflake 1.44.10


Fixes

  • Reverted a change that affected column names generated by Extract Nested Data component which caused issues where jobs containing the Extract Nested Data component would break if created before 1.44.
 

Matillion ETL for Snowflake 1.44.9


Fixes

  • Fixed 'black screen' error caused by XSS filter being enabled by default.







Matillion ETL for Snowflake 1.44.8


New Features

  • Job Start Control allows you to ‘prevent duplicate job’ runs by selecting the option in the scheduler. When selected this allows users to specify that a scheduled job run should not be queued if that same job from a previous schedule is still running, preventing duplicates from stacking up in the queue.
  • Box and Dropbox source options added to Data Transfer Component, allowing users to move their stored files between cloud services.


New Components

  • Amplitude connector allows users to connect to their Amplitude account and load their data into external tables.
  • Box connector allows users to collect data regarding their stored files and load it into external tables.
  • Dropbox connector allows users to collect data regarding their stored files and load it into external tables.
 

Improvements

  • Add New Data Sources added to the Google Analytics Query component allows users a way of adding new tables to the "Data Sources" property without the need to deal with custom schemas.
  • Git Credentials can now be stored so users don’t need to re-enter credentials every time they want to fetch or push a branch.
  • Snowflake private key “passphrase” protection is now available when connecting to Snowflake, offering users an additional layer of security.
  • Project Admin option now added to users defined in Internal Security that allows these users to see and access all projects regardless of the project privacy settings.
 





Matillion ETL for Snowflake 1.42.11

  • Upgrade: Added OAuth 2.0 support for Xero component (OAuth 1.0 now deprecated)
  • Upgrade: Updated Xero Driver
  • Fix: Customer seeing Database Query validation errors after updating from 1.39.8 to 1.42.x
  • Fix: Create Table displaying errors when given a Snowflake Database Name containing a hyphen

 

Matillion ETL for Snowflake 1.42.10

  • Upgraded BigQuery driver to v19
  • Fix: SQS Queue erroneously disconnecting
  • Fix: postgreS error: "could not open relation with OID"
  • Fix: Intermittent Job Failures on Validation
 

 

Matillion ETL for Snowflake 1.42.8

  • Fixed various issues and bugs including:
    • Google OAuth creation to failing.
    • Query To Grid component would sometimes use the wrong database name in advance mode.
    • Instance performance would deteriorate over time when jobs run concurrently.
    • Jobs with many transformation jobs that should run in parallel were sometimes not doing so.
 

Matillion ETL for Snowflake 1.42


New Features

 New Components   Improvements



Matillion ETL for Snowflake 1.41.8

  • Pardot connector lets users connect to their Salesforce Pardot data.
  • Fixed an issue where Git Push to remote repository fails citing an 'Invalid Private Key'.
  • Fixed an issue where the Snowflake driver would cause an Out Of Memory error.
 

Matillion ETL for Snowflake 1.41.5

  • (AWS Hosted Instances Only) Fixed error when upgrading to a t3.xlarge instance that would cause schedules to run and unable to be stopped.
  • Increased possible Column Name lengths for Grid Variable mappings.




Matillion ETL for Snowflake 1.41



New Components


New Features
  • Go beyond a single connection to a target database with the new ‘Concurrent Connections’ option for faster concurrent loading (Enterprise Only)
  • Data Lineage allows you to understand the effect that your complex transformation jobs will have on your data. Track a column backwards to its source to determine where and how calculations are applied. (Enterprise Feature).

Improvements
  • Components
  • Additional Improvements
    • The Shared Jobs now allows creators to mark parameters as ‘required’ so users of the job must provide input for these parameters.
    • LDAP Configuration (External Security) now can be tested before confirming the configuration. The ability to run a test login is also available before committing changes.
    • Recursive Validation means that jobs linked to others via Run Orchestration and Run Transformation will be validated alongside their parent job.




Matillion ETL for Snowflake 1.40.11

  • Twitter driver upgraded to support Ads API v5
  • DoubleClick For Publishers (Google Ads Manager) driver upgraded to support API v201811 and later.
  • Windows Fileshare SMB2/3 Support for Data Transfer component and File Iterator component .
  • SAP Netweaver Query - update to the location of additional files including a "JCo" connection library and the Java wrapper "sapjco3.jar".
  • Fixed an issue where SAP Netweaver Query component was not correctly finding associated libraries.
  • Fixed an issue where Hubspot OAuth registration link would fail.
 

Matillion ETL for Snowflake 1.40 (Features Video)



New Components

New Features
  • Keypair Authentication now supported for Snowflake environment credentials.
  • Git functionality allows users to work with local and remote repositories, managing Project Versions as commits and branches seamlessly inside the Matillion ETL client. (Enterprise Only)
Improvements
  • User Interface
    • Changes to error and validation messaging in product.
    • Improve plus(+) sign and grid visibility.
    • UI Refresh - Tabs with no component selected.
    • Keyboard Shortcut Ctrl +A for single-line text input fields.
    • Order the Component Tree.
    • Add Manage Variables button in Code Editors.
    • Filter added to Sample Tab.
    • Simple JSON or XML parser for Configure API Profile Dialog.
    • All sorting of Project versions in Manage Versions.
  • Components
    • Add more JDBC Table Metadata To Grid Fields.
    • Dynamics CRM Query component now supports CRM2011accesible from CRM Version dropdown.
    • JDBC Incremental Load - Prepopulate URL Field with Connection Prefix.
    • Create File Format Component  has new properties available.
    • The API Profile Generator component is now able to find files in the current directory the user is in, rather than specifying the whole path of the file.
  • Additional Improvements
    • Remove row count from Result.java.
    • AMIs upgraded to Python 3.6.
    • Azure nc and jq packages are pre-installed.