Matillion ETL for BigQuery Release Notes

Matillion ETL for BigQuery Release Notes


Attention: If you started your Matillion ETL instance at 1.42.x or earlier and have manually upgraded the Tomcat package to a newer version than 8.5.42 please be aware that upgrading may cause problems.If your Tomcat version has not been changed this should not affect you, although we would still recommend starting a new instance from the Marketplace to ensure no future upgrade issues.

We highly recommend reading the Updating Matillion ETL article for the safest way to update your instance. Do not hesitate to contact Matillion support for assistance in this matter


Matillion ETL for BigQuery v1.47.8

  • Fixed an issue where the Mailchimp Query component caused an Out Of Memory issue with certain data sources.
  • Fixed an issue where exported credentials via the API were not being encrypted as expected. Warning: Credentials exported prior to 1.47.8 may not be importable in 1.48.7 due to this change. It may be advisable to do an in-place upgrade if this is likely to be an issue for you.
  • Fixed an issue where Table Update would not properly validate when pointing to a table with a RECORD/STRUCT column.
  • Fixed an issue where the Database Query component would erroneously strip loading zeros from string fields.
  • Fixed an issue where Shopify Query was not fetching all data correctly.



Matillion ETL for BigQuery v1.47.7

  • Added the ability to query deleted records using the Zuora Bulk Query component.



Matillion ETL for BigQuery v1.47.6

  • Fixed an issue where the Matillion ETL client would get stuck loading projects.
  • Fixed an issue with the Postgres driver: "The SSLSocketFactory class provided org.postgresql.ssl.DefaultJavaFactory could not be instantiated".
  • Fixed an issue where the JDBC properties file could not longer be edited in 1.47.

Matillion ETL for BigQuery v1.47

Note: Due to some technical issues with this release, we do not recommend customers manually update to this version. We highly recommend users on 1.47 to update to 1.47.6 immediately.

New Features

  • New ‘Create your own connector’ feature. The new API Extract wizard and API Extract component allows you to connect you to any service that offers a REST API, allowing you to create your own connectors to load data.
  • OpenID Connect support for third party login providers:
    • You can now configure Matillion ETL to authenticate with any Open ID Connect provider.
    • Default support for Google, Microsoft and Okta plus a “Generic” option.

    Improvements



    Matillion ETL for BigQuery v1.46.8

    • Fixed an issue in the Matillion ETL v1 API where the flag "onConflict=OVERWRITE" during job import was not working.
    • Fixed an issue where running a Transformation job more than once concurrently can cause it to interfere with itself.
    • Multiple bug fixes for "Assert" Components
      • Multi-line data types now supported
      • Fixed an issue where mismatched value types pass validation
      • No longer validates on size and precision fields where not relevant
      • Multiple fixes to error messages
      • Multiple fixes to asserting Struct metadata types

    Matillion ETL for BigQuery v1.46

    New Components

    • New Zendesk Talk Query component allows users to bring their Zendesk Talk data into a table.
    • New Salesforce Output component allows users to push data from their data warehouse to a Salesforce Object of their choice.
    • New MindSphere Extract component allows users to pull data from their MindSphere account into a table.
    • New “Assert” components can be used to test an existing object (Table, External Table, Scalar Variable or View) against user-defined criteria. This allows users to prevent jobs continuing if resources are not in their expected state.

    New Features

    • New Recycle Bin feature gives users the opportunity to recover any deleted jobs before committing to delete them completely.
    • Additional Data Sources added to the Zuora Query component
      • Revenue Schedule
      • Revenue Event
      • Revenue Event Item
      • Revenue Event Item Invoice Item
      • Revenue Event Item Invoice Item Adjustment
      • Revenue Charge Summary Item
      • Revenue Charge Summary

    Improvements

    • Reworked Component panel organization gives an alphabetized list of all connector components.
    • Connectors that only import data into external tables have been renamed as “Extract” components.
    • Updated Salesforce Query driver
     

    Matillion ETL for BigQuery 1.44.14

    • Updated Google Ad Manager Query component from API v201905 to API v201908, anticipating the deprecation of v201905 in June 2020. Users can override this version change using the "Version" connection option with a value of "v201905".

     
     

    Matillion ETL for BigQuery 1.44.13

    • Fixed an issue where Shared Jobs are not correctly running concurrent connections and transformation jobs.

     



    Matillion ETL for BigQuery 1.44.11

    Notes

    • Users experiencing issues pulling Ticket_Metrics columns in the Zendesk Query component while using a custom RSD config can see this article for potential solutions.


    New Features

    • Project Admin assignment option added to External Security users.
    • Added new data sources for Zendesk Query component: group_memberships, groups, satisfaction_ratings, sla_policies, tags, ticket_audits, ticket_comments, ticket_fields, ticket_forms, ticket_metrics_events_incremental.


    Fixes

    • Fixed an issue where Zendesk Query component was bringing in erroneously-empty date fields.
    • Fixed an issue where Twitter Query component was attempting to use an outdated version of the AdStats data source.
    • Fixed "Error parsing int value [null]" error from Google Ad Manager Query component.
    • Non-functional 'FromFriendlyName' columns have been removed from Gmail Query component data sources.
     

     

    Matillion ETL for BigQuery 1.44.9


    Fixes

    • Fixed 'black screen' error caused by XSS filter being enabled by default.
     

    Matillion ETL for BigQuery 1.44.8


    New Features

    • Job Start Control allows you to ‘prevent duplicate job’ runs by selecting the option in the scheduler. When selected this allows users to specify that a scheduled job run should not be start if that same job from a previous schedule is still running, preventing duplicates from stacking up in the queue.
    • Box and Dropbox source options added to Data Transfer Component, allowing users to move their stored files between cloud services.


    New Components

    • Amplitude connector allows users to connect to their Amplitude account and load their data into external tables.
    • Box connector allows users to collect data regarding their stored files and load it into external tables.
    • Dropbox connector allows users to collect data regarding their stored files and load it into external tables.
     

    Improvements

    • Add New Data Sources added to the Google Analytics Query component allows users a way of adding users a way of adding new tables to the "Data Sources" property without the need to deal with custom schemas.
    • Git Credentials can now be stored so users don’t need to re-enter credentials every time they want to fetch or push a branch.
    • Project Admin option now added to users defined in Internal Security that allows these users to see and access all projects regardless of the project privacy settings.