1.2x Matillion ETL for BigQuery Notes
Matillion ETL for BigQuery 1.42.11
- Upgrade: Added OAuth 2.0 support for Xero component (OAuth 1.0 now deprecated)
- Upgrade: Updated Xero Driver
- Fix: Customer seeing Database Query validation errors after updating from 1.39.8 to 1.42.x
Matillion ETL for BigQuery 1.42.10
- Upgraded BigQuery driver to v19
- Fix: SQS Queue erroneously disconnecting
- Fix: postgreS error: "could not open relation with OID"
Matillion ETL for BigQuery 1.42.8
- Fixed various issues and bugs including:
- Google OAuth creation to failing.
- Query To Grid component would sometimes use the wrong database name in advance mode.
Matillion ETL for BigQuery 1.42
- New API endpoints for Git Integrations in Matillion ETL. Use the Matillion ETL v1 API to initialise and clone repositories, switch branches, pull and push changes.
- Sage Intacct Query Component allows users to connect to their Sage Intacct data.
- Pardot Component offers a streamlined way of connecting to their Pardot account and querying data.
Matillion ETL for BigQuery 1.41.8
- Fixed an issue where Git Push to remote repository fails citing an 'Invalid Private Key'.
Matillion ETL for BigQuery 1.41.5
- Fixed an error where users would exceed the API limit when loading data into tables on BigQuery.
- Increased possible Column Name lengths for Grid Variable mappings.
Matillion ETL for BigQuery 1.41
- Instagram Component connects to your Instagram account and pulls data into Redshift.
- New Query Result To Scalar Component lets users write SQL queries to return scalar variables that can be used directly in Matillion ETL workflows.
- Data Lineage allows you to understand the effect that your complex transformation jobs will have on your data. Track a column backwards to its source to determine where and how calculations are applied. (Enterprise Feature).
- Additional Improvements
- The Shared Jobs now allows creators to mark parameters as ‘required’ so users of the job must provide input for these parameters.
- Added support to various components for “Numeric” data types.
- Increased support for components returning the Row Count export variable.
- LDAP Configuration (External Security) now can be tested before confirming config.
- Recursive Validation means that jobs linked to others via Run Orchestration and Run Transformation will be validated alongside their parent job.
Matillion ETL for BigQuery 1.40.11
- Twitter driver upgraded to support Ads API v5
- DoubleClick For Publishers (Google Ads Manager) driver upgraded to support API v201811 and later.
- Windows Fileshare SMB2/3 Support for Data Transfer component and File Iterator component .
- SAP Netweaver Query - update to the location of additional files including a "JCo" connection library and the Java wrapper "sapjco3.jar".
- Fixed an issue where SAP Netweaver Query component was not correctly finding associated libraries.
- Fixed an issue where Hubspot OAuth registration link would fail.
Matillion ETL for BigQuery 1.40 (Features Video)
- New data staging components for:
- Support for Structured Data
- Git functionality allows users to work with local and remote repositories, managing Project Versions as commits and branches seamlessly inside the Matillion ETL client. (Enterprise Only)
- Support for Structured Data
- Aggregate component now has Array Agg functionality.
- Create Table and Create External Table can not be configured to accept nested data.
- User Interface
- Changes to error and validation messaging in product.
- Improve plus(+) sign and grid visibility.
- UI Refresh - Tabs with no component selected.
- Keyboard Shortcut Ctrl +A for single-line text input fields.
- Order the Component Tree.
- Add Manage Variables button in Code Editors.
- Filter added to Sample Tab.
- Simple JSON or XML parser for Configure API Profile Dialog.
- All sorting of Project versions in Manage Versions.
- Add Nested Data To Sample Grid, Metadata Panel, and Environment Tree.
- Add more JDBC Table Metadata To Grid Fields.
- GCS Load Generator - Parity with GCS Load.
- Dynamics CRM Query component now supports CRM2011accesible from CRM Version dropdown.
- JDBC Incremental Load - Prepopulate URL Field with Connection Prefix.
- The API Profile Generator component is now able to find files in the current directory the user is in, rather than specifying the whole path of the file.
- Additional Improvements
- BigQuery Read Timeout allows users to set custom timeouts for all BigQuery operations.
- Remove row count from Result.java.
- AMIs upgraded to Python 3.6.
- Warning: This upgrade may prevent previously-installed Python modules from working. We recommend users install pip3.6 and then reinstall modules.
Matillion ETL for BigQuery 1.29.9
- New ServiceNow Query component loads data from Servicenow’s IT Service Management (ITSM) platform.
- New YouTube Analytics component can query data from the YouTube Analytics API.
- All query components now allow you to override the output table so you can specify an existing table to load or append to.
- Excel Query can now load files from Google Cloud Storage, as well as Amazon S3
- You only see S3 and/or GCS when you have credentials in the environment, otherwise they are hidden.
- New option to drop a schema from the Environment Tree.
- Specify a region in S3 Unload
- S3 / Google Cloud Storage file browser enhancements
- Set advanced connection options during OAuth flow (e.g. to connect to a Salesforce Sandbox)
- Warning: Manage Backups and View Audit haven't been removed, they have been moved to the Admin menu
Map Values component
IMPORTANT Upgrade Notes: All data-staging components now create a target table with a wider range of target data types. Mostly this will be transparent, however this may have an impact on downstream logic so please test jobs after upgrade.
Matillion ETL for BigQuery 1.28.7
- Improvements to Partitioned Tables support.
- You can create external tables over Google Drive files and Google Sheets.
- New Table Update component can update existing table in a transformation flow without rewriting all data.
- New iteration component Table Iterator allows you to iterate over an existing BQ dataset.
- Added support for Python scripts (Jython, Python2 and Python3) as part of Orchestration flows.
- New Except and Intersect Transformation components to compliment the Unite (Union) component.
- Enterprise Features
- Automatic Job Documentation. Matillion ETL can automatically generate documentation for your ETL process, this tool will recursively search all your jobs and include all job detail including linked notes and descriptions.
- Auditing of User Actions with searchable Audit Log provides fine grained audit of every change to an ETL process.
- Database Query now supports IBM Netezza data warehouses via JDBC.
- All transformation components support multiple outputs.
- Separate Replicate component not required.
- A new Admin Menu allows administrators to:
- Access the server log.
- Apply updates to the Matillion server without using the command line.
- Add, remove and configure users (using either an internal user database or external directory server).
- Configure SSL.