.png){kind=logo}
Multi Table Input
-
DarkLight
Multi Table Input
-
DarkLight
This article is specific to the following platforms - Snowflake - Redshift - Synapse - Delta Lake.
Multi Table Input Component
Read chosen columns from an input table into the job.
The difference between Table Input and Multi Table Input is that Multi Table Input reads data from many input tables based on filtering all available input tables matching a pattern.
The matching tables are expected to be very similar, e.g. Budgets_2012, Budget_2013 and so on, with a common set of columns.
Properties
Snowflake Properties | ||
|---|---|---|
| Property | Setting | Description |
| Name | Text | A human-readable name for the component. |
| Pattern Type | Select | ILike: The available tables are filtered using a case-insensitive SQL syntax pattern. See Snowflake ILike documentation.
Like: The available tables are filtered using a case-sensitive SQL syntax pattern. See Snowflake Like documentation. Regex: The available tables are filtered using a POSIX EXE Regular Expression comparison. See Snowflake Regex documentation. |
| Database | Select | Choose a database to create the new table in. |
| Schema | Select | Select the table schema. The special value, [Environmental Default] will use the schema defined in the environment. For more information on using multiple schemas, see this article. |
| Pattern | Text | The pattern to match available tables to. The pattern syntax depends upon the chosen Pattern Type (see above). |
| Columns | Choice | The available columns are generated by first scanning the available tables, and then providing all columns from any of the inputs. It is expected that the tables matching the pattern are fairly similar and share many columns. Use the Editor to select which columns to pass along. |
| Cast Types | Boolean | True: If the same-named column from multiple tables has a different data type, attempt to cast to a common type. Default is False.
False: If the same-named column from multiple tables has a different data type, it is reported as an error. Default is False. |
| Add Source Table Name | True / False | When set to True, Matillion adds a column, "source_table", containing the input table name that was matched to provide this row. |
Redshift Properties | ||
|---|---|---|
| Property | Setting | Description |
| Name | Text | A human-readable name for the component. |
| Schema | Select | Select the table schema. The special value, [Environmental Default] will use the schema defined in the environment. For more information on using multiple schemas, see this article. External schemas are not supported. |
| Pattern | Text | The pattern to match available tables to. The pattern syntax depends upon the chosen Pattern Type (see below). |
| Pattern Type | Select | Like: The available tables are filtered using an SQL Like comparison. See Redshift Like documentation.
Similar To: The available tables are filtered using an SQL Similar To comparison. See Redshift Similar To documentation. Regex: The available tables are filtered using a POSIX Regular Expression comparison. See Redshift Regular Expression documentation. |
| Column Names | Choice | The available column names are generated by first scanning the available tables, and then providing all column names from any of the inputs. It is expected that the tables matching the pattern are fairly similar and share many columns. |
| Cast Types | Select | Yes: If the same-named column from multiple tables has a different data type, attempt to cast to a common type. Default is No.
No: If the same-named column from multiple tables has a different data type, it is reported as an error. Default is No. |
| Add Source Table Name | Yes / No | When set to Yes, adds a column, "source_table" containing the input table name that was matched to provide this row. |
| Trim Columns | Select | Wraps the column names in a BTRIM function, which will strip out all the leading and trailing spaces. See the Redshift documentation for details. |
Synapse Properties | ||
|---|---|---|
| Property | Setting | Description |
| Name | String | A human-readable name for the component. |
| Schema | Select | Select the table schema. The special value, [Environmental Default], will use the schema defined in the environment. For more information on using multiple schemas, see this article. |
| Pattern | String | The pattern to match available tables to. For more information, please refer to the Microsoft Azure documentation. |
| Columns | Column select | Select the one or more columns to load. |
| Cast Types | Boolean | A CAST command converts an expression of one data type to another. Default is true. For more information, please refer to the Microsoft Azure documentation. |
| Add Source Table | Boolean | Select whether to add the source table to the load. The default setting is false. |
Delta Lake Properties | ||
|---|---|---|
| Property | Setting | Description |
| Name | Text | A human-readable name for the component. |
| Catalog | Select | Select a Databricks Unity Catalog. The special value, [Environment Default], will use the catalog specified in the Matillion ETL environment setup. Selecting a catalog will determine which databases are available in the next parameter. |
| Database | Select | Select the Delta Lake database. The special value, [Environment Default], will use the database specified in the Matillion ETL environment setup. |
| Pattern | Regular Expression | The regular expression pattern used to filter out unwanted tables. Except for * and | characters, the pattern works like a regular expression. * alone matches 0 or more characters and | is used to separate multiple different regular expressions, any of which can match. The leading and trailing blanks are trimmed in the input pattern before processing. The pattern match is case-insensitive. |
| Columns | Column Selector | Select columns to include in the load. Move columns from the left list to the right list to include them. The available columns are generated by first scanning the available tables, and then providing all columns from any of the inputs. It is expected that the tables matching the pattern are fairly similar and share many columns. |
| Cast Types | Boolean | When True, if the same-named column from multiple tables has a different data type, attempt to cast to a common type. When False, if the same-named column from multiple tables has a different data type, an error is reported. The default setting is False. |
| Add Source Table | Boolean | When True, Matillion ETL adds a column, "source_table", containing the input table name that was matched to provide this row. The default setting is False. |
Strategy
Generates a set of select statements, concatenated together using UNION.