This article is specific to the following platforms - Delta Lake on Databricks.
Use the Run Notebook component to execute a Databricks notebook from a Matillion ETL orchestration job. You might find this component useful if you wish to conflate ETL/ELT tasks with the subsequent analysis performed in your Databricks notebooks.
|Name||String||A human-readable name for the component.|
|Cluster||Select||Select the Databricks cluster. The special value, [Environment Default], will use the cluster defined in the active Matillion ETL environment.|
|Notebook||String | Select||The path to a Databricks notebook. Search via a filepath string or browse the list of directories and notebooks based on the connected Workspace in the Matillion ETL environment.
Read Copy notebook path (AWS) or Copy notebook path (Azure) to learn how to retrieve your notebook file path.
|Execution Mode||Drop-down list||Asynchronous: Runs the task and sends a request to Databricks. The status of the task is ignored in terms of continuing the Matillion ETL job.
Synchronous: Runs the task and polls Databricks for a status update. The Matillion ETL job is delayed until Databricks has returned a status of
The default is asynchronous.
|Poll Interval (seconds)||Integer||The polling interval to Databricks. The default is 5 seconds. Only available when Execution Mode is synchronous.|