Passing table names as parameters from dataflow to notebook in pipelines
I want to create a pipeline with two elements. One is a dataflow. This dataflow creates 5 tables in my lakehouse. The second element of the pipeline is a notebook. I want the notebook to load each of these 5 tables, make some transformations on the data. Then the 5 tables need to appended to each other, and added to the lakehouse (if a table exists, it is replaced).