DELTA LIVE TABLES – apply_changes from source dataset not working
I am using delta live tables in medallion architecture.I have columns id, timestamp, operation, deleted_flag. I have operations “Insert” and “delete”. When the operation is “insert” the deleted_flag will be null and for “delete” it will be yes. This is correctly populating in bronze table. but for silver table i am taking bronze table as source and performing apply_changes. But after that for series of insert and delete operations for same id , if last operation is “insert” then the deleted flag should be null. But i am getting as yes. Can someone please help me. I don’t want to delete old records in silver table
DELTA LIVE TABLES – apply_changes from source dataset not working
I am using delta live tables in medallion architecture.I have columns id, timestamp, operation, deleted_flag. I have operations “Insert” and “delete”. When the operation is “insert” the deleted_flag will be null and for “delete” it will be yes. This is correctly populating in bronze table. but for silver table i am taking bronze table as source and performing apply_changes. But after that for series of insert and delete operations for same id , if last operation is “insert” then the deleted flag should be null. But i am getting as yes. Can someone please help me. I don’t want to delete old records in silver table
How to get all parameters in advanced configuration for Databricks Delta Live Table pipeline?
I’d like to get list of parameters defined in Delta Live table configuration section