Relative Content

Tag Archive for pysparkdatabricksdelta

Is the write operation in the delta table performed when schema mismatch is detected

I have one external delta table.It has given below issue.It is related to schema mismatch.I have modified the code to do schema enforcement that is if same columns as of target table present then continue if incoming data does not have target table columns then add those columns with None value.I just want to know has there any write operation performed if schema mismatch is detected.My hunch is if any write operation is performed then I might have to purge those data.Since table is huge I don’t want to take that risk:

Is the write operation in the delta table performed when schema mismatch is detected

I have one external delta table.It has given below issue.It is related to schema mismatch.I have modified the code to do schema enforcement that is if same columns as of target table present then continue if incoming data does not have target table columns then add those columns with None value.I just want to know has there any write operation performed if schema mismatch is detected.My hunch is if any write operation is performed then I might have to purge those data.Since table is huge I don’t want to take that risk: