The from_csv function https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.sql.functions.from_csv.html , used to read a csv column in a dataframe, has a schema option. However this schema is expected to be a DDL string.