Relative Content

Tag Archive for pythonjsonpyspark

PySpark join fields in JSON to a dataframe

I am trying to pull out some fields from a JSONn string into a dataframe. I can achieve this by put each field in a dataframe then join all the dataframes like below. But is there some easier way to do this? Because this is just an simplified example and I have a lot more fields to extract in my project.

json.loads on records sometimes having nulls

I have a column that sometimes contains a geometry object ({“type”: “Point”, “coordinates”: [123.12345, 456.789]} and sometimes is null (this is expected, some items do not have geometry).