Flatten Nested Json in PySpark

Hello Everyone,
This series is for beginners and intermediate level candidates who wants to crack PySpark interviews
Here is the link to the course : www.geekcoders.co.in/courses/...
visit to my website: www.geekcoders.co.in/s/store
json_sample="""{
"id": "0001",
"type": "donut",
"name": "Cake",
"ppu": 0.55,
"batters":
{
"batter":
[
{ "id": "1001", "type": "Regular" },
{ "id": "1002", "type": "Chocolate" },
{ "id": "1003", "type": "Blueberry" },
{ "id": "1004", "type": "Devil's Food" }
]
},
"topping":
[
{ "id": "5001", "type": "None" },
{ "id": "5002", "type": "Glazed" },
{ "id": "5005", "type": "Sugar" },
{ "id": "5007", "type": "Powdered Sugar" },
{ "id": "5006", "type": "Chocolate with Sprinkles" },
{ "id": "5003", "type": "Chocolate" },
{ "id": "5004", "type": "Maple" }
]
}"""
dbutils.fs.put("/FileStore/tables/pyspark_questions/18.json",json_sample,True)
#pyspark #interviewquestions #interview #pysparkinterview #dataengineer #aws #databricks #python #nestedjson #json

[@shivamchandan50]

plz make video on debugging in pyspark

[@bhargaviakkineni]

Please do a video how to flatten the data dynamically requesting sir

[@GeekCoders]

Sure