Question

A data engineer is transforming a bronze Delta table where each row represents an order with a column `items` containing an array of struct values (each struct has `product_id` and `qty`). The goal for the silver layer is to produce one row per individual item, keeping the parent `order_id`. Which PySpark transformation correctly flattens this structure?

Accepted Answer

df.select("order_id", explode("items").alias("item")).select("order_id", "item.product_id", "item.qty")

Answer

df.select("order_id", collect_list("items").alias("item"))

Answer

df.groupBy("order_id").agg(count("items").alias("item_count"))

Answer

df.select("order_id", flatten("items").alias("items"))