Question

A data engineer needs to build a Silver-to-Gold transformation that, for each product category, computes the total revenue, the average order value, and the count of distinct customers in a single DataFrame operation. Which PySpark code correctly produces all three metrics per category?

Accepted Answer

df.groupBy("category").agg(sum("revenue").alias("total_revenue"), avg("order_value").alias("avg_order_value"), countDistinct("customer_id").alias("distinct_customers"))

Answer

df.groupBy("category").sum("revenue").avg("order_value").countDistinct("customer_id")

Answer

df.select("category").agg(sum("revenue"), avg("order_value"), countDistinct("customer_id"))

Answer

df.groupBy("category").agg(sum("revenue", "order_value"), count("customer_id"))