Question

A data engineer calls .cache() on a DataFrame that is reused across several downstream transformations, but query times have not improved. They open the Spark UI to confirm whether the DataFrame is actually being cached in memory. Which tab in the Spark UI should they check to verify how much of the DataFrame is cached and where it is stored?

Accepted Answer

The Storage tab, which lists cached RDDs/DataFrames with their fraction cached and memory/disk usage

Answer

The Jobs tab, which shows the DAG of stages for each action

Answer

The Environment tab, which lists Spark configuration properties

Answer

The Executors tab, which shows per-executor task counts and shuffle metrics