Question

A data engineer is investigating a slow-running dashboard query against a 2 TB table. In the Query Profile, they examine the TableScan operator and see: 'Partitions scanned: 48,120' and 'Partitions total: 48,200'. The WHERE clause filters on an event_date column. What does this observation most strongly indicate?

Accepted Answer

The query is scanning almost all micro-partitions, indicating poor pruning on the filtered column

Answer

Pruning is highly effective, so the bottleneck must be elsewhere, such as an exploding join

Answer

The result cache was used, which is why nearly all partitions appear in the profile

Answer

Disk spillage is occurring because too many partitions were loaded into memory