Question

A data engineer runs a nightly pipeline that uses a Copy activity to load about 40 GB of CSV data from Azure Blob Storage into a Fabric warehouse table. The load consistently takes several hours, and monitoring shows the bottleneck is row-by-row inserts into the warehouse. The engineer wants to significantly improve throughput without changing the source format. What should they do?

Accepted Answer

Enable staging on the Copy activity so data is bulk-loaded via an intermediate staging location instead of using inline inserts.

Answer

Increase the pipeline's timeout value and add a retry policy to the Copy activity.

Answer

Split the source into multiple smaller pipelines that each run on a different schedule.

Answer

Convert the target warehouse table to a heap and disable statistics collection during the load.