Question

A data engineer manages a nightly Fabric pipeline that copies 200 GB of data from an Azure Blob Storage source into a Lakehouse table. The source data is currently stored as uncompressed CSV files, and the pipeline routinely exceeds its allotted maintenance window. The engineer wants to reduce copy duration and downstream query time without changing the source system's row count or schema. Which change will most effectively improve overall performance?

Accepted Answer

Convert the source files to Snappy-compressed Parquet before ingestion and load into a Delta table

Answer

Increase the pipeline's retry count and add a 5-minute retry interval to the copy activity

Answer

Enable staged copy through a Warehouse instead of writing directly to the Lakehouse

Answer

Schedule the pipeline to run during off-peak hours to avoid capacity contention