Question

A data engineer needs to load only the CSV files from a cloud storage directory that also contains JSON and Parquet files mixed together. The engineer wants to use COPY INTO to load these files into an existing Delta table, ensuring only files matching a specific naming pattern (e.g., files starting with 'sales_') are ingested. Which combination of COPY INTO options should be used?

Accepted Answer

Use FILEFORMAT = CSV with the PATTERN option set to 'sales_*.csv'

Answer

Use FORMAT_OPTIONS('fileType' = 'csv') with a WHERE filename LIKE clause

Answer

Use FILEFORMAT = CSV and rely on schema inference to skip non-CSV files automatically

Answer

Use the cloudFiles.format option set to 'csv' with a glob PATTERN filter