Question

A retail analytics team lands raw daily CSV and Parquet files in a Cloud Storage data lake. Analysts run ad-hoc exploratory queries only a few times a week over recent files, and the raw files are frequently overwritten by an upstream process. The team wants to minimize storage duplication and avoid an ETL step, while accepting slower query performance for these occasional queries. What is the most appropriate way to make this data queryable in BigQuery?

Accepted Answer

Create BigQuery external tables (BigLake) pointing to the Cloud Storage files and query them in place.

Answer

Load the files into native BigQuery tables each day with a scheduled batch load job.

Answer

Copy the files into a Bigtable instance keyed by date and query through the BigQuery Bigtable connector.

Answer

Stream the file contents into BigQuery using the Storage Write API as files arrive.