Question

A retail analytics team includes several business analysts who understand data transformation logic but do not write code. They need to build repeatable ETL pipelines that ingest CSV files from Cloud Storage, join them with reference data, apply cleansing rules, and load the results into BigQuery. Leadership wants the analysts to be self-sufficient, wants the pipelines to be version-controlled, and prefers a managed service that provides a visual, drag-and-drop interface with prebuilt connectors and transformations. Which approach best meets these requirements?

Accepted Answer

Use Cloud Data Fusion to build the pipelines through its graphical interface, leveraging built-in connectors and Wrangler for transformations

Answer

Have the analysts write Apache Beam pipelines in Python and deploy them as Dataflow Flex Templates

Answer

Provision a Dataproc cluster and have the analysts submit PySpark jobs that perform the joins and cleansing

Answer

Load the raw CSVs into BigQuery and have the analysts write scheduled SQL scripts for all transformations