Question

Your company runs analytics workloads on Google Cloud but the leadership team has mandated a multi-cloud strategy to avoid vendor lock-in. They want the batch data processing logic to be able to run on Google Cloud today and potentially move to another cloud provider or on-premises in the future with minimal code rewrites. The data engineering team currently uses a mix of managed services. Which approach best supports this portability requirement?

Accepted Answer

Build the pipelines using Apache Beam with the Dataflow runner, and store data in open columnar formats such as Parquet on Cloud Storage.

Answer

Rewrite all pipelines as BigQuery SQL scripts and store the data in BigQuery native storage for best performance.

Answer

Use Cloud Data Fusion with proprietary connectors and store intermediate data in Bigtable.

Answer

Implement all processing in Dataprep by Trifacta recipes and export results to Google Sheets for downstream consumption.