Collaborating within and across teams to manage data and models

A data science team prototypes models in a Vertex AI Workbench notebook using a 50 GB sample pulled into a Pandas DataFrame. The full training dataset lives in BigQuery and is roughly 8 TB, requiring complex windowed aggregations and joins across several tables. The team wants a preprocessing approach that scales to the full dataset for production training runs, minimizes custom infrastructure management, and produces feature tables that both training and future batch jobs can reuse. Which approach should they adopt?

Two data science teams share a Vertex AI Workbench environment to prototype churn models against a large BigQuery table. During prototyping, one team discovers that a colleague's earlier notebook produced a model with 4% higher AUC, but nobody can reproduce it because the underlying training data was overwritten by a nightly ETL job and the feature engineering code was iterated in place. Leadership asks you to establish a practice so future prototype results can be reliably reproduced and compared across teams. Which approach best addresses both data and model reproducibility?

A data science team prototypes models in Vertex AI Workbench notebooks. Their preprocessing code uses pandas to read and transform a 400 GB BigQuery table, but the notebook kernel repeatedly crashes with out-of-memory errors. The team wants to keep writing familiar pandas-style code in the notebook while pushing the heavy computation down to BigQuery so the data never has to fit in the notebook's memory. What should they do?

A data science team at a logistics company needs to explore three years of shipment event logs stored as newline-delimited JSON files in a Cloud Storage bucket (roughly 40 TB, organized in Hive-style date directories like /year=2023/month=05/). Analysts want to run ad hoc exploratory SQL in BigQuery to profile the data before committing to any modeling approach, but leadership is worried about query costs and does not want to duplicate the raw data into BigQuery-managed storage yet. Which approach best supports low-cost, iterative exploration?

Your ML team needs to preprocess 8 TB of raw clickstream logs stored in Cloud Storage before training. The pipeline must join the logs against a 200 GB user-profile table in BigQuery, apply session-based windowing, and write cleaned features back to BigQuery for downstream training. Multiple teams will reuse this pipeline on a scheduled daily basis. Which approach best satisfies scalability, reusability, and maintainability requirements?

Your team runs a nightly Dataflow batch pipeline that reads raw event logs from Cloud Storage, applies transformations, and writes the results to a BigQuery table used by three other teams for feature exploration. Recently, downstream data scientists complain that record counts vary between runs even when the input files are identical, and they see duplicate rows after failed-and-retried jobs. You need to guarantee that repeated pipeline runs produce a consistent, deduplicated table without manual cleanup. What should you do?

Your data engineering and ML teams share a clickstream pipeline. Raw events arrive continuously on a Pub/Sub topic and must be cleaned, windowed into 5-minute session aggregates, and written to BigQuery so both teams can build features from the same processed data. The processing logic includes stateful sessionization and must handle late-arriving events with watermarks. Which approach best meets these requirements?

A retail ML team preprocesses large volumes of transaction data for a churn model. During exploration in a Vertex AI Workbench notebook, engineers apply normalization and vocabulary-based categorical encoding using pandas. When the model is deployed, the online serving pipeline must apply the exact same transformations, but the team discovers subtle differences between training and serving preprocessing that degrade predictions. They want a scalable, reproducible approach that guarantees identical transformation logic in both batch training preprocessing and online serving. Which approach should they adopt?

You are an ML engineer joining a large enterprise with hundreds of BigQuery datasets spread across multiple projects owned by different business units. Before prototyping a churn model, you need to discover which existing tables contain customer transaction data, understand their schemas, check data quality metrics, and confirm you have appropriate access—without manually contacting each team. Which Google Cloud approach best enables this organization-wide data exploration and governance?

A data science team wants to explore several terabytes of raw event logs stored in Cloud Storage as Parquet files. They need to run ad hoc PySpark transformations and profiling from within their Vertex AI Workbench notebooks, but they do not want to provision or manage a persistent Spark cluster and want to minimize idle costs between exploration sessions. Which approach best meets these requirements?

Three ML teams at your company each maintain their own preprocessing pipelines that compute nearly identical customer-level features (e.g., 30-day purchase frequency, average session length). This leads to duplicated engineering effort, inconsistent definitions, and drifting values across teams. Leadership wants a governed, reusable approach so features are defined once and shared for both training and low-latency online serving. Which approach best addresses this?

A data science team of six people at a healthcare analytics company needs to collaborate on model prototyping using Vertex AI Workbench notebooks. Their requirements are: each member should have an isolated compute environment, all notebook code must be version-controlled and peer-reviewed before merging, and no protected health information (PHI) should be persisted in notebook outputs committed to the repository. Which approach best satisfies all three requirements?

A data science team is prototyping several scikit-learn models inside a Vertex AI Workbench notebook. They want to automatically capture parameters, metrics, and artifacts for each training run so that runs can later be compared in a shared experiment view, with minimal manual instrumentation code. Which approach best meets this requirement?

Two data science teams at your company are prototyping churn-prediction models in separate Vertex AI Workbench notebooks. One team uses TensorFlow, the other uses XGBoost. Leadership wants a single, consolidated view to compare all runs across both teams using the same metrics (AUC, log loss) and to reproduce the exact hyperparameters and datasets that produced the best model. Which approach best supports this cross-team comparison and reproducibility?

A data science team runs dozens of model prototyping iterations in Vertex AI Workbench notebooks. Each run produces a training dataset snapshot, a preprocessing script, hyperparameters, and a model artifact stored in Cloud Storage. Months later, an auditor asks the team to show exactly which dataset version and parameters produced a specific model that was later promoted to production. The team currently only logs metrics to a spreadsheet. What is the most effective way to reliably capture this lineage going forward with minimal custom tooling?

A data science team is prototyping several TensorFlow model variants in Vertex AI Workbench notebooks. Different members run experiments independently, changing hyperparameters and preprocessing steps. Leadership wants a centralized, reproducible way to log parameters, metrics, and artifacts for each run so the team can compare model performance across everyone's experiments and identify the best configuration. What should the team do?

Your data science team maintains a BigQuery table that is refreshed daily by a Dataflow pipeline with customer-level aggregated features. Several downstream teams want to retrieve the latest feature values with low-latency online lookups for their real-time recommendation services. You want to expose these features from the existing BigQuery source with minimal engineering effort and keep the online store automatically up to date after each daily refresh. Which approach should you use?

Your ML team serves a real-time fraud detection model on Vertex AI. Multiple teams independently compute customer features (e.g., 30-day transaction totals) in their own pipelines, and you've discovered that the feature values used at training time differ from those fetched at prediction time, causing training-serving skew. The team also wants to reuse these features across three other models. Which approach best addresses both the skew and the reuse requirements?

A data science team at a lending company is building a training dataset for a loan default model. Feature values (customer balances, transaction counts) are updated daily and stored in Vertex AI Feature Store. Each training label corresponds to a loan application timestamp that varies per row. During an early prototype, the model performed suspiciously well offline but degraded in production. The team suspects features observed after the application date leaked into training. What should they do to correctly assemble the training data?

Two ML teams at a logistics company both consume a shared 'shipment_features' set from Vertex AI Feature Store. The routing team requests a schema change that adds a new derived feature and modifies the logic of an existing one. The demand-forecasting team's production models must continue to reproduce predictions using the exact feature definitions they trained on, while the routing team needs the updated definitions immediately. What approach best satisfies both teams while preserving reproducibility?