Data Preparation for Machine Learning

A data science team stores raw event logs as compressed JSON in Amazon S3. They repeatedly run the same complex SQL aggregation in Amazon Athena to build a feature table for model training, and each run scans hundreds of gigabytes, causing high query costs and slow iteration. They want a repeatable, cost-efficient way to persist the engineered feature dataset in an analytics-friendly format for downstream SageMaker training jobs. Which approach best meets these requirements with the least ongoing operational overhead?

A healthcare analytics team ingests raw CSV files into an S3 bucket that will be used to build a training dataset for a readmission-prediction model. Before the data can be used, compliance requires that any personally identifiable information (PII) such as names, phone numbers, and Social Security numbers be automatically discovered and redacted at scale during the ETL process. The team already uses AWS Glue for their transformation jobs. Which approach best meets this requirement with the least custom development?

A data engineer is preparing a customer transactions dataset for a fraud-detection model. One column, 'merchant_id', contains over 50,000 unique values and is highly predictive. The team wants to convert this feature into a numeric representation without creating tens of thousands of sparse columns that would explode dimensionality and slow training. Which feature-engineering approach best addresses this requirement?

A machine learning team at a fraud-detection company uses Amazon SageMaker Feature Store. They need to serve engineered features with single-digit millisecond latency to a real-time inference endpoint that scores credit card transactions as they occur. Separately, a data science team needs to query the full history of those same features to build training datasets and run batch analytics with Amazon Athena. Which configuration best meets both requirements?

A data engineer is preparing a large tabular dataset stored as CSV files in Amazon S3 to train a SageMaker built-in algorithm (Linear Learner). Training jobs are running slowly because the algorithm must parse the raw CSV text on every epoch. The engineer wants to use an AWS Glue ETL job to output the data in a format that SageMaker built-in algorithms can stream efficiently with Pipe mode. Which output format should the Glue job produce?

A data engineering team stores raw ML training data across three S3 prefixes, each containing CSV files with different but overlapping schemas. Analysts need to run ad hoc Athena queries to explore and validate the data before feature engineering, and the team wants to minimize the manual effort of maintaining table definitions as new files arrive daily. Which approach best meets these requirements?

A data scientist is preparing a dataset for a regression model that predicts customer lifetime value. One feature, 'annual_purchase_amount', is heavily right-skewed with a long tail of high-value customers, and the model's linear assumptions are being violated. The team uses AWS Glue DataBrew for their preparation workflow and wants to reduce skewness so the feature approximates a more normal distribution before training. Which transformation should they apply?

A data engineering team receives raw event data as JSON files in Amazon S3. Each record contains a nested array field called 'items', where each element holds product attributes for a single purchase. The ML team needs a flat, tabular dataset in which every product within an order becomes its own row so features can be computed per product. The team wants a low-code, visual approach that produces a reusable transformation recipe. Which approach best meets these requirements?

A data engineer is preparing a customer churn dataset for training a logistic regression model in Amazon SageMaker. The dataset contains a 'contract_type' column with three unordered values: 'month-to-month', 'one-year', and 'two-year'. The engineer wants to convert this nominal categorical feature into a numeric representation that does not impose any artificial ordinal relationship between the categories, using a low-code approach. Which transformation should the engineer apply?

A data engineer is preparing a tabular dataset of manufacturing sensor readings for a regression model. Exploratory analysis shows that a small number of temperature readings are physically impossible (e.g., 9999 degrees) due to sensor malfunctions, and these extreme values are skewing the model. The team wants a low-code, visual way to profile the data, identify these anomalies, and apply cleaning transformations before writing the results back to Amazon S3 for training. Which approach best meets these requirements with the least custom code?

A data engineer is preparing a customer transactions dataset in AWS Glue DataBrew for an ML churn model. The 'signup_date' column contains values in several inconsistent string formats (e.g., '01/15/2023', '2023-01-15', 'Jan 15 2023') because the data was merged from three legacy systems. Before the dates can be used to engineer tenure features, they must be standardized into a single consistent date type. What is the most efficient way to accomplish this within the data preparation workflow?

A data engineering team uses AWS Glue DataBrew to interactively clean and transform a customer dataset for a churn-prediction model. After building and validating a sequence of cleaning steps in a DataBrew project, they now need to apply the exact same transformations automatically to new monthly data files that arrive in Amazon S3, without repeating the interactive work each month. What is the MOST appropriate way to operationalize these transformations?

A data engineer is preparing a tabular dataset for a k-nearest neighbors model. The dataset includes 'annual_income' (values from 20,000 to 500,000) and 'age' (values from 18 to 90). During evaluation, the team notices that predictions are almost entirely driven by income differences, while age has negligible influence, even though domain experts consider age highly relevant. What is the MOST appropriate feature preparation step to address this?

A data engineer is preparing a tabular dataset of 5 million customer transactions stored as Parquet in Amazon S3 for a binary classification model. The team wants reproducible train/validation/test splits so that the exact same records land in each partition every time the preparation pipeline runs, regardless of the order in which Glue reads the files. Which approach best guarantees deterministic, reproducible splits?

A data engineer is preparing a customer dataset for a churn prediction model. The 'subscription_tier' column contains inconsistent values entered by different systems, such as 'Premium', 'premium', 'PREMIUM ', and 'Prem.'. These should all represent the same category before one-hot encoding. The engineer wants a low-code, visual approach to detect and consolidate these variants without writing custom Spark code. Which AWS Glue DataBrew capability best addresses this requirement?

A data engineer is preparing a ride-sharing demand forecasting dataset in an AWS Glue ETL job. The raw data contains a single 'pickup_timestamp' column. The team wants the model to learn recurring daily and weekly demand patterns (for example, that 11 PM and 1 AM are close in time, and that Sunday and Monday are adjacent). Which feature engineering approach should the engineer apply to the extracted hour-of-day and day-of-week values?

A retail company is preparing a customer dataset in AWS Glue for a churn prediction model. The source data was merged from multiple CRM systems and contains many duplicate customer records that are not exact matches — names have typos, addresses use different abbreviations, and phone formats vary. These fuzzy duplicates would inflate certain customers' behavior signals and bias the training data. The data engineering team needs a scalable way to identify and remove these near-duplicate records within their existing Glue ETL workflow. Which approach best addresses this requirement?

A machine learning engineer is preparing a dataset stored in Amazon S3 to train a binary classification model that predicts loan default. Before training, the team wants to measure whether the training data over-represents or under-represents specific demographic groups (such as age bands) relative to the positive label, so they can document and mitigate potential bias. The team wants a managed capability that produces bias metrics like class imbalance and difference in positive proportions in labels with minimal custom code. Which approach best meets this requirement?

A data engineer builds an AWS Glue ETL job that reads semi-structured JSON order records from S3 and writes them to a curated Parquet dataset used to train a demand-forecasting model. Some records store the field 'quantity' as an integer (e.g., 5) while others store it as a string (e.g., "5"). When the job writes to Parquet, Glue creates two separate columns, 'quantity_int' and 'quantity_string', causing the downstream training job to fail. Which approach resolves this schema conflict most efficiently within the Glue job?

A data engineering team runs an AWS Glue Spark ETL job that transforms 3 years of raw transaction logs into a curated dataset used to build ML training sets. Data scientists query this dataset in Amazon Athena, almost always filtering by a specific transaction_date range and by region. Currently the Glue job writes all output as a single set of Parquet files to one S3 prefix, and Athena queries scan the entire dataset, driving up cost and query time. Which change to the Glue job most effectively reduces the data scanned per query?