Medium — MLA-C01 Practice Quiz

Question 1

A data scientist is training a deep neural network on a sparse text-classification dataset using SageMaker. Training with plain stochastic gradient descent (SGD) at a fixed learning rate converges very slowly, and different features clearly need different effective learning rates because some tokens appear frequently while others are rare. The team wants an optimizer that adapts the per-parameter learning rate automatically to speed up convergence without extensive manual learning-rate tuning. Which optimizer change best addresses this need?

Accepted Answer

Switch to the Adam optimizer, which maintains per-parameter adaptive learning rates using running estimates of first and second moments of the gradients.

Answer

Keep SGD but increase the global fixed learning rate by 10x to force faster convergence across all parameters.

Answer

Keep SGD and add L2 weight decay to the loss so that rare-feature weights receive larger updates.

Answer

Switch to plain batch gradient descent computed over the full dataset each step to smooth the gradient signal.

Question 2

A media company deploys a document-processing model that accepts PDF uploads up to 500 MB and can take several minutes per request to run OCR and summarization. Requests arrive in unpredictable bursts, and during idle periods the team wants to avoid paying for running instances. Clients do not need an immediate synchronous response and can poll for results. Which SageMaker deployment option best fits these requirements?

Accepted Answer

SageMaker Asynchronous Inference with autoscaling configured to scale in to zero instances

Answer

A real-time endpoint with target-tracking autoscaling on invocations per instance

Answer

SageMaker Serverless Inference with provisioned concurrency

Answer

A batch transform job triggered on a fixed hourly schedule

Question 3

A data science team stores raw event logs as compressed JSON in Amazon S3. They repeatedly run the same complex SQL aggregation in Amazon Athena to build a feature table for model training, and each run scans hundreds of gigabytes, causing high query costs and slow iteration. They want a repeatable, cost-efficient way to persist the engineered feature dataset in an analytics-friendly format for downstream SageMaker training jobs. Which approach best meets these requirements with the least ongoing operational overhead?

Accepted Answer

Use an Athena CREATE TABLE AS SELECT (CTAS) statement to materialize the aggregated features into a new S3 location as partitioned Parquet, then point training jobs at that output.

Answer

Increase the Athena query result cache retention and rerun the same SELECT query before each training job so results are served from cache.

Answer

Export the query results manually from the Athena console to CSV and upload the file to the training bucket before each run.

Answer

Provision an Amazon Redshift cluster, load the raw JSON, and run the aggregation there for every training iteration.

Question 4

A data scientist trains a binary classifier to predict whether a customer will churn. Only 4% of customers in the dataset actually churn. The initial model reports 96% accuracy on the validation set, but the business complains that almost no real churners are being flagged. Which evaluation approach should the data scientist adopt to properly assess and compare model performance for this problem?

Accepted Answer

Evaluate the model using the area under the precision-recall curve (AUPRC) along with recall for the churn class, because these metrics focus on performance on the rare positive class.

Answer

Continue optimizing for overall accuracy, since 96% indicates the model generalizes well to the validation set.

Answer

Switch to root mean squared error (RMSE) to measure the distance between predicted and actual churn labels.

Answer

Increase the size of the training dataset until accuracy exceeds 99%, which will automatically improve churn detection.

Question 5

A retail company retrains a churn-prediction model weekly. Every night, they need to generate churn scores for their entire customer base of 40 million records stored as Parquet files in S3. The scoring job must complete before business hours, but there is no requirement for individual real-time predictions during the day. The ML engineer wants the most cost-effective deployment approach that avoids paying for idle compute. Which SageMaker deployment option should the engineer choose?

Accepted Answer

Use a SageMaker Batch Transform job scheduled nightly to process the S3 dataset

Answer

Deploy a real-time endpoint with autoscaling and invoke it in a loop each night

Answer

Deploy a Serverless Inference endpoint and send each record as an individual invocation

Answer

Configure an Asynchronous Inference endpoint and submit the full dataset as one payload

Question 6

A data scientist trains a binary classifier that flags manufacturing defects. The training set contains 4% defective units. The initial model achieves 96% accuracy but misses most defects during validation. Management requires that the model catch as many true defects as possible while keeping false alarms manageable, and they want a single metric that balances both concerns for model selection. Which evaluation metric should the team optimize?

Accepted Answer

F1 score, the harmonic mean of precision and recall

Answer

Overall classification accuracy on the validation set

Answer

R-squared computed against the defect labels

Answer

Mean absolute error between predicted and actual classes

Question 7

A machine learning team runs a production SageMaker real-time endpoint serving a recommendation model. They need to deploy a new model version but are concerned that a faulty version could cause a spike in inference errors affecting customers. They want SageMaker to automatically shift traffic to the new version gradually, monitor CloudWatch metrics during the shift, and automatically roll back to the previous version if error thresholds are breached. Which approach should they configure?

Accepted Answer

Use a blue/green deployment with canary traffic shifting and CloudWatch alarm-based auto-rollback in the endpoint deployment configuration

Answer

Create a second endpoint with the new model and manually switch the application's endpoint name after testing

Answer

Deploy the new model as a production variant on the same endpoint and manually adjust variant weights over several days

Answer

Use a batch transform job to validate the new model on historical data before replacing the endpoint

Question 8

A machine learning engineer is training a neural network in a SageMaker training job to classify support tickets into 12 mutually exclusive categories. The labels are provided as integer class indices (0 through 11) rather than one-hot encoded vectors. The final layer uses a softmax activation with 12 units. During training, the loss function raises a shape-mismatch error expecting the target to have the same shape as the model output. Which change resolves the error with the least preprocessing effort?

Accepted Answer

Switch the loss function to sparse categorical cross-entropy, which accepts integer class indices directly.

Answer

Change the final layer activation from softmax to sigmoid so it outputs per-class probabilities independently.

Answer

Reduce the number of output units to 1 and treat the task as ordinal regression on the class index.

Answer

Convert the softmax output to a single argmax value before computing standard categorical cross-entropy.

Question 9

An ML engineering team deploys SageMaker real-time endpoints across development, staging, and production accounts. Currently each environment is provisioned manually through the console, causing configuration drift and inconsistent instance types between accounts. The team wants a programmatic, version-controlled way to define the endpoint infrastructure using a familiar general-purpose programming language (TypeScript), reuse the same construct across all three accounts, and integrate the deployment into their existing CI/CD pipeline. Which approach best meets these requirements?

Accepted Answer

Use the AWS CDK to define the model, endpoint configuration, and endpoint as reusable constructs in TypeScript, synthesize CloudFormation templates, and deploy the same stack to each account through the CI/CD pipeline.

Answer

Write a Python script using the boto3 SDK that calls CreateModel, CreateEndpointConfig, and CreateEndpoint, and run it manually in each account when a deployment is needed.

Answer

Author a hand-written CloudFormation JSON template for each environment separately, editing instance types inline per account before uploading each template to the console.

Answer

Use SageMaker Pipelines steps to create the endpoints and rely on pipeline parameters to switch between accounts at runtime.

Question 10

A machine learning platform team wants to define their SageMaker infrastructure (endpoints, endpoint configs, and IAM roles) as code. The team is composed of Python developers who want to use loops, conditionals, and reusable classes to generate the infrastructure definitions, while still deploying through AWS's native provisioning and rollback engine. Which approach best meets these requirements?

Accepted Answer

Author AWS CDK stacks in Python, which synthesize into CloudFormation templates for deployment

Answer

Write raw CloudFormation templates in YAML and manually parameterize them for each resource

Answer

Use the AWS CLI in a shell script to create endpoints imperatively at deploy time

Answer

Use Terraform with the AWS provider to manage all SageMaker resources

Question 11

A security team wants to detect unusual patterns in network traffic logs to flag potential intrusions. The dataset contains millions of unlabeled records with numeric features such as packet count, byte volume, and connection duration. The team has no labeled examples of attacks and needs a scalable SageMaker built-in algorithm that can identify statistical outliers in this high-dimensional data. Which algorithm best fits this requirement?

Accepted Answer

Random Cut Forest (RCF)

Answer

XGBoost with a binary objective

Answer

Linear Learner in classifier mode

Answer

Factorization Machines

Question 12

A retail company wants to forecast weekly product demand across 5,000 store-item combinations. The dataset includes historical sales, promotional flags, holiday calendars, and price changes. The team needs a single model that can produce probabilistic forecasts (with prediction intervals) for all series simultaneously and learn shared patterns across related items. Which SageMaker built-in algorithm best fits this requirement?

Accepted Answer

SageMaker DeepAR

Answer

SageMaker Linear Learner

Answer

SageMaker XGBoost with a separate model trained per series

Answer

SageMaker Random Cut Forest

Question 13

A retail company wants to build an image classifier that identifies which of 40 product categories appears in a photo. They have only about 3,000 labeled images total (roughly 75 per category) and limited GPU budget. The ML team needs the fastest path to a reasonably accurate model without collecting more labeled data. Which modeling approach best fits this situation?

Accepted Answer

Use transfer learning with SageMaker's built-in image classification algorithm in transfer-learning mode, fine-tuning a pretrained network on the labeled images

Answer

Train a deep convolutional neural network from scratch on the 3,000 images using random weight initialization

Answer

Use the SageMaker K-Means algorithm to cluster the images into 40 groups and label the clusters by category

Answer

Use the SageMaker Linear Learner algorithm on the raw pixel values to classify the 40 categories

Question 14

A data science team at a real estate company needs to predict house sale prices from a tabular dataset containing 40 numeric and categorical features and about 120,000 rows. The relationships between features and price are highly non-linear, and the team wants strong predictive accuracy with minimal feature engineering while training efficiently on SageMaker. Which modeling approach should they choose?

Accepted Answer

Use the SageMaker XGBoost built-in algorithm configured for regression (reg:squarederror).

Answer

Use the SageMaker Linear Learner algorithm in regression mode.

Answer

Use the SageMaker K-Means algorithm to cluster homes and assign each cluster's mean price.

Answer

Use the SageMaker Random Cut Forest algorithm to score each record's price.

Question 15

A media company wants to test five different article headline variants on its homepage to maximize click-through rate. The traffic is high, and business stakeholders want to minimize the number of impressions wasted on clearly underperforming headlines while the test is running, dynamically shifting more traffic toward better-performing variants as evidence accumulates. Which modeling approach best fits this requirement?

Accepted Answer

A multi-armed bandit approach that adaptively allocates more traffic to higher-performing headlines during the experiment

Answer

A fixed 20/20/20/20/20 A/B/n split held constant until a predetermined sample size is reached, then a chi-square test

Answer

A linear regression model predicting click-through rate from headline text length

Answer

A time-series forecasting model to predict daily click volume for each headline

Question 16

A manufacturing company trains an XGBoost binary classifier in SageMaker to detect defective parts on an assembly line. Only 1.5% of parts are defective. The initial model achieves 98.5% accuracy but misses nearly all defective parts, which is unacceptable because undetected defects cause costly recalls. The team wants to improve the model's ability to catch defects while keeping the training pipeline simple and avoiding synthetic data generation. Which approach BEST addresses this problem?

Accepted Answer

Set the XGBoost scale_pos_weight hyperparameter to increase the penalty for misclassifying the minority (defective) class, and evaluate using recall and F1 rather than accuracy.

Answer

Increase the number of boosting rounds (num_round) and lower the learning rate (eta) to let the model train longer on the existing data.

Answer

Continue using accuracy as the objective metric but add more non-defective samples to increase the total training set size.

Answer

Switch the objective to reg:squarederror so the model outputs continuous defect scores instead of class labels.

Question 17

A team manages a SageMaker real-time endpoint entirely through a CloudFormation stack that is deployed identically across dev, staging, and production. During an incident, an on-call engineer manually changed the production endpoint's instance count and instance type through the SageMaker console to handle a traffic surge. Later, the team wants to programmatically detect whether the running production resources still match what the CloudFormation template declares, before running their next stack update. Which approach lets them identify these out-of-band changes with the least custom effort?

Accepted Answer

Run CloudFormation drift detection on the stack and review the resources reported as MODIFIED.

Answer

Enable termination protection on the stack so no manual changes can occur in the future.

Answer

Create an AWS Config custom rule with a Lambda function that queries the endpoint configuration on every change.

Answer

Delete and recreate the stack so the deployed resources match the template exactly.

Question 18

A financial services company must maintain a tamper-evident audit trail of all administrative API calls made against its SageMaker resources (creating endpoints, updating endpoint configs, deleting models) across every AWS Region. The compliance team needs to detect any unauthorized configuration changes and be able to query historical activity months later. Which approach best meets these requirements?

Accepted Answer

Enable a multi-Region AWS CloudTrail trail that logs management events to an S3 bucket with Object Lock enabled, and analyze the logs with Athena.

Answer

Enable SageMaker Model Monitor with a data quality baseline to capture all API calls and store violations in CloudWatch.

Answer

Turn on SageMaker Debugger and configure rules to flag any endpoint configuration changes across Regions.

Answer

Stream SageMaker endpoint invocation metrics to CloudWatch Logs and set metric filter alarms on administrative actions.

Question 19

An ML engineer runs a fraud-detection endpoint monitored by two SageMaker Model Monitor jobs: one for data quality drift and one for model quality (accuracy) degradation. Each job publishes a CloudWatch metric, and separate CloudWatch alarms already trigger an automated retraining pipeline via EventBridge. The team complains that during noisy input spikes, the data-quality alarm fires transient alerts and needlessly launches expensive retraining jobs, even when model accuracy is still acceptable. They want retraining to be triggered only when BOTH drift is detected AND accuracy has actually degraded, without deleting the existing individual alarms used for dashboards. What is the most operationally efficient way to meet this requirement?

Accepted Answer

Create a CloudWatch composite alarm with an ALARM rule that requires both the data-quality drift alarm and the model-quality alarm to be in ALARM state, and route only the composite alarm to the retraining EventBridge rule.

Answer

Increase the evaluation periods and datapoints-to-alarm on the data-quality alarm so it only fires after prolonged drift, and keep it as the sole retraining trigger.

Answer

Delete the two individual alarms and create a single metric-math alarm that averages the drift and accuracy metrics into one combined threshold.

Answer

Add a Lambda function subscribed to both alarm SNS topics that maintains state in DynamoDB and starts retraining only after receiving both notifications within a time window.

Question 20

A company hosts several real-time SageMaker endpoints for internal analytics teams. The ML platform lead notices the monthly inference bill has grown steadily, but many endpoints show very low invocation counts during business hours and none overnight. The lead wants an automated, low-effort way to identify underutilized endpoints so that the team can decommission or consolidate them, while avoiding manual review of each endpoint. Which approach best meets this goal?

Accepted Answer

Publish the SageMaker Invocations, CPUUtilization, and MemoryUtilization metrics to a CloudWatch dashboard and set CloudWatch alarms that notify the team via SNS when Invocations remains near zero over a multi-day period.

Answer

Create a CloudWatch alarm on the InvocationsPerInstance metric with a low static threshold and configure it to terminate the endpoint automatically.

Answer

Enable SageMaker Model Monitor data-quality monitoring on each endpoint to detect when no data is being captured.

Answer

Switch all endpoints to SageMaker Batch Transform so that idle compute is never provisioned.

Question 21

A machine learning engineer operates a real-time SageMaker endpoint serving a recommendation model. Recently, downstream applications have reported intermittent failures. The engineer wants to be automatically notified whenever the endpoint returns a sustained elevated rate of server-side errors (HTTP 5xx) so the on-call team can respond quickly. Which approach requires the least custom instrumentation to meet this requirement?

Accepted Answer

Create a CloudWatch alarm on the built-in Invocation5XXErrors metric emitted by the endpoint in the AWS/SageMaker namespace, with an SNS action to notify the on-call team.

Answer

Enable SageMaker Model Monitor data quality monitoring and configure it to publish a violation whenever 5xx errors exceed a threshold.

Answer

Add custom logging in the inference container to write error counts to CloudWatch Logs, then build a metric filter and alarm on the parsed count.

Answer

Use AWS X-Ray tracing on the endpoint and configure an alarm on trace error rates to trigger an SNS notification.

Question 22

A machine learning engineer manages a real-time SageMaker endpoint serving a recommendation model. Business stakeholders complain that during peak hours the recommendations feel slow, but the endpoint's average latency in CloudWatch looks acceptable. The engineer needs to detect and alert on the tail-latency experienced by the slowest requests so the team can react before customers are affected. Which CloudWatch approach best addresses this requirement?

Accepted Answer

Create a CloudWatch alarm on the p99 percentile statistic of the ModelLatency metric emitted by the endpoint.

Answer

Create a CloudWatch alarm on the Average statistic of the OverheadLatency metric emitted by the endpoint.

Answer

Create a CloudWatch alarm on the SampleCount statistic of the Invocations metric emitted by the endpoint.

Answer

Enable SageMaker Model Monitor data quality baselines to capture latency drift over time.

Question 23

A retail company runs three SageMaker real-time endpoints, each with Model Monitor data-quality schedules that emit violation counts as CloudWatch metrics. The operations team wants a single view where on-call engineers can see current drift violation counts across all three endpoints side by side, without navigating to each endpoint separately, and share it with stakeholders. Which approach requires the least custom code and best meets this requirement?

Accepted Answer

Build a CloudWatch dashboard with metric widgets that plot the Model Monitor violation metrics from all three endpoints together.

Answer

Enable AWS X-Ray tracing on all three endpoints and view the service map to observe drift violations.

Answer

Stream the Model Monitor CloudWatch metrics into Amazon Kinesis Data Firehose and build a QuickSight dashboard from the raw JSON.

Answer

Write a Lambda function that queries each endpoint's monitoring report from S3 every minute and posts an aggregated summary to an SNS topic.

Question 24

A data science team must build a text-classification model to route incoming support tickets into 12 categories. They have only 400 labeled tickets total but millions of unlabeled historical tickets. Management wants the highest accuracy possible while minimizing new labeling effort. Which modeling approach best fits this situation?

Accepted Answer

Fine-tune a pre-trained language model (transfer learning) on the 400 labeled tickets.

Answer

Train a linear classifier from scratch on the 400 labeled tickets using bag-of-words features.

Answer

Use k-means clustering on all tickets and label each cluster manually as one of the 12 categories.

Answer

Discard the labeled data and rely solely on rule-based keyword matching for each category.

Question 25

A data science team trains a new version of a customer churn classifier weekly. Each version is registered as a new model package version in an existing SageMaker Model Registry model package group. Before promoting a new version to production, the team must compare its offline evaluation metrics (AUC, F1) against the currently approved production version and record the decision, keeping a full history of which version was approved and by whom. Which approach best meets these requirements with the least custom development?

Accepted Answer

Store each version's metrics in the ModelMetrics of the model package, query and compare versions in the Model Registry, and update the ModelApprovalStatus to Approved for the chosen version.

Answer

Write each model's metrics to a custom DynamoDB table and build a Lambda function that compares them and copies the winning model artifact to a fixed 'production' S3 prefix.

Answer

Save evaluation metrics as CloudWatch custom metrics and use a CloudWatch dashboard to visually compare AUC across versions before manually redeploying the endpoint.

Answer

Enable SageMaker Model Monitor on the production endpoint and rely on its drift reports to decide whether the newest trained version should replace the current one.