Question

A startup deploys a document-classification model behind a SageMaker endpoint. Traffic is highly intermittent: several bursts of requests during business hours followed by long idle periods overnight and on weekends. Each request has a small payload (under 1 MB) and must return synchronously within a few seconds. The team wants to minimize cost and avoid paying for idle compute, and they are willing to tolerate occasional cold-start latency. Which deployment option best fits these requirements?

Accepted Answer

A SageMaker Serverless Inference endpoint with an appropriate memory size and max concurrency

Answer

A real-time endpoint on a single ml.m5.large instance with target-tracking autoscaling down to a minimum of one instance

Answer

A SageMaker Batch Transform job scheduled to run every hour to score accumulated requests

Answer

A SageMaker Asynchronous Inference endpoint with autoscaling configured to scale in to zero instances