Question

A retail company deploys a product-recommendation model to a SageMaker real-time endpoint. Traffic is steady during business hours but spikes 5x during flash sales that occur unpredictably. The team wants the endpoint to add instances automatically when load rises and remove them when traffic subsides, while keeping per-instance concurrency near a healthy level. What is the MOST appropriate way to configure this?

Accepted Answer

Attach an Application Auto Scaling target-tracking policy to the endpoint variant using the SageMakerVariantInvocationsPerInstance metric with a target value.

Answer

Manually update the endpoint's instance count with UpdateEndpointWeightsAndCapacities before each scheduled flash sale.

Answer

Switch the endpoint to a serverless inference configuration and set the maximum concurrency to the peak expected load.

Answer

Create a scheduled scaling action that increases instance count every day at a fixed time and decreases it at night.