Question

A retail company has trained 400 separate XGBoost models, one per product category, to predict demand. Each model is small (under 50 MB) and receives sporadic, low-volume traffic throughout the day. The ML engineering team wants to serve all models for real-time inference while minimizing the number of endpoints and controlling infrastructure cost. Which SageMaker deployment approach best meets these requirements?

Accepted Answer

Deploy a SageMaker multi-model endpoint that dynamically loads models from Amazon S3 into a shared container as they are requested

Answer

Deploy each of the 400 models to its own real-time endpoint with automatic scaling enabled

Answer

Deploy all 400 models using SageMaker Batch Transform jobs triggered on demand

Answer

Deploy a single real-time endpoint hosting one model and retrain it nightly with all category data combined