Question

A media company has 40 genre-specific recommendation models, each trained on a different content category. Traffic to individual models is unpredictable and often sparse, but occasionally one genre spikes during a promotion. The team wants to minimize the number of endpoints they manage and reduce idle infrastructure cost, while still being able to invoke any single model on demand by specifying its identifier at request time. Which SageMaker deployment approach best fits these requirements?

Accepted Answer

Deploy all 40 models behind a single SageMaker multi-model endpoint (MME), loading models from S3 into memory on demand.

Answer

Deploy each model as a separate production variant on one real-time endpoint and route traffic by variant weight.

Answer

Create 40 individual real-time endpoints, one per model, each with its own autoscaling policy.

Answer

Deploy all 40 models in a single multi-container serial inference pipeline behind one endpoint.