Question

A data engineering team needs to generate embeddings for a one-time backfill of 40 million product descriptions using Amazon Bedrock's Titan Text Embeddings model. The job has no strict deadline and can run over several days. During an initial test using synchronous InvokeModel calls from a fleet of Lambda functions, the team hits frequent ThrottlingException errors and their per-embedding cost is higher than expected due to Lambda invocation overhead and idle wait time. Which approach best optimizes cost for this workload?

Accepted Answer

Submit the embedding generation as an Amazon Bedrock batch inference job, which processes records asynchronously from S3 at a lower per-token price than on-demand.

Answer

Purchase provisioned throughput with a 6-month commitment for the Titan Embeddings model to guarantee capacity for the backfill.

Answer

Request a service quota increase for on-demand InvokeModel requests-per-minute and add exponential backoff with jitter to the Lambda retry logic.

Answer

Enable prompt caching on the embedding requests so repeated product descriptions reuse cached embeddings and reduce token charges.