Question

A financial services company is building a RAG chatbot using Amazon Bedrock Knowledge Bases backed by Amazon OpenSearch Serverless. During load testing with 50 million document chunks, the team notices that OpenSearch storage and OCU (compute) costs are much higher than budgeted, while retrieval latency at p99 exceeds their 400ms SLA. They currently use Amazon Titan Text Embeddings V2 configured to output 1024-dimension vectors. Business stakeholders confirm that a small reduction in retrieval accuracy is acceptable if it meaningfully reduces cost and latency. Which change should the team make FIRST?

Accepted Answer

Reconfigure Titan Text Embeddings V2 to output 256-dimension vectors and re-ingest the corpus, then re-evaluate retrieval quality against a labeled test set.

Answer

Switch the vector store from OpenSearch Serverless to a self-managed OpenSearch cluster on EC2 to gain finer control over shard sizing.

Answer

Enable provisioned throughput on the Titan embedding model to lower per-token inference cost during ingestion.

Answer

Increase the chunk size so fewer total chunks are stored, reducing the number of vectors in the index.