Question

You are building a customer-facing RAG chatbot on Databricks using LangChain LCEL. The chain calls a pay-per-token Foundation Model API endpoint for generation. During load testing, occasional 429 rate-limit errors and transient endpoint failures cause the entire chain to fail and return errors to users. You want the chain to automatically try a secondary provisioned-throughput endpoint when the primary call fails, without rewriting the chain logic. Which LangChain approach best satisfies this requirement?

Accepted Answer

Wrap the primary model runnable with .with_fallbacks([secondary_model]) so LCEL invokes the backup endpoint when the primary raises an exception.

Answer

Increase the retriever's top_k so more context is passed, reducing the chance the model endpoint returns an error.

Answer

Set a higher temperature on the primary model so it produces output even when the endpoint is rate-limited.

Answer

Use RunnableParallel to always call both endpoints simultaneously and return whichever finishes first.