Question

A Generative AI Engineer builds a customer support assistant using a Databricks Foundation Model API chat endpoint. The system prompt currently reads: 'You are a helpful assistant. Answer the user's question.' Support reps complain that the model frequently produces long, hedging responses like 'I'm not entirely sure, but it might possibly be...' even when the retrieved context clearly contains the answer. The engineer wants direct, confident answers grounded strictly in the provided context, and an explicit statement when the context does not contain the answer. Which change to the prompt best achieves this?

Accepted Answer

Rewrite the system prompt to instruct the model to answer concisely using only the provided context, and to reply 'I don't have enough information to answer that' when the context is insufficient.

Answer

Increase the temperature parameter to 1.0 so the model generates more decisive and creative responses.

Answer

Append the user's raw question multiple times to the prompt to reinforce what is being asked.

Answer

Switch to an embedding endpoint so the model returns vectors instead of text, avoiding hedging language entirely.