Question

You are building an Azure AI Foundry agent for a customer support scenario where conversations can span dozens of turns. Users complain that after long sessions the agent begins to lose track of earlier details, and you also notice per-request token costs climbing steeply as the thread grows. You want the agent to retain the essential context of the entire conversation while keeping the prompt token count bounded. What is the most appropriate approach?

Accepted Answer

Implement a conversation summarization strategy that periodically condenses older turns into a running summary while keeping recent turns verbatim in the thread.

Answer

Increase the model deployment's max token limit and disable any truncation so the full thread is always sent to the model.

Answer

Lower the temperature parameter to 0 so the model focuses more consistently on the earliest messages in the conversation.

Answer

Create a new thread after every few turns and discard the previous thread to reset the token count.