Question

A company is deploying an internal LLM-based assistant that uses a tool/function-calling framework, allowing the model to invoke backend APIs (ticketing, HR lookups, database queries) on behalf of employees. Security testing shows that a crafted prompt can coax the model into chaining tool calls to retrieve data the requesting user is not authorized to see. Which control most directly reduces the impact of this attack?

Accepted Answer

Enforce authorization at each tool/API endpoint using the requesting user's identity and scope, rather than relying on a single service account for all model tool calls

Answer

Increase the model's context window so it can retain the full conversation history and better track prior authorization decisions

Answer

Add a system prompt instructing the model to refuse requests for unauthorized data and to only return information the user is permitted to access

Answer

Fine-tune the model on a curated dataset of approved queries so it learns to avoid generating unauthorized tool calls