Question

A logistics company is building an assistant that answers questions about uploaded warehouse photos, such as 'Is the emergency exit blocked?' Auditors require that every answer include verifiable visual evidence — the specific region of the image the model used to reach its conclusion — so answers cannot be based on hallucinated details. Which approach best satisfies this grounded visual question-answering requirement?

Accepted Answer

Use a multimodal model that returns both a natural-language answer and the bounding-box coordinates of the referenced objects/regions, and surface those regions alongside the answer

Answer

Generate a dense caption for the whole image first, then run a text-only LLM over the caption to answer the question

Answer

Increase the temperature on the multimodal model so it produces more detailed, descriptive answers about the image

Answer

Store the image in a vector index and return the nearest-neighbor image as the evidence for the answer