Question

A media company built a foundation-model application that generates short summaries of news articles. Before launch, the team wants an automated, repeatable way to measure how closely the model-generated summaries match a set of human-written reference summaries. Which evaluation approach is most appropriate for this task?

Accepted Answer

Use the ROUGE metric to compare generated summaries against the reference summaries

Answer

Track the model's inference latency and cost per token during summary generation

Answer

Measure the F1 score of the classifier that categorizes each article by topic

Answer

Monitor the temperature parameter to ensure summaries remain deterministic