Question

A media company uses a foundation model to generate short creative story openings for a children's book app. The team wants to measure how engaging, age-appropriate, and imaginative the outputs are before launch. Automated metrics like BLEU and ROUGE produced scores that did not match the team's own sense of quality. Which evaluation approach is MOST appropriate for assessing this type of output?

Accepted Answer

Use human evaluators with a scoring rubric covering engagement, appropriateness, and creativity

Answer

Rely on perplexity scores to rank the story openings by fluency

Answer

Compute BLEU scores against a fixed set of reference story openings

Answer

Measure exact-match accuracy against a labeled dataset of correct answers