Evaluation, Safety, and Responsible Use — CCAO-F Practice Quiz

Question 1

Your startup is preparing to launch a consumer-facing wellness app built on Claude that offers general lifestyle and stress-management tips. During a pre-launch review, the product manager suggests marketing it as a tool that can 'diagnose mental health conditions and recommend prescription treatments' to differentiate from competitors. As the responsible-deployment lead, what is the most appropriate action aligned with Anthropic's usage policies and a responsible-deployment mindset?

Accepted Answer

Reject positioning the app as providing medical diagnosis or prescription advice, and instead scope it to general wellness with clear disclaimers and escalation to licensed professionals for high-stakes health concerns.

Answer

Approve the feature since Claude produces fluent, confident responses that users will find helpful and authoritative.

Answer

Launch as proposed but add a temperature setting of 0 so the medical advice is deterministic and therefore safe to rely on.

Answer

Allow the diagnostic feature but require users to accept a terms-of-service checkbox waiving all liability, which removes the responsibility from the deployer.

Question 2

A team builds a Claude-powered internal help desk that answers questions from a company knowledge base. During testing, they notice that when a user asks about a topic the knowledge base does not cover, Claude confidently invents a plausible-sounding but incorrect policy. The retrieval step is working correctly and only returns relevant passages. Which change to the prompt would most directly reduce these fabricated answers?

Accepted Answer

Explicitly instruct Claude that if the provided knowledge base passages do not contain the answer, it should say it does not know rather than guess.

Answer

Increase the temperature so Claude produces more varied responses across attempts.

Answer

Raise the max_tokens limit so Claude has more room to fully explain each policy.

Answer

Add a role prompt telling Claude it is a highly experienced, authoritative policy expert.

Question 3

Your team has built a test set of 500 prompts to evaluate a customer-support summarization feature before each release. Manually reading and scoring every model output takes two engineers a full day, which is delaying releases. The outputs are free-form summaries where correctness is somewhat subjective. Which approach best lets the team scale their evaluation while keeping quality judgments trustworthy?

Accepted Answer

Use an LLM-based grader with a clear rubric to score all 500 outputs automatically, and have humans spot-check a sample to validate the grader's alignment with human judgment.

Answer

Reduce the test set to 20 prompts so the two engineers can finish grading in under an hour each release.

Answer

Skip the evaluation for minor releases and rely on user complaints to surface any quality regressions after deployment.

Answer

Switch entirely to exact-string matching against a single approved summary for each prompt to make grading fully automatic.

Question 4

Your team maintains a customer-facing summarization assistant with an automated eval suite of 200 test cases that all pass with high scores. After deployment, users report the assistant occasionally invents product features that don't exist when summarizing sparse or off-topic support tickets. Your current test set contains only well-formed, feature-rich tickets. What is the most effective next step to improve your evaluation process?

Accepted Answer

Add representative test cases covering sparse, ambiguous, and off-topic tickets—including expected 'no relevant features found' outputs—so the eval catches the failure mode

Answer

Raise the passing score threshold on the existing 200 test cases so the eval is stricter overall

Answer

Switch from automated grading to manually reviewing every production output before it reaches users

Answer

Increase the model's temperature so summaries have more variety and feel less repetitive

Question 5

A team maintains a customer-support summarization prompt that scores 92% on their curated evaluation set of 200 labeled examples. A developer proposes a prompt change that they believe reads more clearly and 'feels better' in a few manual spot-checks. Before deploying the change to production, what is the most responsible next step consistent with an evals-driven workflow?

Accepted Answer

Re-run the full evaluation set against the new prompt and compare scores to the 92% baseline before deciding to deploy

Answer

Deploy the new prompt immediately since it reads more clearly and the developer's spot-checks looked good

Answer

Add the new prompt's spot-check outputs to the evaluation set as additional ground-truth examples

Answer

Increase the temperature on the new prompt so its outputs vary more and cover more cases during spot-checks

Question 6

A team is building a customer-facing assistant that drafts support replies. Leadership says the replies must be 'good.' Before writing any evals, the team wants to translate this into something they can actually measure. Which approach best defines useful success criteria for their evaluation?

Accepted Answer

Specify concrete, measurable targets across the dimensions that matter—such as factual accuracy against the knowledge base ≥95%, correct tone on a rubric, and no policy violations—each with a threshold.

Answer

Ask several team members to read a handful of outputs and give a thumbs-up or thumbs-down based on overall impression.

Answer

Set a single target of 'user satisfaction should improve' and check it after the assistant has been live for a quarter.

Answer

Adopt whatever benchmark scores the model reports on public leaderboards as the definition of quality for this use case.

Question 7

A team maintains a customer-support summarization prompt in production. Before shipping any prompt change, they want a reliable way to confirm the new version does not degrade quality compared to the current version. They have 200 representative support transcripts. Which approach best supports catching quality regressions when they update the prompt?

Accepted Answer

Establish scored baseline results for the current prompt on all 200 transcripts using a fixed grading rubric, then re-run and compare the new prompt's scores against that baseline before shipping

Answer

Run the new prompt on the 200 transcripts, spot-check a handful of outputs by reading them, and ship if those few look reasonable

Answer

Increase the temperature on the new prompt so it produces more varied summaries, making regressions easier to notice by eye

Answer

Ask the model to grade its own new summaries and approve the change if the model reports high confidence

Question 8

Your team built an evaluation set of 200 examples to measure the quality of a document-summarization prompt. Each summary can be correct in many valid phrasings, so exact string matching against a reference summary produces misleadingly low scores. You need a grading method that scores summaries reliably at this scale without a human reviewing all 200 outputs each run. Which grading approach best fits this evaluation?

Accepted Answer

Use an LLM-based grader ('model-graded eval') with a clear rubric that checks whether each summary captures the required key points and stays faithful to the source

Answer

Keep exact-match scoring but lower the pass threshold so more summaries count as correct

Answer

Drop the evaluation set and rely on the team informally reading a few outputs to judge whether the prompt 'feels' good

Answer

Have one engineer manually grade all 200 summaries by hand on every prompt iteration

Question 9

A team iterating on a customer-support classification prompt has been tuning it repeatedly against the same 50-example test set. Their scores keep improving, but when they deploy, real-world accuracy is noticeably lower than their eval suggested. What is the most likely cause and best corrective action?

Accepted Answer

The prompt has been overfit to the small test set; they should maintain a separate held-out set that they do not tune against and use it to validate generalization.

Answer

The model temperature is too high; they should set it to 0 and the eval scores will match production.

Answer

The test set is too large to be reliable; they should shrink it to 10 examples for faster iteration.

Answer

Automated grading is inherently inaccurate; they should abandon the eval and rely on manual spot-checks instead.

Question 10

A team has been improving a customer-support summarization prompt by reading a few outputs each day and tweaking wording whenever a summary 'feels off.' Recent changes seem to help some cases but silently break others, and no one can tell whether overall quality is trending up or down. Which change to their process best addresses this problem?

Accepted Answer

Establish a representative evaluation set with defined success criteria and graded metrics, then measure each prompt change against it before shipping.

Answer

Increase the model's temperature so summaries have more variety, making quality differences easier to notice by eye.

Answer

Have each team member review more sample outputs per day so subjective impressions are based on a larger informal sample.

Answer

Switch to a larger, more capable model so that manual spot-checking becomes unnecessary.

Question 11

Your team runs automated evaluations on Claude's customer-email drafts using another Claude call as an LLM-based grader. Reviewers notice the grader assigns wildly different scores to nearly identical drafts across runs, making it hard to trust regression results. What change would most improve the reliability of these automated grades?

Accepted Answer

Give the grader a detailed scoring rubric with explicit criteria and examples, and set a low temperature for the grading call

Answer

Increase the temperature of the grading call so the grader considers more diverse interpretations of quality

Answer

Reduce the test set to only the three drafts reviewers agree are perfect examples

Answer

Replace the LLM grader with a single human reviewer who scores drafts whenever they have time

Question 12

Your team built an eval suite for a customer-support assistant that generates free-form answers to billing questions. You chose exact-string match against golden answers as your grading metric. In practice, the assistant produces correct, well-phrased responses that differ in wording from your golden answers, so your eval reports a very low pass rate even though humans judge the outputs as good. What is the best fix?

Accepted Answer

Switch the grading approach to a semantic or LLM-based rubric grader that scores whether the response conveys the correct information, rather than requiring identical wording

Answer

Delete the golden answers and rely on periodic manual spot-checks by the team instead of an automated eval

Answer

Lower the temperature to 0 so the model always produces the exact wording in your golden answers

Answer

Reduce max_tokens so responses are shorter and more likely to match the golden answers exactly

Question 13

Your team has a customer-support summarization prompt in production, backed by a stable test set of 200 representative cases with defined success metrics. Anthropic releases a newer model version, and a teammate proposes switching to it because it 'feels smarter' after trying a few chat examples. Before committing to the new model, what is the most responsible evaluation practice?

Accepted Answer

Run the existing 200-case eval set against the new model and compare metric scores to the current baseline before deciding

Answer

Deploy the new model immediately to all users since newer versions are always improvements

Answer

Build a brand-new test set specifically flattering to the new model's strengths, then measure it

Answer

Have the teammate try 10 more chat examples and switch if the outputs subjectively look better

Question 14

Your team is iterating on a prompt for a customer-support summarization feature. A teammate proposes deciding whether each new prompt version is better by having two engineers read a handful of outputs and share their gut reaction in a Slack thread. As the person responsible for quality, what is the strongest objection to this approach and the better alternative?

Accepted Answer

Ad hoc 'vibe checks' on a few outputs are not repeatable or comprehensive; instead build a representative test set with defined success criteria and score each prompt version against it.

Answer

The approach is fine as long as the two engineers are senior enough; seniority is what makes manual review reliable at scale.

Answer

Manual review is too slow, so the team should skip evaluation entirely and ship whichever version was written most recently.

Answer

The real fix is to raise the temperature so outputs become more varied, making quality differences easier for reviewers to notice.

Question 15

A team has been improving a prompt for classifying customer support tickets into 12 categories. So far they have judged quality by manually reading 5 outputs after each prompt change and deciding whether it 'feels better.' They now want a repeatable way to know whether a change genuinely improves accuracy without introducing regressions in less common categories. Which change to their process best addresses this?

Accepted Answer

Build a labeled test set of representative tickets covering all 12 categories (including rare ones) with known-correct answers, and score each prompt version against it automatically.

Answer

Increase the temperature so the model explores more classification options, then keep reviewing 5 outputs by hand each time.

Answer

Keep the manual spot-check but add a second reviewer so two people read the same 5 outputs and average their impressions.

Answer

Switch to a larger model, which will make classifications accurate enough that a formal test set becomes unnecessary.

Question 16

A team building a customer-support summarization feature achieves 95% on their automated eval, which grades summaries by ROUGE overlap against reference summaries. However, support agents complain that the deployed summaries frequently omit the customer's actual requested action, making them useless in practice. What is the most likely root cause and the best corrective step?

Accepted Answer

The eval metric measures surface word overlap rather than whether the required outcome (capturing the requested action) is present; redefine the success criteria and add a targeted check that verifies the requested action is captured.

Answer

The model temperature is too high, causing inconsistent summaries; lower it to 0 and rerun the same eval.

Answer

The test set is too small; add more examples and the ROUGE score will better reflect real usefulness.

Answer

The prompt lacks a role assignment; give Claude a 'senior support analyst' persona and the omission problem will resolve.

Question 17

Your team builds an internal HR assistant on Claude that answers employee questions about company benefits and leave policies. During testing, you notice the model sometimes confidently invents specific policy details (like exact reimbursement caps) that don't exist in your documentation. Which change would MOST effectively reduce these hallucinations?

Accepted Answer

Provide the relevant policy documents in the prompt and instruct Claude to answer only from those sources, cite them, and say 'I don't know' when the answer isn't present

Answer

Raise the temperature setting so the model explores more varied phrasings of its answers

Answer

Add a system prompt telling Claude it is an expert HR professional with 20 years of experience

Answer

Increase max_tokens so Claude has more room to fully explain each policy in detail

Question 18

A company builds an internal HR assistant on Claude that answers employee questions about benefits. During testing, the team notices Claude sometimes invents plausible-sounding policy details (like specific reimbursement limits) that do not appear in the official benefits handbook. The handbook is already provided in the prompt as reference material. Which change would MOST directly reduce these fabricated details?

Accepted Answer

Instruct Claude to answer only using information found in the provided handbook, and to say it cannot find the answer when the handbook does not cover the question

Answer

Increase the temperature so Claude produces more varied and creative responses to employee questions

Answer

Remove the handbook from the prompt so Claude relies on its broad general knowledge of common HR practices

Answer

Raise the max_tokens limit so Claude has more room to fully explain each benefits policy

Question 19

A team builds a customer-facing chatbot that answers questions about their software product using an internal knowledge base. In testing, the bot occasionally invents feature names and version numbers that don't exist, stating them with full confidence. The team wants to reduce these fabricated details while keeping answers helpful. Which change most directly addresses the root cause?

Accepted Answer

Retrieve relevant knowledge-base passages and instruct Claude to answer only from the provided passages, cite the source, and say it doesn't know when the passages don't cover the question

Answer

Raise the temperature setting so responses are more varied and less likely to repeat the same fabricated details

Answer

Increase max_tokens so Claude has room to fully explain each feature it describes

Answer

Add a system prompt telling Claude it is a highly accurate expert who never makes mistakes

Question 20

A SaaS company builds a Claude-powered support agent that can call internal tools to resolve customer requests automatically. The product team wants to add a tool that permanently deletes a customer's entire account and all associated data when the customer asks. During a responsible-deployment review, what is the most appropriate way to handle this specific capability?

Accepted Answer

Require a human agent to review and confirm the deletion before it is executed, since account deletion is a high-stakes, irreversible action

Answer

Let Claude execute the deletion automatically as soon as it detects the customer's intent, to minimize response latency

Answer

Improve the tool's description and input schema so Claude rarely misinterprets the request, then allow fully autonomous deletion

Answer

Add adversarial test cases to the evaluation set so the deletion behavior is well tested, then deploy it without human oversight

More CCAO-F practice