Home › Study guides › CCAO-F › Domain 2 › Lesson 2.3

CCAO-F · Domain 2 · 21% of the exam · Lesson 2.3 · 18 min read

Fact-checking and validating Claude's output

How to fact-check Claude's work before it goes out: check claims against the source of record, open every citation, trace quotes and recompute numbers.

Written against objective 2.3 of the official CCAO-F exam guide (Version 1.0, effective July 2026). An independent resource, not affiliated with Anthropic; the practice questions are written from scratch.

2.3.1 Why a draft full of citations still needs checking

It is Wednesday afternoon, and the grant application is due on Friday. You write grants for a small education nonprofit that runs after-school reading tutoring, and this one asks a community foundation for two years of funding. Claude drafted the needs statement, the section that shows why the funding is needed, from the programme evaluation PDF you attached and from a web search for state figures. The search turned up the state education department's report and several news stories. The draft is persuasive. It contains eleven figures, and every one ends in a neat citation.

You could treat those citations as proof that the checking is done. Instead, you start looking the figures up, and before you are halfway through, three have failed in three different ways. One is a real statistic from an out-of-date edition. One cites a page that does not contain the figure. One is a percentage calculated wrongly from correct counts.

None of this makes Claude a poor tool for grant writing. It means a draft is a set of claims, and a citation is a note about where to look. Anthropic's Help Center is plain about the limit. Claude can display quotes that look authoritative but are not grounded in fact, and you should not rely on it as your single source of truth. Confidence is not evidence, and neither is a citation.

Fact-checking means confirming each claim that matters in an authoritative source. Validation is the routine around it: choosing the source, testing citations, tracing claims to evidence and recomputing numbers. Anthropic's AI Fluency course calls the habit behind both deployment diligence: taking responsibility for verifying and vouching for what you share. The foundation will hold your organisation to these figures, not Claude.

2.3.2 Check against the source of record

Here is the question hiding under most validation questions: checked against WHAT? The draft says 38% of third-graders in the state read below grade level, citing a news story. Check the figure against that story and it passes, because the story really does say 38%. But the story is not where the number was born. It repeated a figure from the 2019 edition of the state report, and the department has published a new edition every year since.

The place a fact is born is its source of record. That is the primary document that creates or officially publishes the fact, such as a regulation, a contract, a dataset or an official report, in its current, approved version. Everything else is a secondary summary, including Claude's own draft. Think of a friend who tells you the train leaves at 8:10. The tip is useful, but before you run you check the departure board. A summary tells you a fact exists and roughly where to find it; the check happens at the source.

Memorise the three roles. In an exam question, ask of each option which of the three it checks against.

Web claims need two extra questions: who published it, and when? A page from the agency that collected the data is a source of record; a page quoting it is a summary, however polished. The Help Center's advice for web search agrees: cross-reference the cited sources, and use authoritative sources for critical decisions. When no single body owns a claim, such as what research generally shows about how often tutoring should happen, confirm it in a second independent source. Independent means it reached the finding on its own rather than copying the first, so three websites repeating one number are still one source.

So you open the current edition on the department's own site. The same measure now reads 42%, not 38%, and the needs statement gets the current figure, cited to the report itself.

2.3.3 A citation is a pointer, not proof

The second failure is in a citation. The draft says participants "gained an average of 1.4 reading levels (Programme evaluation, p. 12)". A citation that specific feels like evidence. It names the right document, and a page number suggests someone looked. Yet page 12 describes attendance patterns and contains no reading gains at all.

A citation tells you where Claude found, or associated, a piece of information. It does not prove that the source says what the draft says, or that the source is the right one. Think of a library call number: it tells you which shelf to walk to, and you still have to open the book. So open every citation behind a claim that matters, and ask five questions in order.

The first three questions test the source. The last two test the fit between the source and your sentence, and a quick look misses them. Page 12 fails SUPPORTS: it is real, but it does not say this. A search for "1.4" finds the figure on page 17, but only for students who attended 30 or more sessions; the average for all participants was 1.1. So even the right page fails APPLIES, because the number belongs to a narrower group than the sentence describes. The claim becomes: an average gain of 1.1 reading levels, and 1.4 for students who attended 30 or more sessions.

Web citations fail in their own ways. The Help Center notes that links occasionally do not work, and that original pages may contain context missing from Claude's synthesis. A date range or a footnote can change what a figure means. A working link answers only the first question.

2.3.4 Ask for evidence, not reassurance

Two failures in, it is tempting to ask Claude, "Are you sure about the others?", or to have it rate each figure out of ten. Resist it: Claude will answer, but nothing new has been checked. A firm yes is Claude's say-so, and so is an apology followed by a new number. A self-rating scores the draft with the same judgement that wrote it.

What works is asking for evidence you can check instead of assurance you cannot. It starts with grounding: tying Claude's answer to documents you supply, so every figure can be traced to a page you hold. Anthropic's guide to reducing hallucinations recommends three moves for this. Tell Claude to use only the documents you provide, and give it a way to say a fact is not there. Ask for the exact quote that supports each claim. And have Claude remove any claim it cannot back with a quote.

One more line sorts the claims. Ask Claude to label each one as a fact stated in the source, an inference drawn from it, or an assumption with no source at all, because each needs a different check. A fact is checked against its quote. An inference needs your judgement on whether the quote really leads there. An assumption needs a source of its own, or it comes out.

Here is the prompt for the second draft, with all three sources of record attached. Look at the Quote and Type lines, and the instruction for a missing figure.

Rewrite the needs statement using only the three attached documents: the state education department's current annual report, our final programme evaluation and our enrolment records. Do not use web search or general knowledge for any figure.
Under every figure or factual claim, add three lines:
Source: document name and page.
Quote: the exact sentence or table row that supports the claim, copied word for word.
Type: FACT (stated in the source), INFERENCE (your conclusion from the source) or ASSUMPTION (not in the source).
If a figure I ask for is not in any of the documents, write [NOT IN SOURCES] instead of supplying one.
When you finish, reread each claim against its quote. Remove any claim the quote does not fully support and mark the gap with [ ].

The quotes turn a day of rereading into a short check: search the document for each one and confirm it is there, word for word, saying what the claim says. Until you find it, a quote is still a pointer. Keep the self-review line too, since it catches some unsupported claims. But it is Claude checking Claude, and Anthropic's guide says these techniques reduce hallucinations without eliminating them. The self-review shortens your list of doubts; it does not replace your check.

2.3.5 Recompute every number

Quotes catch a misread source, but not bad arithmetic. The third failure had no citation problem at all. The draft says enrolment grew from 240 students in 2023-24 to 372 in 2025-26, "a 65% increase". Both counts match your enrolment records. The percentage does not, yet nothing about the sentence looks wrong: a wrong figure in fluent prose looks exactly as sure as a right one.

The Help Center's guidance is direct: Claude can perform calculations, but complex or mission-critical ones should be verified with specialised mathematical software or by hand. A figure a funder may repeat in its own board papers is mission-critical. So recompute every number that carries weight: percentages, totals, averages and changes over time, anything worked out rather than copied.

There are two practical ways to recompute. Put the source figures in a spreadsheet and let a formula do the work, leaving a record anyone can recheck. Or upload the data file and ask Claude to calculate with code execution, the feature Anthropic calls code execution and file creation. Claude writes code, a sandboxed computing environment runs it on your file, and the number comes from a calculation that actually ran, not from the wording of Claude's reply. Ask Claude to show the formula and the input values. The feature is available on every plan, though on Team and Enterprise plans an owner can switch it off.

Either way, check two things no calculation can check for you. The inputs must be the right figures from the source of record, and the formula must measure what the sentence claims. Dividing 240 by 372 gives about 65%, the old count as a share of the new one, which is not growth. Growth is the change divided by the starting value, as the diagram shows.

2.3.6 The exam traps

Every trap in this objective swaps a real check for something that only feels like one. Of the four next steps a question offers, usually only one goes to the source.

• ✗ Sending a draft because every figure has a citation. ✓ Open the citations behind the claims that matter. A citation says where to look; it does not prove the source says that.
• ✗ Checking a figure against the article that quoted it. ✓ Go to the source of record in its current, approved version. A summary can be outdated or misread.
• ✗ Asking Claude "are you sure?" or asking it to rate its confidence. ✓ Ask for the exact supporting quote, then find it in the document yourself. Any reply is still Claude's say-so.
• ✗ Trusting a percentage because the words around it are right. ✓ Recompute it from the source figures in a spreadsheet or with code execution, and check the formula.
• ✗ Counting three websites that repeat a figure as confirmation. ✓ Trace them back. If they copy one original, they are one source; a second source must be independent.
• ✗ Rewording the draft, or deleting every figure, after finding one error. ✓ Fix the claims that fail and keep the ones that pass. Polish does not change accuracy, and dropping the evidence weakens the work.

2.3.7 Put it together: validate a draft before it goes out

You now have the whole routine. Find each claim's source of record, open every citation and ask the five questions, make Claude show quotes you can confirm, and recompute every number. By Thursday lunchtime all eleven figures have been through it, and the three that failed now say exactly what their sources support. The quickest way to make the routine stick is to watch grounding work, then take it away.

Checking tells you whether a claim is true; it does not settle who must sign off. Deciding when human review or extra verification is required (2.4) scales the checking to the stakes and names when a person with the right expertise must approve the work. Choosing output formats (2.6) turns the source, quote and type lines into an evidence table a reviewer can check at a glance. And in Domain 5, a grounding rule like "use only the attached documents and quote your source" moves into a Project's instructions, so every draft starts out checkable.

Key takeaways

• ✓ A claim is checked only when you have seen it in an authoritative source; Claude's confidence and citations are not evidence.
• ✓ Validate against the source of record, the primary document in its current, approved version; secondary summaries only point to it.
• ✓ For web claims, check who published it and when, and confirm a claim with no single owner in a second independent source.
• ✓ A citation is a pointer: confirm it exists, is authoritative, is current, applies to your case and supports the exact claim.
• ✓ "Are you sure?" and self-ratings are not validation; ground Claude in the attached documents, ask for the exact quote behind each claim, and find each quote yourself.
• ✓ Label each claim as a fact, an inference or an assumption, because each needs a different check, and treat Claude's self-review as a first pass, never the check.
• ✓ Recompute every number that carries weight, in a spreadsheet or with code execution, and check the inputs and the formula.

Check your understanding

4 questions written for this lesson, then one from the CCAO-F question bank on the same topic. Every answer option is explained, including the ones you did not pick. Nothing is stored.

Sources

• Reduce hallucinations
• Claude is providing incorrect or misleading responses
• Enable and use web search
• How does Claude handle mathematical equations and calculations?
• Create and edit files with Claude
• A closer look at Diligence (AI Fluency)
• The CCAO-F exam guide, summarised