How a Four-Layer Configuration Stops Claude Code from Fabricating Answers

In production, every coding agent can fabricate information, and Claude Code is no exception: it invents function names, imports, and claims tests passed without actually running them.

Why Claude Code Fabricates

The large model predicts the next plausible text. When it does not know whether a function exists, it generates a name that looks plausible in the project, and both real and fake names read naturally. Developers waste time debugging these false symbols.

Four‑Layer Solution

Layer 1: Honesty Rules in CLAUDE.md

Place the following at the top of a CLAUDE.md file (preferably within the first 50 lines):

## Honesty rules (read every turn)

Before claiming a function, class, or import exists, verify it by reading the file or running a grep. Never fabricate symbols.

If you cannot verify something, say "I haven't verified this" explicitly.
Do not write code that depends on the unverified claim.
If a task asks you to use a library you've never seen referenced in this project, ask before adding it.
If a task involved tests or builds, do not claim success unless you actually ran the test or build command in this session.
Never invent error messages, API responses, or stack traces. If you didn't see them, say so.
When you genuinely don't know, the correct answer is "I don't know" or "I need to check first." Both are better than a confident guess.

The key is the final sentence: the model must admit uncertainty instead of guessing.

Layer 2: Verification Protocol Before Writing Code

Add the following to CLAUDE.md:

## Verification protocol

Before writing or editing code that uses a symbol (function, class, type, constant), do one of:

1. Read the file where it's defined and confirm the signature
2. Run `grep -r "symbolName" .` or use the Glob tool to find it
3. Check package.json, requirements.txt, Cargo.toml, or equivalent for the dependency

If you skip verification, prefix the code with a comment:
`// UNVERIFIED: I have not confirmed this symbol exists`

Plan‑then‑execute mode is preferred for any task touching more than one file. Use Shift+Tab to enter plan mode before starting.

This adds a few tool calls but is usually cheaper than debugging a fabricated import.

Note that grep only proves a string appears; it cannot guarantee the symbol is callable. In TypeScript projects, tools like tsc, LSP, or SCIP are more reliable. In Python, use pyright or similar.

Layer 3: Hooks to Immediately Flag Fake Code

Configure Claude Code hooks so that after each file write, a type checker or linter runs automatically. Errors are fed back into Claude's context.

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Write(*.ts|*.tsx)|Edit(*.ts|*.tsx)",
        "hooks": [
          { "type": "command", "command": "npx tsc --noEmit --pretty false 2>&1 | head -20" }
        ]
      },
      {
        "matcher": "Write(*.py)|Edit(*.py)",
        "hooks": [
          { "type": "command", "command": "ruff check --quiet $file && pyright $file 2>&1 | head -20" }
        ]
      },
      {
        "matcher": "Write(*.rs)|Edit(*.rs)",
        "hooks": [
          { "type": "command", "command": "cargo check --message-format=short 2>&1 | head -20" }
        ]
      }
    ]
  }
}

Running a full tsc on large projects can be slow, so lightweight checks can stay in PostToolUse while heavier analysis runs in a Stop hook.

"Stop": [
  {
    "hooks": [
      { "type": "command", "command": "npm test 2>&1 | tail -30" }
    ]
  }
]

This prevents Claude from claiming tests passed without actually executing them. Hook output must go to stdout so Claude can see the error; silent logging is ineffective.

Layer 4: A Fact‑Checking Sub‑Agent

Create a sub‑agent that never writes code but verifies Claude's factual claims. Place it at .claude/agents/fact-checker.md:

---
name: fact-checker
description: Use this agent after Claude has made claims about what code does, what tests passed, or what a library supports. Invoke before any commit, before any user‑facing summary, and after any task that involved new dependencies.
tools: Read, Grep, Glob, Bash
model: sonnet
---

You verify claims, you do not write code.

When invoked, do this:

1. Identify every factual claim in the recent conversation. Examples:
   "the function X does Y", "the tests pass", "library Z supports W", "this import is correct".

2. For each claim, verify it independently:
   - Code claims: read the actual file and confirm
   - Test claims: run the tests yourself
   - Library claims: check the actual package or its docs
   - Import claims: confirm the package is in the dependency manifest

3. Produce a report:
   - VERIFIED: claim, evidence (file:line or command output)
   - WRONG: claim, what's actually true
   - UNVERIFIABLE: claim, why you couldn't check it

Never accept "trust me" claims. Never make claims of your own. If you can't verify, the correct output is UNVERIFIABLE.

Run this agent before committing or sending a summary so that only verified evidence is emitted.

How to Tell It Works

Claude asks before adding a new dependency.

It references file paths and line numbers, e.g., validateToken in src/auth/middleware.ts:47.

It explicitly says it hasn't verified something instead of presenting a guess as fact.

It leaves a receipt: commands run, files changed, external call results, and verification outcomes.

The reliability comes from the model stopping when it lacks evidence, not from getting every answer right.

Common Pitfalls

Writing a long handbook in CLAUDE.md – the model may skim; keep honesty rules short and at the top.

Only adding prompts without checkers – rules alone don't expose errors; the toolchain must surface them.

Hooks that run silently – errors must be fed back to stdout.

Skipping plan mode – planning before multi‑file changes reveals false assumptions early.

Forgetting to invoke the fact‑checker – it must be integrated into the commit workflow.

Penalizing the model for saying "I don't know" – encouraging honesty requires both configuration and user feedback.

Five‑Minute Implementation

Place the honesty rules and verification protocol into a CLAUDE.md at the project root (under 50 lines).

Add the hook JSON to ~/.claude/settings.json or .claude/settings.json.

Create the .claude/agents/fact-checker.md file with the content above.

Run a typical task you would manually review and verify that Claude now shows tsc, lint, or test output before claiming success.

The author notes this can reduce fabrication to near zero, though no silver bullet exists when the verifier itself is an LLM. The overall direction—making fabrication expensive by requiring type checks, tests, stdout evidence, file paths, and line numbers—is valuable in production.