减少 AI 生成代码中的幻觉：一份审查清单

AI coding agents are extraordinarily good at sounding correct. When they generate code that looks plausible but does not work, the model is not lying. It is hallucinating: producing output that matches the statistical patterns of correct code without implementing a correct solution.

This article is for engineers and agent operators. It covers verifiable checks you can run in a harness, a PR review, or a self-critique prompt before you merge. If you are a non-programmer reviewing AI output by behaviour rather than by code, use How to Review AI-Generated Code When You Are Not a Programmer instead — that guide owns the beginner product-test path.

A 2024 study found ChatGPT answered 52% of Stack Overflow questions incorrectly, delivering wrong answers with confident-sounding phrasing (source: Communications of the ACM). For generated code, the stakes are higher — a hallucinated call can compile, pass a happy-path test, and still corrupt data in production.

What does an AI code hallucination look like?

Hallucinations fall into recognisable patterns. Once you know what to look for, you start seeing them in almost every long agent session.

The five-point engineer verification checklist

Run every AI-generated block against these five checks before you commit. Prefer automated gates where you can; use prompts where you cannot.

1. Verify every import and dependency

The model is likelier to hallucinate a library name than you think. Before you run any agent output, check that every import resolves to a real package or built-in module.

# The AI generated this — does cachetools exist on PyPI?
from cachetools import LRUCache

# Quick check in your terminal:
# pip install cachetools

If an import does not exist on PyPI, npm, or your language’s package registry, the model most likely invented it. Replace it with a real alternative or implement the feature manually. In a harness, fail the step when npm ls / pip check / the language equivalent cannot resolve imports.

2. Cross-reference every API call against docs for your version

When the model writes something like response = client.query(embedding=vector, top_k=5), check the library documentation — not the model’s memory — for the actual parameter names and the version you have installed.

I have code that calls client.query() with a parameter called top_k. I am using the Pinecone Python SDK version 4.1. Check whether top_k is a valid parameter for the query() method. If it is not, tell me the correct parameter name and show the fix.

Provide your answer as a yes-or-no with the evidence from the docs.

A prompt like this forces the model to reveal the source of its knowledge. If it cannot cite a specific version of the docs, treat the API call as suspect and verify it yourself.

3. Run the code path, not just the happy path

AI models train on code that mostly works — they are optimised for the common case. This means they skip error handling, miss edge cases, and assume inputs are well-formed.

For each function the model writes, add a test for at least two of these scenarios. If the model did not handle them, you have caught a hallucination-by-omission — code that works under ideal conditions but fails everywhere else. Acceptance criteria that an agent can verify belong in How to Write Acceptance Criteria Your AI Agent Can Actually Verify.

4. Check for invented error classes and types

Models frequently invent exception classes that sound realistic. A try-except block that catches DatabaseTimeoutError is almost certainly hallucinated unless you recognise it from the specific driver you are using.

The fix is to check the library’s exception hierarchy. Most well-documented libraries list their custom exceptions in a single page. If the class is not there, change the catch to a standard exception or handle the failure mode differently. Type-checkers and compile steps catch many of these automatically — put them in the harness gate.

5. Trace the data flow end to end

The most subtle hallucinations happen in the middle layers of a pipeline. The model generates a transformation function that looks correct in isolation but silently drops rows, misaligns column indices, or assumes sorted data where none exists.

Trace the following data flow step by step and tell me where data could be lost or corrupted.

[Paste the relevant functions here]

For each risk you identify, suggest a guard (assertion, type check, or test) that would catch it during development.

This prompt treats the model as a code reviewer for its own output — a pattern that catches many hallucinations before they become bugs.

Wire the checklist into an agent harness

Manual review does not scale. Encode the checks as gates:

1. Install / resolve — fail if dependencies do not resolve.
2. Type-check / lint — fail on invented symbols the compiler can see.
3. Unit + edge tests — fail if empty/null/malformed paths are missing or red.
4. Self-review prompt — optional inferential check for logic the compiler cannot see.
5. Human approve — required for auth, payments, migrations, and anything touching production credentials.

That loop — propose, verify, correct, approve — is the core of Harness Engineering. Without it, every hallucination depends on a tired engineer noticing it in a diff.

When the model invents an API

Here is a concrete example. A developer asked an AI assistant to write a script that batches API requests. The model produced this:

import asyncio
from aiohttp import ClientSession, ClientTimeout

async def fetch_all(urls, max_concurrent=5):
    semaphore = asyncio.Semaphore(max_concurrent)
    timeout = ClientTimeout(total=30)

    async with ClientSession(timeout=timeout) as session:
        tasks = [fetch_one(session, url, semaphore) for url in urls]
        return await asyncio.gather(*tasks)

The code looks correct. It uses asyncio, aiohttp, a semaphore for concurrency control — everything a Python developer would expect. The hallucination is invisible: aiohttp’s ClientSession does not accept a timeout parameter at the session level in the version the developer had installed. The parameter belongs on individual requests.

The fix is small, but it illustrates the pattern: the model used the right vocabulary and idiom — except for one parameter that does not exist in the real library. Without the review checklist, this code fails with a cryptic TypeError.

Trust but verify — then automate

AI coding tools are transformative, but they are also probabilistic engines that optimise for plausible output, not correct output.

The five-point checklist — verify imports, cross-reference APIs, test edge cases, check error classes, trace data flow — turns review from vague anxiety into a repeatable process. Apply it to every generated block, then move the checks into your harness so humans only spend attention where risk is high.

For the full control-loop design, see Harness Engineering. For team-level delegation and supervision patterns, see Agentic Coding Pro. For session discipline that reduces hallucination volume before review, see Vibe Coding Pro.