agentage.hu / Articles / Guide

Chain-of-Verification: Adding a Checking Step to AI News Drafts

Separate the drafting phase from the verification phase to catch hallucinations before an editor ever sees the text.

By Zoltán Varga2026-11-056 min read

Why drafting and checking must be separate steps

Drafting and verifying are distinct cognitive tasks that require different system configurations. When an AI generates text and checks its own facts simultaneously, it often prioritizes fluency over accuracy. The model fills gaps with plausible-sounding details because it lacks the space to pause and cross-reference external data. Separating these steps forces the system to stop generating narrative flow and switch to a retrieval mode.

Research indicates that large language models frequently produce hallucinations when asked to write complex articles without intermediate checks. These errors range from minor factual slips to entirely fabricated events. A single-pass generation model treats the prompt as a creative writing exercise rather than a data synthesis task. Editors reviewing such output spend excessive time correcting basic facts instead of refining the story angle.

Implementing a verification pass creates a mandatory checkpoint in the production pipeline. The drafting agent produces a rough version containing all potential claims. A second agent then analyzes this text specifically for factual assertions. This architecture mirrors human editorial workflows where a reporter files a draft and a fact-checker verifies the data. The separation ensures that the final output reaches the human editor only after automated validation.

The psychological polarization of agent engineering decisions means teams often choose between speed and safety. A unified system favors speed but sacrifices reliability. A separated system introduces latency but significantly improves trustworthiness. For a newsroom, the cost of a single public error outweighs the delay of an extra processing step. The workflow must prioritize accuracy over raw generation speed.

Turning a draft into checkable claims

The first technical hurdle is breaking a continuous text into discrete, testable units. An AI draft contains sentences that blend opinion, context, and hard facts. The verification agent must identify which parts of the text represent verifiable claims. It ignores stylistic flourishes and focuses on statements that assert specific entities, dates, or relationships. This decomposition turns a paragraph into a list of atomic propositions.

This process relies on the concept of atomic factual precision. Each claim must stand alone as a true or false statement without needing surrounding context to make sense. For example, a sentence like "The company launched in 2023 after a merger" splits into two claims: one about the launch date and one about the merger. This granularity allows the system to reject a draft if even one specific detail is unsupported.

Without this decomposition, the verification agent struggles to isolate errors. A long paragraph might contain nine correct facts and one lie. If the agent evaluates the whole block, it might pass the entire text as acceptable. By isolating the single false claim, the system flags the specific sentence for revision. This precision prevents the propagation of errors through the rest of the article.

The transformation from draft to claim list requires clear instructions to the AI. The system must be told to ignore narrative flow and focus strictly on factual assertions. It extracts the subject, the predicate, and the object of each statement. This structured output serves as the input for the next stage of the pipeline. The quality of the final article depends entirely on the accuracy of this extraction step.

• Identify every sentence that asserts a specific fact about an entity or event.
• Split compound sentences into separate atomic claims for independent testing.
• Remove subjective opinions and rhetorical questions from the verification list.
• Ensure each claim is self-contained and does not rely on previous context.

Matching claims to stored source documents

Once the claims are isolated, the system must retrieve evidence to support or refute them. This step connects the draft to the organization's internal knowledge base or approved external sources. The verification agent queries the database using the specific entities found in the claims. It looks for documents that explicitly mention the relationship between those entities. The goal is to find a direct match between the claim and the source text.

Grounding rates in current research show that only a fraction of claims in AI-generated text are naturally supported by existing data. Many claims rely on the model's training data rather than specific, up-to-date documents. A robust verification pass forces the agent to find a citation for every claim. If no source document exists, the claim is marked as unverified. This prevents the system from inventing details to fill gaps in the narrative.

The matching process must handle variations in phrasing and terminology. The claim might say "acquired," while the source document says "bought." The system needs semantic understanding to recognize these as equivalent. It compares the meaning of the claim against the content of the retrieved documents. If the source confirms the relationship, the claim receives a positive score. If the source contradicts the claim, the system flags it immediately.

This stage also reveals the limitations of the available data. Sometimes the source documents are incomplete or outdated. The verification agent reports these gaps to the human editor. This transparency is crucial for maintaining editorial integrity. The editor knows exactly which facts are backed by data and which ones require manual investigation. The AI acts as a filter, surfacing only the claims that need human attention.

Scoring factual precision

After matching claims to sources, the system assigns a score to the draft based on factual precision. This metric calculates the ratio of verified claims to total claims. A high score indicates that most statements are grounded in reliable data. A low score suggests the draft contains significant hallucinations or unverified assertions. This numerical value provides an objective measure of the draft's quality before human review.

Metrics like FActScore demonstrate the value of evaluating atomic units rather than overall coherence. A text can be grammatically perfect and logically consistent while still being factually wrong. The scoring system ignores style and focuses solely on truthfulness. It penalizes the draft for every claim that lacks a supporting citation. This approach shifts the priority from writing ability to data accuracy.

The threshold for acceptance varies depending on the publication's standards. A breaking news alert might require a lower score than a long-form investigative piece. The system can be configured to automatically reject drafts that fall below a certain precision level. This automation saves editors from wasting time on low-quality outputs. It ensures that only drafts with a high probability of accuracy reach the human desk.

Continuous monitoring of these scores helps refine the underlying models. If a specific type of claim consistently fails verification, the system can adjust its generation parameters. This feedback loop improves the performance of the drafting agent over time. The newsroom gains insight into where the AI struggles most. This data informs training efforts and helps build more reliable systems for future use.

Limits: what verification passes cannot catch

Verification passes are powerful but not infallible. They rely on the quality and completeness of the stored source documents. If the database lacks information on a specific event, the system cannot verify the claim. It may mark the claim as unverified, but it cannot confirm if the claim is false. This limitation requires human oversight for topics outside the known corpus.

The system also struggles with claims that involve interpretation or nuance. Facts like "the market reacted positively" depend on subjective judgment. A verification agent can find data on stock prices, but it cannot determine if the reaction was truly positive. These interpretive claims fall outside the scope of automated fact-checking. Editors must apply their expertise to evaluate these sections manually.

Another risk involves adversarial attacks or subtle manipulations of the source data. If the source documents themselves contain errors, the verification agent will validate the incorrect claim. The system assumes the source is ground truth. It does not independently verify the credibility of the source material. This dependency means the newsroom must maintain high standards for its data ingestion processes.

Finally, the verification process adds complexity and latency to the workflow. It requires additional computational resources and time to run the checks. In fast-breaking news scenarios, this delay might be unacceptable. The newsroom must balance the need for speed with the need for accuracy. Sometimes, a partial verification or a disclaimer is the only viable option. The system must adapt to the urgency of the situation.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.