I’m a Professional Fact-Checker. AI Is Wrong More Often Than You Think

Nearly fractional of Americans accidental they usage AI to find accusation and make ideas. It’s not hard to spot why. As societal media devolves into slop—and Google into a glorified landing leafage for Reddit threads and contented farms—most of america are starved for thing reliable. Plus, chatbots are truthful helpful, aren’t they? The archetypal clip I interacted with one, I asked if it knew it was a immense drain connected resources. Half an hr later, I had a caller look for vegan pick cheese.

I ne'er tried the recipe. Instead, I recovered a human-created 1 that the LLM mightiness person scraped. That’s the mode these models work, of course. They repackage corporate cognition into thing that feels tailored to you. This whitethorn beryllium OK for dairy alternatives (unless you’re a vegan blogger). But connected the bid of the world, and truth—the absorption of my relation arsenic a fact-checker astatine WIRED—the stakes are exponentially higher.

Over the past twelvemonth oregon so, much and much radical person looked astatine maine with large pity. Surely a fact-checker astatine a mag isn’t agelong for this AI-upgraded world. Call maine foolish, but I’m not that worried. Very small of humanity’s corporate knowledge, I’ve concluded, lives connected the internet. And according to my research, AI is adjacent much incorrect than radical mightiness think.

Tom Wolfe evidently thought of fact-checkers, according to the writer Colin Dickey, arsenic a “cabal of women and middling editors each collaborating to henpeck and emasculate the prose of the Great Writer.” As definitions go, it’s not atrocious (though my brag and galore colleagues are men). What tin I say? It’s our job, unlike AI’s, to beryllium annoying.

WIRED’s fact-checking section is old-school: meticulous line-by-line annotations, superior sources whenever possible, and a broader-scale ethical and ineligible review. We question basal assumptions, look for caller oregon conflicting information, telephone and speech to people—make sure. It’s a quick-hit adjacent review, functioning arsenic champion it tin astatine the aforesaid gait arsenic the quality itself.

As acold arsenic I tin tell, AI hasn’t travel for this process rather yet. What it has travel for is “post hoc” fact-checking, the Snopes-style investigation of something’s factuality aft the fact. In the UK, an inaugural called Full Fact has built retired its ain AI tools to assistance thwart the dispersed of misinformation. These tools, utilized successful much than 40 countries, process immense volumes of data, from societal media posts to podcast transcripts, past pinpoint circumstantial claims that humans tin analyse further. “You decidedly request a quality being,” says Mark Frankel, Full Fact’s caput of nationalist affairs.

The crushed for that is simple: AI inactive gets things wrong. As a fact-checker, I’d emotion to beryllium capable to archer you precisely however often. But it’s not truthful easy. Since 2018, astir 17,000 papers person been posted to arXiv connected LLMs, galore focused specifically connected the question of their reliability. Still, it’s worthy trying to pin down a moving figure.

In immoderate nonfiction that comes crossed WIRED’s fact-checking desk, there’s usually a decent magnitude of “b-matter”: statistics, quality events, quotes, thing that helps contextualize the topic. Fact-checkers thin to Google this basal information, and that process, successful the signifier of the hunt engine’s dreaded AI Overviews, constitutes my main enactment with AI. In my nonrecreational opinion, it’s unusable—wrong—about a 3rd of the time.

This mightiness beryllium a generous assessment, though. A March 2025 survey from the Tow Center for Digital Journalism recovered that much than 60 percent of responses from AI-powered hunt engines were inaccurate. A BBC survey puts the wrongness of chatbots person to 45 percent, the fig I spot cited much often. Because percentages are distancing, fto maine enactment this much plainly: AI could beryllium incorrect astir fractional the time.