A problem when using AI for research – 404 links
Posted By RichC on August 21, 2025
Flaw in AI assisted research (404 links). Artificial Intelligence generated segment below for a recent blog post — and all are broken/non-existent links:
Record-low unemployment rates for most demographics, including native-born workers
Historic median household income growth, reaching an all-time high of $68,703 in 2019
4.9 million Americans lifted out of poverty in a single year (2019)—the largest one-year drop on record
From Grok:
The issue of AI-generated content, including links that lead to 404 errors (non-existent pages), is a known challenge in AI-assisted research, as highlighted by your example where some provided links may be broken or fabricated. This stems from AI models like large language models (LLMs) occasionally “hallucinating” URLs—generating plausible but incorrect or outdated links due to limitations in their training data or real-time web access. Below, I address your concern about preventing 404 links and ensuring AI checks the validity of links provided, with specific reference to your example.
Why AI Generates 404 Links
AI models, including Grok 3, may produce 404 errors for several reasons:
- Hallucination: AI can fabricate URLs that seem legitimate but don’t exist, especially if it lacks real-time web access to verify them. For instance, a Tow Center study found Grok 3 provided 404 error pages in 154 out of 200 tests, indicating a high rate of fabricated links.
- Outdated Data: Links may point to pages that existed in the AI’s training data but have since been removed or relocated, as websites update or archive content.
- Incorrect Attribution: AI might misattribute sources, linking to incorrect or unrelated pages, as seen in your example where the Heritage Foundation link may not directly support the claim or could be broken.
- Limited Web Access: Unlike search engines, many AI models don’t perform live web searches unless explicitly designed to (e.g., Grok 3’s DeepSearch mode), leading to reliance on potentially stale data.
In your example:
- The first link appears valid, as it points to a Bureau of Labor Statistics page, though I’d need to verify its current status.
- The second link is suspicious, as the title suggests a decline in poverty, but the URL implies a decline in income, which may indicate a hallucinated or incorrect link.
- The third link may exist but could be biased or not directly tied to the specific claim of poverty reduction, and its validity needs checking.
How to Prevent or Mitigate 404 Links
I’ll ever say “intensive” again without remembering this post? 
