South Africa set out to lead the continent into an AI-powered future. Instead, it handed the world a cautionary tale about trusting machines with high-stakes decisions. The country’s draft national AI policy has now been withdrawn. The reason: fabricated academic citations embedded in the document exposed a serious credibility problem at the heart of its digital ambitions.
AI Policy pulled after fake references surface
Communications Minister Solly Malatsi ordered the withdrawal after a review flagged deep flaws in the document’s sourcing. Investigators found that at least six of the AI policy’s 67 academic citations were invented. The journals referenced were real. The articles were not.
“The most plausible explanation is that AI-generated citations were included without proper verification. This should not have happened,” Malatsi said.
He did not stop there.
“This failure is not a mere technical issue but has compromised the integrity and credibility of the draft policy,” he wrote on X.
The AI policy draft had already entered public comment when the problems came to light. Its goals were ambitious: position South Africa as a leader in AI development while tackling the ethical, economic, and social questions surrounding emerging technologies.
An ambitious framework is now on hold
Before officials pulled the document, it outlined significant structural changes. The AI policy proposed establishing a national AI commission, an ethics board, and a dedicated regulatory authority to govern how AI gets developed and deployed across sectors.
On the economic side, it included tax breaks, grants, and subsidies designed to pull private investment into AI infrastructure and forge stronger ties between government and industry.
All of that is now on pause. Authorities say they will revise the document and reissue it for public consultation, though no timeline has been set.
How the problem came to light
Local news outlet News24 first spotted the discrepancies. Editors from well-regarded journals — including the South African Journal of Philosophy, AI & Society, and the Journal of Ethics and Social Philosophy — confirmed independently that the cited papers did not exist.
The revelation raised urgent questions. How did a government policy document pass internal review with fabricated sources intact? Who verified the research before publication?
Malatsi said consequences would follow. He pledged to examine how the failure occurred and hold accountable those responsible for drafting the AI policy.
“This unacceptable lapse proves why vigilant human oversight over the use of artificial intelligence is critical. It’s a lesson we take with humility,” he wrote.
The hallucination problem is growing
This incident does not exist in isolation. It reflects a widening concern across academia, government, and business about how generative AI systems handle gaps in their knowledge.
When an AI model encounters unfamiliar or poorly documented territory, it sometimes generates confident-sounding but completely fabricated information. Researchers call this “hallucination.” The model predicts what a citation or fact should look like based on patterns in its training data — and produces something plausible but wrong.
Systems like ChatGPT and Gemini are built to predict language sequences, not to fact-check outputs. That distinction matters enormously when they are used to draft policy or research.
A 2025 study published in Nature put numbers to the problem. More than 2.5 percent of academic papers published that year contained at least one potentially hallucinated citation. In 2024, that figure stood at just 0.3 percent. In raw terms, over 110,000 papers published in 2025 may carry invalid references.
Trust becomes the central issue
South Africa’s misstep arrives as governments worldwide scramble to define their AI strategies. Nations want speed. They want competitiveness. But this episode shows what happens when verification gets skipped in the rush.
Treating AI-generated content as authoritative without validation is not just risky — it is costly. It damages institutional credibility and slows down the very progress these policies aim to achieve.
Policymakers now carry a dual responsibility. They must harness AI to remain relevant and competitive. At the same time, they must build the checks, processes, and accountability structures that prevent failures like this one from repeating.
For South Africa, the immediate task is to rebuild trust. The revised AI policy will likely demand stricter sourcing standards and explicit rules governing AI use in official government work.
What does this mean beyond South Africa?
No government is immune to this risk. Any institution that uses AI-assisted tools for drafting, research, or analysis faces the same exposure if human review is weak or absent.
This incident may be embarrassing for South Africa. But it also offers a clear lesson for every country drafting AI governance frameworks right now: AI can accelerate the work, but it cannot replace the judgment, rigor, and accountability that governance demands.
The draft will return. When it does, the credibility of South Africa’s AI vision will depend on how seriously officials take that lesson.
What do you think — should governments place strict limits on AI use in policymaking, or is tighter verification the better answer? Please share your views in the comments and join the conversation.