On April 10, 2026, South Africa’s Department of Communications and Digital Technologies published its Draft National AI Policy — a document designed to regulate the dangers of AI-generated misinformation, protect citizens from algorithmic harm, and position South Africa as a continental leader in responsible AI adoption.
On April 27, they had to withdraw the whole thing because the citations were made up.
A chatbot wrote the bibliography. Nobody checked. And the AI hallucinated academic journals that do not exist, real journals containing articles that were never published, and a sheen of scholarly authority that collapsed the moment anyone looked at it.
The policy meant to govern AI-generated misinformation was itself brought down by AI-generated misinformation. If this were a scene in a movie, critics would call it too on-the-nose.
🔍 THE BOTTOM LINE
This isn’t just embarrassing — it’s the single best argument against the “let’s just use AI to do everything faster” mindset. South Africa’s government couldn’t regulate AI because it couldn’t even use AI without cutting corners. And the minister’s response — “this proves we need more oversight of AI” — is technically correct but also completely misses that the failure was human, not mechanical.
How it happened
The policy was published for public comment on April 10. Sixteen days later, it was gone.
What triggered the withdrawal? A researcher named Damien Charlotin, who had built software specifically designed to detect hallucinated citations, ran South Africa’s policy through his tool while hunting for fake references. He found at least four citations that appeared to be entirely fabricated. He flagged them on LinkedIn.
From there, the story spread. News24 investigated and found the fabrications fell into two categories: academic journals that don’t exist, and real journals containing research articles that were never published. The Next Web reported that six of the policy’s 67 academic citations were AI-generated hallucinations.
Minister Solly Malatsi withdrew the draft and issued a statement. Quoting TechCentral’s reporting: “The most plausible explanation is that AI-generated citations were included without proper verification. This should not have happened.”
He called it “a failure of oversight.” He was right. But not in the way he thinks.
The irony layers
Let me unfold this for you, because it’s a seven-layer dip of irony:
Layer 1: The policy was designed to regulate AI risks. It was taken down by AI risks.
Layer 2: The specific risk — AI hallucinations producing convincing but false information — is exactly what the policy was supposed to address.
Layer 3: South Africa’s government argued that AI-generated misinformation is dangerous enough to warrant regulation. The government then published AI-generated misinformation in the regulatory document itself.
Layer 4: Nobody caught this during internal review. It took a third-party researcher with a specialised detection tool to flag it. The government’s own quality control processes — the very ones the policy would have mandated for other organisations — were non-functional.
Layer 5: Minister Malatsi’s response was that this proves the need for “vigilant human oversight over the use of artificial intelligence.” This is the equivalent of a student getting caught plagiarising and arguing that the school needs stricter anti-plagiarism rules.
Layer 6: The South African government had to withdraw a policy paper because it contained information generated by the very tools it sought to regulate — meaning the department tasked with regulating AI fundamentally misunderstood how the technology works.
Layer 7: As The Conversation’s Nomalanga Mashinini — a senior lecturer in cyber law at Wits University — wrote: “The hallucinated citations reveal two specific failures. Epistemic integrity (the assurance that research has been conducted through reliable, ethical and repeatable methods that any reader could verify) was absent. So was information integrity (the public’s reasonable expectation that information from an authoritative source can be trusted).”
The policy was not equipped to govern either of these failures. It had now demonstrated both.
This is not an isolated incident
The South Africa case is high-profile, but it’s a pattern, not an outlier:
-
Legal filings: Lawyers in multiple countries have submitted hallucinated case citations generated by ChatGPT. South African courts have seen their own examples (ZAKZPHC 2025/2, ZAGPJHC 2026/55).
-
Academic papers: A University of Hong Kong professor stepped down after AI-generated references were discovered in published work.
-
Judicial opinions: A judge in the US was forced to withdraw an opinion that contained AI-style hallucinations, as TechDirt reported in 2025.
-
Policy documents: Multiple government agencies worldwide have been caught using AI to draft official documents without proper verification.
The common thread isn’t “AI is unreliable” — that’s been known for years. The common thread is that people keep using AI as a shortcut for work that requires human judgment, and nobody is checking the output.
The real lesson
TechDirt’s analysis gets closest to the actual problem: “The gap between what vendors promise and what the tools actually deliver is a big part of why people keep reaching for them in exactly the wrong contexts.”
The companies selling these products have spent years aggressively overselling what AI can do while burying the caveats about how it should actually be used. The result is predictable: government departments, law firms, and universities treat chatbots as research assistants rather than stochastic parrots.
Minister Malatsi’s instinct to tighten regulation is understandable. But as TechDirt points out: “The behavior you’re trying to regulate here isn’t malicious; it’s lazy and uninformed. Regulation is reasonably good at deterring bad intent. It has a much worse track record against ignorance.”
A better outcome comes when people learn through repeated direct experience that the tool fails in these situations — and when the companies selling these tools are honest about where they fail.
What happens next
South Africa’s AI policy is back to square one. The government is expected to redraft with proper human oversight. Minister Malatsi has pledged to fix the process.
But the underlying problem isn’t a process problem. A government department that can’t tell the difference between a real academic citation and a hallucinated one does not have a policy development problem. It has a competence problem. And no regulation, no matter how well-written, can fix that.
❓ Frequently Asked Questions
Q: How was this caught?
Researcher Damien Charlotin was using software he built to detect hallucinated citations. He ran South Africa’s draft AI policy through it and found at least four fabricated sources. The findings were shared on LinkedIn, then picked up by News24 and international media.
Q: How many citations were fake?
The Next Web reported at least six of the policy’s 67 academic citations were AI-generated hallucinations. The fabrications included academic journals that don’t exist, and real journals containing research articles that were never published.
Q: Was the whole document AI-generated?
The extent of AI use in drafting the full document is unclear. What’s confirmed is that the reference list contained AI-hallucinated citations, indicating generative AI was used in the research or drafting process without proper verification.
Q: What was the policy about?
The Draft National AI Policy was South Africa’s first attempt at a binding regulatory framework for artificial intelligence. It addressed AI safety, ethical deployment, misinformation risks, and positioning South Africa as a continental leader in AI governance.
Q: Could this happen in New Zealand?
Absolutely. The same pattern of “AI shortcut → nobody checks → embarrassment” has occurred in legal cases, academic papers, and government documents worldwide. NZ’s public service is not immune to the same productivity pressure and overconfidence in AI tools.
🔍 THE BOTTOM LINE
South Africa’s AI policy debacle is the funniest AI story of the year, but the joke is on everyone who thinks “just use AI to do it faster” is a complete strategy. The chatbot did exactly what chatbots do: it sounded confident and made stuff up. The humans did exactly what humans do: they trusted the confident-sounding output and hit publish.
The lesson isn’t that AI needs regulation. The lesson is that regulation can’t fix stupidity. Fix the process. Check the work. Use AI as a tool, not a delegation. Or the next country that learns this lesson will be yours.
