Fifteen privacy and consumer protection organisations — including the Electronic Frontier Foundation, Electronic Privacy Information Center, Demand Progress, and the National Consumers League — have urged the Federal Trade Commission to reject X’s bid to terminate a 2022 consent order requiring independent audits of the platform’s data handling. X argues it has rebranded, restructured, and now faces equivalent obligations under the EU’s GDPR. The advocates say Musk has made things worse, not better — and the order should stay.
🔍 THE BOTTOM LINE
The practice at the centre of this fight — training AI on user-generated content without meaningful consent — is not an X problem. It is an industry problem. Every major platform does it. X is simply the most transparent about it, and the most personally convenient target for the commentary class. The right answer is not “keep the order as-is” or “terminate it.” It is modify — keep targeted audits on the high-risk areas (AI training pipelines, cross-entity data flows with xAI and SpaceX, security incidents) while sunsetting the duplicative burdens. And then replace case-by-case personality-driven enforcement with industry-wide rules on training data transparency and consent. Because the next platform to industrialise user data for AI training will not hold a press conference. It will update its terms of service at 2am on a Friday and hope nobody reads them.
The Actual Regulatory Fight
The consent order dates to a pre-Musk era. The FTC found that a coding error in Twitter’s systems had caused users’ contact information — submitted for two-factor authentication — to be improperly shared for ad targeting. The order requires independent audits and gives the FTC authority to demand documents without filing a new lawsuit.
X’s legal argument is straightforward: the company has rebranded, it operates under GDPR in Europe, and the order imposes costs that are now duplicative. Former US Attorney General William Barr filed a supporting comment, arguing the FTC has made “hundreds” of information demands and that “permanent agency control of private companies” should not be the default.
The 15 advocacy groups co-signed a letter calling X’s petition “a brazen attempt to escape accountability” and listing a cascade of concerns: Grok’s global backlash, a lawsuit alleging the chatbot generated non-consensual intimate images, a 2.8 billion record data leak, DOGE’s access to sensitive government data, and X’s decision to train AI on user posts “without meaningful or explicit user consent.”
The FTC must decide whether X’s legal standard for termination is met — not whether Elon Musk is a nice person.
The Practice, Not the Person
Here is the part the coverage buries. The Cambridge Analytica comparison quoted in the Ars Technica report makes the point inadvertently:
“Other platforms perform equivalent extraction while maintaining ambiguity in their terms of service. X’s brazenness — Musk announcing Grok’s capabilities without pretense of user benefit — might paradoxically prevent effective regulation by making the surveillance mechanism obvious rather than hidden.”
Read that twice. Other platforms do the same thing. Meta trains AI on Instagram posts. Google trains on everything it touches. TikTok’s entire business model is population-level behavioural data. The difference is that Musk said it out loud, and the others bury it in terms-of-service updates nobody reads.
If the regulatory response to “transparent about extraction” is harsher scrutiny than “quiet about extraction,” the incentive structure is backwards. The platforms that hide the practice get left alone. The one that announces it gets a Senate hearing.
This is not a defence of Musk’s practices, which can be blunt and sometimes harsh. But the pattern of personalising systemic problems — turning every regulatory filing into a referendum on one man’s character — is how the actual issue disappears. The issue is that no major platform asks for consent before training AI on user data. That is true of X. It is true of Instagram. It is true of TikTok. It was true of Facebook before Cambridge Analytica made it a scandal, and it is still true now.
The Concentration Argument Has Merit
Where the Musk-focused coverage does hold weight is on concentration of power. This is genuinely different from a normal CEO running a platform:
- X is a major social media platform
- xAI/Grok trains on that platform’s data
- X was folded into SpaceX, a defence contractor
- Musk ran DOGE with access to sensitive government systems
- The FTC already found Musk “had directed employees to take actions that would have violated” the consent order
One person holding data across social, AI, defence, and government is a legitimate reason for heightened scrutiny — regardless of whether you like or dislike the person. The advocates make this argument, and it is the strongest part of their letter. The FTC should weigh it.
But the scrutiny should be about the structural risk, not the personality. The difference matters because the next billionaire to consolidate this much data power may be quieter than Musk — and the regulatory reflexes built around reacting to his tweets will not fire.
The Tension You Can’t Resolve
Here is the honest tension that most coverage refuses to sit with: AI progress requires massive data. Over-restriction favours closed models and incumbents who already hoarded their data empires before anyone started asking questions. Under-restriction erodes trust and invites the kind of backlash that sets the whole field back.
Both sides are right. The advocates are right that training on user data without consent is a real problem. X is right that AI needs data to improve, and that burdensome compliance costs fall hardest on newer entrants. The answer is not to pick a side — it is to hold the tension explicitly.
X and Grok’s approach — more open data plus public scrutiny — has genuine advantages over the opaque alternatives. Grok’s public square feature, where users can see and challenge AI outputs in real time, is more transparency than Instagram or TikTok offer. The BBC’s investigation into AI delusions showed Grok has real problems. But the principle of visible, challengeable AI is the one thing worth keeping — and it is the thing the commentary class least wants to credit, because crediting it complicates the narrative.
The concentration of power — one person holding data across social, AI, defence, and government — demands vigilance regardless of who is in charge. That vigilance does not require personalising every regulatory filing. It requires structural safeguards that work whether the billionaire is loud or quiet.
The NZ Angle
New Zealand users’ posts on X are being trained on too. The Privacy Act 2020 requires informed consent for collecting personal data, but the question of whether training AI on public social media posts constitutes “collection” under the Act remains untested. The Office of the Privacy Commissioner has not issued guidance on AI training data specifically.
This means NZ users are in the same position as American ones — subject to a foreign regulator’s decision about a platform that operates globally. If the FTC lifts the order, there is no equivalent oversight body stepping in. If the FTC keeps it, the protection extends to NZ users incidentally, not by design.
The broader pattern — sovereign AI infrastructure as the answer to data sovereignty — applies here. Without domestic regulatory capacity, NZ depends on whatever the FTC, the EU, or the platforms themselves decide is convenient.
The Other Side
X’s legal argument is not frivolous. Consent orders are not meant to be permanent. The company did rebrand. It does face GDPR obligations. The FTC’s information demands have been extensive. William Barr’s point about “permanent agency control” has a legitimate legal basis — consent orders that never end are a form of ongoing regulatory supervision that courts have historically viewed with scepticism.
The advocates’ strongest counter is timing: the order is four years old, Musk agreed to it when he bought Twitter, and the company is in the same business — social media with targeted advertising — plus a new AI business that makes the oversight more relevant, not less. That is a compelling argument. It does not require a personality referendum to land.
But the better path is the one neither side is arguing for: modify, not terminate. Keep targeted audits focused on the high-risk areas — AI training pipelines, cross-entity data flows between X, xAI, and SpaceX, and security incident response — while sunsetting the duplicative burdens that X reasonably complains about. This balances the reality that AI needs data with the reality that concentrated data power needs independent verification.
The even better path — and the one that would make the X order a footnote instead of a precedent — is industry-wide rules rather than case-by-case enforcement. Clear opt-in requirements for training AI on non-public user data. Labelling requirements for synthetic media so people can identify AI-generated content. Data provenance standards so users can trace what was used to train models that affect them. These are not radical ideas. They are the minimum framework for a market that currently operates on “we updated our terms, good luck finding the opt-out buried in paragraph 47.”
❓ FAQ
What is the FTC consent order? A 2022 order requiring X (then Twitter) to submit to independent privacy audits and give the FTC document access, after a coding error shared users’ two-factor authentication data for ad targeting.
Why does X want it lifted? X argues the company has rebranded and restructured, faces equivalent GDPR obligations in Europe, and the order’s costs are burdensome and duplicative.
Why do privacy groups want it kept? They cite Grok’s AI training on user posts without consent, a CSAM/NCII lawsuit, a 2.8 billion record data leak, and Musk’s DOGE data access as evidence the platform needs more oversight, not less.
Is this just a Musk problem? No. Every major platform trains AI on user data without explicit consent. X is the most transparent about it, which makes it the easiest target. The practice is industry-wide.
What happens if the FTC lifts the order? X loses a binding oversight mechanism. No other regulator is currently positioned to fill the gap — the GDPR investigation is ongoing but covers EU users only.
What’s the better option than keep-or-terminate? Modify the order — keep targeted audits on AI training pipelines, cross-entity data flows (X/xAI/SpaceX), and security incidents. Sunset the duplicative burdens. Then push for industry-wide rules: opt-in for non-public training data, synthetic media labelling, and data provenance standards.
🔍 THE BOTTOM LINE
The FTC should modify the order, not terminate it. Keep targeted audits on the areas that genuinely warrant scrutiny — AI training pipelines, cross-entity data flows between X, xAI, and SpaceX, and security incident response. Sunset the duplicative burdens. Then do something neither X nor the advocates are asking for: write industry-wide rules. Opt-in for non-public training data. Synthetic media labelling. Data provenance standards. Rules that apply to Meta, Google, and TikTok with the same force they apply to X.
The personality is the sideshow. The practice is the point. And the practice is everywhere.
If the regulatory system can only mobilise when a billionaire makes it personal, it will miss every case where the billionaire is quiet. Over-restriction starves open AI development and entrenches incumbents who already hoarded their data. Under-restriction erodes the trust that makes the whole ecosystem function. The honest answer lives in that tension — and pretending it doesn’t is how you get policy that satisfies nobody and protects no one.
