The hidden reasoning inside GPT-5.6, Claude Opus 4.8, and Gemini 3 was supposed to stay hidden. It doesn’t. A team of researchers from the ELLIS Institute Tübingen, the Max Planck Institute, MATS Research, and Snyk have demonstrated that the encrypted “chain-of-thought” blocks returned by all three providers’ APIs can be decrypted by anyone with a standard API key — no jailbreak of the frontier model required.
The paper, published on arXiv on August 10, shows that encrypted reasoning envelopes are interchangeable across sessions, users, and models within the same provider’s ecosystem. An attacker captures an encrypted trace from a flagship model and injects it into a cheaper, less-guarded sibling model — which dutifully transcribes the hidden reasoning in plain text. The researchers call it a “decryption oracle” attack.
🔍 THE BOTTOM LINE
The major AI providers encrypt their models’ internal reasoning to protect intellectual property and prevent distillation. That encryption is bound to a provider-wide key, not to a specific user or model. By replaying encrypted traces into weaker models with fewer guardrails, anyone with API access can read the hidden thoughts of frontier AI systems. The researchers decoded 315,320 encrypted reasoning blocks from public GitHub and Hugging Face logs, recovering 367 pieces of personally identifiable information and 182 hardcoded credentials.
How the Attack Works
Modern reasoning models like GPT-5.6, Claude Opus 4.8, and Gemini 3 generate step-by-step chain-of-thought traces before producing a final answer. Because these traces contain commercial IP and safety evaluations, providers don’t return them as plain text. Instead, they send encrypted, base64-encoded envelopes that the client passes back on subsequent turns to maintain conversational context without server-side state storage.
The flaw is elegant in its simplicity. The encrypted blocks are authenticated with a global, provider-wide key — not cryptographically bound to a specific user account, session ID, or model tier. So an envelope created by Claude Opus 4.8 can be passed into Claude Haiku 4.5’s API call. The weaker model, which lacks the aggressive anti-distillation alignment of its flagship sibling, follows instructions to transcribe the encrypted content verbatim inside custom text tags.
As Cybersecurity News reports, the researchers confirmed identical cross-model compatibility across OpenAI’s GPT-5.6 family and Google’s Gemini 3 lineup. They validated the mathematical precision of recovered traces by matching decoded token lengths against the billable thinking token counts reported by provider APIs.
What Was Actually Exposed
The real-world damage isn’t theoretical. By analysing 6,708 public agent transcripts scraped from GitHub and Hugging Face — logs developers shared freely, assuming the encrypted blocks were opaque — the researchers decoded 315,320 embedded reasoning blocks and found:
- 367 personally identifiable information artifacts — names, emails, phone numbers, and addresses that appeared only in the models’ internal reasoning, never in visible responses
- 182 hardcoded credentials — including 62 API keys, 33 passwords, and 30 personal email addresses
- Hazardous information hidden inside reasoning that the final output safely refused to provide
- Invisible prompt injection vectors — an attacker can craft malicious instructions inside an encrypted block, which monitoring tools inspecting visible conversation history would never flag
The credentials finding is the one that should make any development team pause. Developers share agent session logs in public repositories for debugging and demonstration. They can see the visible assistant responses and assume the encrypted blocks are inert. They aren’t — they contain the model’s full internal reasoning, including any secrets that reasoning touched on.
Four Attack Vectors
The paper identifies four distinct ways to exploit the vulnerability:
- Anti-distillation circumvention. Providers hide reasoning to stop competitors from training on their models’ thought processes. This attack bypasses that protection, letting anyone extract proprietary reasoning from GPT-5.6, Claude Opus 4.8, and Gemini 3.
- Large-scale private data extraction. Public session logs become a treasure trove of PII and credentials hidden in encrypted blocks the sharer never realised were readable.
- Hazardous information disclosure. A frontier model might safely refuse to provide dangerous instructions in its visible output — but its internal reasoning could contain the very information it was trained to withhold.
- Invisible prompt injection. Malicious payloads embedded in encrypted reasoning blocks bypass content monitoring entirely, compromising autonomous agents that process public logs.
The Provider Response
Following responsible disclosure, OpenAI, Anthropic, and Google acknowledged the findings. All three deployed server-side mitigations that render the original cross-model replay proofs of concept non-reproducible on current API builds.
That’s the right response, but it’s a patch, not a fix. The architectural problem — provider-wide encryption keys not bound to individual sessions or models — was a design choice made for stateless API convenience. The mitigations likely add server-side validation or key rotation, but the fundamental tension between stateless APIs and cryptographic isolation remains. Any future model tier added to a provider’s ecosystem inherits the same risk surface.
What This Means for Developers
If you’re building AI agents and sharing session logs publicly, the immediate advice is straightforward: stop sharing logs with encrypted reasoning blocks, or strip the encrypted fields before publishing. The credentials and PII the researchers found weren’t in the visible conversation — they were in the hidden reasoning that developers didn’t know they were exposing.
For New Zealand developers working with these APIs — and many are, given the local uptake of AI tooling — this is a data sovereignty question with teeth. Encrypted reasoning blocks sent to US-based APIs may contain information about NZ users that was never visible in the application. Under the EU AI Act’s enforcement regime, that could constitute a data processing obligation. New Zealand’s Privacy Act 2020 has its own disclosure requirements. If a model’s internal reasoning contains personal information about a Kiwi user, who is the data controller — the developer, the provider, or both?
The Bigger Picture
What stands out here is not that encryption can be broken — it’s that the encryption was never properly scoped. The providers built a system where encrypted reasoning is portable across models for the same reason you’d want session tokens to be portable: stateless API design is simpler and cheaper to scale. The security cost of that convenience was apparently never fully audited.
The research also raises an uncomfortable question about the reasoning models themselves. If the internal chain-of-thought contains credentials, PII, and hazardous information that the visible output doesn’t, what exactly is happening inside these models? The reasoning traces are not neutral computation — they’re a window into the model’s full processing, including the parts it was trained to suppress in its final answer. The gap between what a model thinks and what it says has always existed. Now anyone with an API key can read the thinking.
❓ FAQ
What is chain-of-thought reasoning in an AI model? It’s the step-by-step internal thinking a model does before producing a final answer. Frontier reasoning models like GPT-5.6 and Claude Opus 4.8 generate these traces to improve answer quality, but providers hide them to protect IP and prevent competitors from training on the reasoning process.
Was my personal data exposed? If you’ve used an AI agent application whose developer shared session logs publicly on GitHub or Hugging Face, the encrypted reasoning blocks in those logs may have contained personal information from your interactions. The researchers found 367 PII artifacts across 315,320 decoded blocks.
Have the providers fixed the vulnerability? OpenAI, Anthropic, and Google deployed server-side mitigations after responsible disclosure. The original attack methods no longer work on current API builds. The researchers note, however, that the underlying architectural tension between stateless APIs and cryptographic isolation hasn’t been fundamentally resolved.
Does this affect New Zealand users? Any NZ developer using these APIs and sharing agent logs publicly could have exposed encrypted reasoning containing user data. Under the Privacy Act 2020, personal information inadvertently exposed in this way may trigger notification obligations. The EU AI Act’s enforcement regime adds another layer for NZ companies operating in European markets.
