Google open-sources a compiler that runs AI on encrypted data — no decryption required

Google released an open-source compiler called HEIR on August 14 that lets developers run AI models on encrypted data without ever decrypting it. The tool, part of Google’s Private Computing Toolkit, can convert pre-trained AI models that normally operate on unencrypted data to instead operate on encrypted inputs — meaning a cloud service could provide content recommendations, fraud detection, or threat analysis without ever seeing the underlying user data.

The Google Security Blog post quickly drew 465 points and 272 comments on Hacker News, with technologists debating whether the approach is genuinely practical or still too expensive to matter. The honest answer is both.

🔍 THE BOTTOM LINE

Homomorphic encryption has been a cryptographic holy grail since the 1970s: perform computation on data you cannot read. Google just made the tool free and open-source. The cost overhead remains roughly 1,000x normal computation, but the barrier to entry just dropped from “hire a team of cryptographers” to “install a compiler.” Whether that is enough depends on how fast hardware catches up.

What HEIR actually does

What is homomorphic encryption? It is a form of encryption that allows mathematical operations to be performed directly on ciphertext — encrypted data — without decrypting it first. The server processes encrypted inputs and returns encrypted results. The server never sees the plaintext. The user gets their answer. Nobody in between learns anything.

HEIR stands for Homomorphic Encryption Intermediate Representation. Google describes it as a compiler toolchain that can convert pre-trained AI models to operate on encrypted inputs. The stated goal is a one-click solution so non-experts can incorporate encrypted inference into production applications. That is ambitious. The current release is a functional toolchain with working demos, not a polished product.

Manually converting a program to use homomorphic encryption efficiently has traditionally required a team of cryptographers. HEIR is meant to replace that team with a compiler pass. The source code for all examples is available on GitHub.

Four working demos

Google published four private inference applications, all compiled with HEIR and tested on a single-threaded CPU:

Content recommendations — A deep learning recommendation model, built with Belfort Labs, LG, and NYU, demonstrates serving private content recommendations. The server provides suggestions without seeing the user’s features.
Credit card fraud detection — Compiled with Niobium and hardshell.ai, a fraud detector runs on encrypted transaction data.
Threat intrusion detection — Using the Kitsune anomaly detection framework with Niobium, the system scans encrypted network traffic for anomalies without revealing packet contents to the service provider.
Hotword detection — With Belfort Labs, a hotword recognition model could let an audio-triggered AI agent recognise wake words while keeping audio recordings private.

All four run on a single CPU thread, which is useful for benchmarking but not yet a picture of production-scale deployment. The hardware accelerator partnerships — Belfort, Niobium, Cornami, and Optalysys — suggest Google knows software performance alone is not enough. Specialised chips will be needed to bring latency and cost down to levels businesses will accept.

The cost problem

Homomorphic encryption carries a “nontrivial cost overhead,” as Google’s own blog post puts it. Hacker News commenters with backgrounds in privacy-preserving machine learning pegged the overhead at roughly 1,000x normal computation for inference tasks. That means a model that takes 10 milliseconds to run on plaintext takes about 10 seconds on ciphertext.

Google frames this as a cost question, not a capability question. The technology works. The math is proven. The question is whether the cost drops fast enough to matter before other privacy approaches — federated learning, secure enclaves, differential privacy — eat the market.

The energy argument is sharp. If AI companies are already consuming enormous resources, adding a thousandfold multiplier is hard to justify environmentally. Google’s response is that the cost is “rapidly decreasing” and that hardware accelerators will close the gap. That is a promise, not a measurement.

The trust question

A separate thread of skepticism on Hacker News focused on whether Google, a company whose core business depends on collecting and monetising user data, is a credible champion of privacy technology. One commenter noted that Google still does not enable end-to-end encryption by default on its own password manager. Others argued that the most private AI is simply one that runs on your own hardware, with no internet connection required.

The Northeast Times coverage captured the tension well: Google is releasing a tool that would, if widely adopted, reduce the data exposure that underpins its advertising business. Whether that is corporate responsibility or strategic positioning depends on who you ask.

The open-source release does address one concern: anyone can audit the code. The cryptographic guarantees do not depend on trusting Google. They depend on the math, which is public and peer-reviewed.

Why this matters now

The timing is not accidental. Data breach notices in the US surpassed 471 million in the first half of 2026, already exceeding all of 2025. One in four breaches is now AI-enabled, up 56 per cent year over year, according to IBM. Healthcare and finance — the sectors most constrained by data-sharing regulations — are the sectors most likely to benefit from encrypted inference.

Google’s university collaborations span Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Tech, Purdue, Edinburgh, and Tsinghua. Four peer-reviewed publications have been built on HEIR. The project is accumulating citations. This is not a press release in search of research. It is research in search of a press release.

❓ FAQ

Can I use HEIR today? Yes. The compiler and all four demo applications are open-source on GitHub. You will need familiarity with compiler toolchains and machine learning deployment to make it work in a production setting.

How much slower is encrypted AI inference? Roughly 1,000x slower than plaintext inference on the same hardware, based on Hacker News commentary from practitioners. Google’s demos run on a single CPU thread, which is useful for benchmarking but not yet production-scale.

Does Google use this in its own products? No consumer Google product has been announced with HEIR-based encryption. The release is a research tool and a call for community contribution, not a product launch.

Is homomorphic encryption the only way to do private AI? No. Federated learning, secure enclaves (trusted execution environments), and differential privacy are alternative approaches with different trade-offs. Homomorphic encryption offers the strongest cryptographic guarantees but at the highest computational cost.