On-Device AI vs Cloud AI: Which One Actually Protects Your Encrypted Messages?

On-device AI is more private than cloud AI, but only under conditions most products don’t meet. When inference runs on your device, inside the encryption boundary, the plaintext of your messages never leaves hardware that was already permitted to read it. When inference runs in the cloud, even in a “private” cloud, the message must be decrypted elsewhere, which adds a third participant to a conversation designed for two. The distinction that matters is not cloud versus device. It is whether plaintext crosses the encryption boundary at all, and whether the vendor can prove it doesn’t.

‍

We have spent several years on this blog arguing that AI poses a risk to secure communication, that consumer messaging apps become national security liabilities once models are pointed at them, that inference into private conversations creates a surveillance surface, and that “AI-powered” and “confidential” are usually in tension. We still think all of that is true.

This article explains why we are nonetheless shipping AI features and what had to be true architecturally before we were willing to do so.

Where this picks up

Our previous piece, E2EE vs AI: Can Encrypted Messaging and AI Assistants Coexist?, set out the three architectures now in production: plain cloud inference, confidential cloud inference inside a Trusted Execution Environment, and fully on-device inference, and argued that the question worth asking a vendor is not whether a platform is encrypted but where the model runs. If you haven’t read it, start there; this article assumes the framing.

What that piece did not do is stress-test it. That is this article’s job. Specifically: does confidential computing actually deliver what it claims, what does the published evidence say when researchers attack it, and what are the honest costs of the on-device alternative we are recommending?

‍Two questions, not one

Before any of that, a distinction the industry routinely collapses. And that we have collapsed ourselves in earlier writing.

“Cloud versus device” is not one question. It is two, and they are independent:

- Who controls the infrastructure? This is the sovereignty question. It is answered by jurisdiction, key custody, and contract.

- Does plaintext leave the device at all? This is the end-to-end encryption question. It is answered by architecture, and only by architecture.

Self-hosting answers the first question. It does not answer the second.

If your ministry’s own server decrypts a conversation to summarise it, that conversation is no longer end-to-end encrypted between the two people having it. Your server is now a third endpoint. It is a far better third endpoint than a vendor’s: your jurisdiction, your keys, your logging policy, no foreign preservation order, but the guarantee has still changed. You have traded “nobody can read this” for “only my organization can read this.”

For a great many workloads that trade is correct, and it is exactly what on-premise and sovereign-node deployments are for. For a classified exchange between two officers, it is not.

Which gives three cases worth drawing.

Case 1 — a third-party vendor’s model

This is the default configuration of almost every AI messaging feature shipping today. The plaintext leaves your device, is decrypted on infrastructure you neither own nor audit, and may be retained under a policy you do not set.

‍

Case 2 — your own model, on infrastructure you own

Materially better, and the right answer for anything a handset cannot carry. Sovereignty is genuinely solved: the operator is you. The end-to-end property between the two participants does not hold.

‍

Case 3 — inference on the device

The plaintext is decrypted only where it already legitimately exists, and never transmitted. No new party gains the ability to read the message, because there is no new party.

‍

Self-hosting solves the sovereignty problem. On-device solves both. That is the sharper version of this article’s thesis, and it is the one we will defend for the rest of it.

One option worth dispatching before it comes up: why not run inference directly on encrypted data using fully homomorphic encryption, and get cloud capability with no trust at all? Because it doesn’t work at this scale yet, the cryptographer Matthew Green assesses that the best available schemes handle only very small models, the kind you could already run on a weak client device. FHE is the right answer eventually, not one you can ship in 2026.

The cryptographers got there first.

The argument that cloud inference breaks end-to-end encryption is not a vendor talking point. It is the published position of people with no product to sell.

A December 2024 paper from researchers at NYU and Cornell — How to Think About End-to-End Encryption and AI — argues that integrating AI assistants forces us either to redefine what an “endpoint” is, or to admit the encryption boundary has been extended. Three of its recommendations matter here:

- Training is off the table. Using end-to-end encrypted content to train shared AI models is flatly incompatible with E2EE. Not a grey area, not a consent problem — incompatible.

- Processing can work, conditionally. Running AI features over E2EE content is compatible with the guarantee only if you “prioritize endpoint-local processing where possible,” and only if no third party can see or use the content at any point.

- Off by default. AI features in an encrypted system should ship disabled, enabled only by explicit opt-in rather than by a product decision made on the user’s behalf.

That middle condition is the one the industry has quietly skipped. On-device inference is not a marketing preference. It is the first item on a peer-reviewed list of what it would take for AI and E2EE to coexist at all.

Signal drew the sharpest line. President Meredith Whittaker has described agentic AI as “putting your brain in a jar,” arguing that an assistant useful enough to act on your behalf needs something close to root access across every application database, usually in the clear, and will almost certainly process that data in the cloud. She concludes that granting such access would undermine Signal’s ability to provide privacy at the application layer at all. Signal has shipped no generative AI features.

That is a coherent answer to this problem. It is just not the only one.

What the audits found

Worth keeping in proportion: the two architectures below are the good case — what serious engineering looks like when a vendor takes the problem seriously. The default is still plain cloud inference, where failures are mundane rather than exotic. Meta AI users discovered that their prompts were public due to a misunderstood share feature. Nothing was broken. The architecture simply had no way to stop it.

Two vendors have done real work here:

- Apple's Private Cloud Compute (June 2024) is the most transparent cloud-AI security architecture anyone has built. One of its five commitments is enforced by cryptography rather than by promise: an Apple device will only send data to a node that can cryptographically attest it is running publicly listed software. Apple published build images, released source code, built a research environment for outside scrutiny, and extended its Security Bounty to cover the system.

- WhatsApp's Private Processing is TEE-based, stateless and opt-in, using Oblivious HTTP and anonymous credentials so Meta cannot link a request to an account. Meta also published the most useful sentence in the genre — a warning about its own architecture: "vulnerability researchers have repeatedly demonstrated the ability to bypass TEE guarantees."

If you must do cloud inference, this is how to do it. Which makes what the auditors found more instructive, not less.

Trail of Bits audited Private Processing before launch and reported 28 issues, eight of high severity. The most telling one: the system loaded configuration files containing environment variables after the attestation measurement was taken. A malicious insider at Meta could have injected an LD_PRELOAD variable to load arbitrary code into the enclave, and the attestation would still have verified as valid. Meta fixed the findings, added per-session nonces against attestation replay, and published the reports. The auditors also noted that fully reproducible builds of the enclave images remain difficult to achieve, limiting independent verification. They summarized that "TEEs aren't a silver bullet." The deeper lesson is in that first finding: an attestation only covers what it measures, and everything loaded afterward sits outside the guarantee.

‍

‍

The trust surface that remains

TEE guarantees also keep failing at the hardware layer. In October 2025, researchers at Georgia Tech and Purdue built a DDR5 memory-bus interposer for under $1,000, and published what it broke:

- Intel SGX, Intel TDX and AMD SEV-SNP — including SEV-SNP with Ciphertext Hiding enabled

- Private ECDSA signing keys, extracted from a constant-time OpenSSL implementation running inside an SEV-SNP virtual machine

- A Provisioning Certification Key, pulled from a fully updated Xeon server in trusted status

- Attestation itself, forged — and because confidential VMs anchor NVIDIA's confidential GPU computing, GPU attestation with it.

One honest qualification: the attack needs physical access to the server. That rules out remote attackers, but not the cloud operator, the data-center technician, or anyone able to compel their cooperation. Which is to say, it applies precisely to the threat model that confidential computing exists to address.

It also lands close to Apple's newest deployment. 

In June 2026, Apple extended Private Cloud Compute onto Google Cloud infrastructure running NVIDIA confidential computing GPUs and Intel TDX, rooting attestation in at least two independent vendor trust sources. That dual-root design is real mitigation, and more than most vendors would build. But the trust chain is now longer than it was on Apple silicon, not shorter.

The conclusion is not that TEEs are worthless. It is narrower:

Confidential cloud inference reduces the trust surface. On-device inference eliminates the provider.

Matthew Green puts the principle better than we can. For decades, he writes, the point of cryptography has been to remove trust, "to replace 'I promise not to look' with 'I can't.'" He thinks confidential computing probably works against the adversary it was designed to protect against. His verdict still follows from the principle: Private Cloud Compute is a far weaker guarantee than encryption, because its security rests on a great deal of hardware and software being correct rather than on the mathematics of a cipher.

‍

The four-way threat model

Now that self-hosting has its own column, the table stops being a two-sided argument and starts being useful.

‍

Two rows deserve expanding, plus one thing the table cannot show.

Who can read content. “Nobody” in the last column is doing real work, and “your own admins” in the third is not a euphemism. Note also that plaintext is not the only thing worth protecting: embeddings are not anonymization. Research presented at EMNLP 2023 showed that a method called Vec2Text can “recover 92% of 32-token text inputs exactly” from their embedding vectors, and recovered 89% of full names from a corpus of embedded clinical notes. The authors concluded that embeddings should be treated as highly sensitive private data and protected exactly as raw text would be. If your architecture ships embeddings anywhere, even to your own vector store,  you have shipped the text.

Subpoena and retention. In May 2025, a US magistrate judge ordered OpenAI to preserve and segregate all output log data that would otherwise have been deleted, regardless of whether deletion had been requested by the user or mandated by privacy regulation. The order reached more than 400 million users. By November, the court had ordered that 20 million de-identified conversations be produced to the news plaintiffs; a district judge affirmed in full in January 2026. Note what that overrode: not merely a company policy, but a user’s explicit deletion request and a regulator’s retention limit. Retention policies are promises. Court orders are not optional.

There is also a time dimension most threat models miss. Under-harvest now, decrypt later: post-quantum cryptography protects the transport but does nothing for a plaintext copy sitting in a provider’s logs, which requires no cryptanalysis at all. Data that never leaves has no harvest window.

What on-device does not fix?

Moving inference on-device does not abolish risk. Your device becomes the entire trusted computing base, and device-resident malware, OS-level compromise, or targeted spyware defeats on-device AI exactly as thoroughly as it defeats end-to-end encryption itself.

The best illustration is Microsoft Recall. Recall runs entirely locally, no cloud, no provider, no transfer. By the logic of this article’s first half,f it should have been a privacy win. Instead, the initial release stored its index without adequate protection, and Microsoft pulled the feature to rebuild it. The relaunched version added encryption at rest, Windows Hello gating and opt-in defaults, with Microsoft stating the new architecture would restrict malware “trying to ‘ride along’ with a user authentication to steal data.” In April 2026, the researcher who broke the first version, Alexander Hagenah, published a proof of concept showing it still could: malware running as a standard user, no admin rights, no kernel exploit. Microsoft disputes that this is a security flaw.

The lesson isn’t that local AI is bad. Recall’s local-first architecture is still more private than shipping the same screenshots to a vendor. The lesson is that “on-device” is a necessary condition, not a complete one. For the claim to hold, all of the following must be true:

- No cloud fallback. Not for hard queries, long documents, or low-memory devices.

- No content telemetry. Crash logs, quality pipelines, ES, and analytics must never carry message content.

- Encrypted local storage. Model outputs, caches and indexes inherit the message database’s protection.

- A verifiable update channel. Update requests must not leak which features a user relies on.

- Honest scoping. State plainly that a compromised device defeats the guarantee.

These are auditable engineering commitments. A vendor who won’t put them in writing hasn’t actually made the claim.

The third condition is the one the industry most often fails to meet, and it is worth showing what failure looks like. In May 2026, researchers at Mysk disclosed that WhatsApp stores its entire chat history in a plaintext SQLite database within a shared app container, one readable by any app from the same developer group, which, in Meta’s case, means Facebook and Instagram. Not a bug. An architectural decision, in place for years, affecting billions of users. We wrote about it at the time.

Note what that means for on-device AI. A local model summarizing messages on that architecture would be reading from and writing to storage that another company can already access. “It runs on your phone” would be true and worthless. Local inference is only a privacy gain if the local data it touches is encrypted at rest and isolated from other applications, which is why we built that layer years before we built the models, and why we are willing to run inference against it now.

There is one more failure mode specific to this architecture, and it is better raised here than discovered by a reader. Simon Willison calls it the lethal trifecta: access to private data, exposure to untrusted content, and the ability to send external communications. Combine all three, and you have an exfiltration machine, because an attacker can hide instructions inside the untrusted content and have the model act on them. The industry takes this seriously in practice, in June 2026, OpenAI shipped a lockdown mode that simply cuts off web search to prevent documents from leaking through prompt injection.

An on-device messaging assistant has the first two legs and deliberately lacks the third. Incoming messages are absolutely untrusted content; the model can absolutely read your private threads. But with no network egress, there is no channel for extracted data to leave. That is not an accident of the design — it is the reason for it.

‍

What remains is an integrity problem rather than a confidentiality one. A message crafted to manipulate a local summarizer could produce a misleading summary, or a suggested reply the user never intended. Nothing leaks, but the user is deceived. This is real and largely unsolved, and the honest mitigations are unglamorous: treat message content as data rather than instructions, constrain output format, and never let a summary trigger an action the user doesn’t see.

Which features actually belong on-device

Small models are not small versions of frontier models. They are genuinely weaker at general knowledge, multi-step reasoning, and long-context synthesis. Apple says so about the roughly 3-billion-parameter model in its own Foundation Models framework, which it states plainly is not for world knowledge, code, mathematics or factual question answering — it is built for summarization, extraction, refinement and short dialog.

That list is almost exactly the feature set of a secure messaging assistant, which is why this works with three to four billion parameters. These are transformations of text the user already supplied, not questions about the world.

‍The concession worth quoting

In this article, we have cited Matthew Green against cloud inference. It is only fair to report what he says about ours.

Describing an assistant that performs inference but cannot reach the internet, Green writes that your data is perfectly safe, but the assistant is “worthless for all but the simplest tasks: for example, summarizing inbound messages for your consumption, or helping draft text messages.”

Read that list again. Summarizing inbound messages. Helping draft messages.

That is not a footnote to our product. That is our product. The most prominent skeptic of private AI inference concedes that this exact configuration is safe. He simply finds it boring.

For secure communications, boring is the specification.

Where we stand

RealTyme’s position is not that we have added AI. It is that AI runs where your messages already live: on your device, inside the same encryption boundary that protects the conversation itself.

In practice that means four things a security officer can verify rather than take on faith:

1. No plaintext leaves the handset. Not for hard queries, not for long documents, not as a quiet fallback on low-memory devices.

2. No inference endpoint joins your conversation. There is no third party to attest, audit, or trust.

3. No retention policy stands between your data and a court order. There is no record to preserve.

4. No cross-border transfer analysis is required. The data never crosses a border.

And four trade-offs we accept rather than hide:

- A model scoped to what small models genuinely do well, and not marketed as more.

- Honest limits on dialectal Arabic, with per-dialect figures available on request rather than a flattering average.

- An integrity risk from prompt injection that we mitigate rather than claim to have solved.

Where a workload genuinely exceeds what a handset can do, the answer is not a third-party model endpoint. It is a deployment you control: on-premises, an air-gapped sovereign node, or national cloud infrastructure within your own jurisdiction, under your own key management. That option existed before we shipped a single model, and it remains the right one for anything that cannot run locally.

Frequently Asked Questions

Does confidential computing (TEE) fully protect encrypted messages from AI processing?

No — it significantly reduces exposure compared to plain cloud AI, but it doesn't eliminate it. Independent audits, including Trail of Bits' review of WhatsApp's Private Processing, found real implementation flaws (28 issues, 8 high-severity) even in one of the industry's most transparent TEE deployments. TEEs reduce the trust surface; they don't remove the provider from it.

Can trusted execution environments (TEEs) be broken?

Yes. In October 2025, researchers demonstrated TEE.fail, a sub-$1,000 hardware attack using a DDR5 memory-bus interposer that extracted cryptographic keys and forged attestation on Intel SGX, Intel TDX, and AMD SEV-SNP — including SEV-SNP with Ciphertext Hiding enabled. The attack requires physical access to the server, which limits it to insiders and data-center operators rather than remote attackers — but that's precisely the threat model confidential computing exists to address.

Is on-device AI completely safe from privacy risks?

No. On-device AI eliminates the cloud provider as a party to the conversation, but the device itself becomes the entire trusted computing base. Device-resident malware, OS-level compromise, or targeted spyware can defeat on-device AI the same way it defeats end-to-end encryption itself. Microsoft Recall is a documented example of a fully local architecture that still shipped with exploitable security gaps.

Which AI features are actually safe to run on-device?

Small on-device models (roughly 1-4 billion parameters) are well-suited to transformations of text the user already supplied — smart reply, rewriting, chat and document summarization, and speech-to-text. They are not well-suited to open-domain question answering, which requires frontier-scale world knowledge that small models don't have.

Does self-hosting an AI model solve end-to-end encryption?

No — self-hosting solves a different problem. It answers who controls the infrastructure (sovereignty), not whether plaintext leaves the device at all (the end-to-end encryption guarantee). If your own server decrypts a conversation to process it, that conversation is no longer end-to-end encrypted between the two people having it — your server has become a third endpoint, even if it's a trusted one.