Engineering

We cut a coding agent's cloud bill by about half — by letting a local model read your files first

Private AI is usually a promise about intent. We wanted a cost number that survives a hostile test — so we sealed the keys, ran nine real coding tasks twice, and measured what the cloud model was actually billed for.

· Armore Edge

Most AI coding agents work the same way: they read your files, paste the contents into the prompt, and ship the whole thing to a frontier model in someone else's data centre. You pay for every one of those tokens, and your source code takes a trip off your machine to get an answer that a much smaller model could often have produced locally.

We build Armore Edge to break that habit. It runs an AI model on your own device and connects to the AI tools you already use, so they can ask questions of your own files and get answers from a model running on your own computer, with only the access you grant. This post is the measurement behind that sentence: what it does to your bill, what it does to how much of your code leaves the machine, and — because the question keeps coming up — how it sits next to Claude Code's experimental Code Mode.

The problem: you pay twice, in dollars and in exposure

Two costs ride along with every agent turn, and most tools only ever talk about the first one:

The pitch for keeping work on-device is obvious in the abstract. The honest question is whether it actually costs less once you measure it, or whether it is a privacy story dressed up as a savings story. So we measured it.

The experiment: sealed keys, nine coding runs, priced on the real model

We ran a paired A/B over nine real coding tasks — the kind of ticket you would actually hand an agent: find where a behaviour lives, read the relevant code, propose a fix.

The discipline matters more than the result, so here is the method in full:

Two axes, and why they don't give the same number

The most important thing we learned is that "fewer tokens" and "a smaller bill" are two different measurements, and quoting one as if it were the other is exactly the kind of rounding we refuse to do.

Why the difference? Because most of what a coding agent sends is cheap, cached context, billed far below fresh output tokens. Cutting raw token volume by 82% does not cut the invoice by 82%, because a lot of those 82% were the cheapest tokens on the bill. The dollar figure is the one you can take to your finance team, so the dollar figure is the headline: about 55% less, measured.

Measured against that standard grep-and-read agent, routing the task through Armore's on-device answer-first path cut tokens ~82% and the dollar bill ~55%. The saving comes from the frontier not re-reading the files once the on-device tier hands it a complete, trusted answer — so it requires using that path, not just installing Armore.

This is the on-device answer-first path — and it only pays off when you actually route work to it. Left to its defaults, Claude Code will grep and read the files itself, and you see about 0% saving. Exposing Armore only as a search the model follows — where it still reads the files through — is also about 0%. The ~82% / ~55% above is the regime where the ticket is handed to Armore's on-device tier and the frontier trusts the answer it returns. It is not a number you get just by installing the connector.
Four-panel dark infographic of Armore Edge measurement results. Panel one: 65 to 90 percent fewer tokens reach the cloud model. Panel two: about 55 percent lower bill in real dollars. Panel three: about 93 percent of a document's tokens never leave your device. Panel four: 9 of 9 answers correct, no quality loss.
Measured over nine sealed-key coding runs, priced on claude-opus-5-5. The egress figure is per-document tokens (N=3).

Results

What we measuredResultScope / caveat
Tokens reaching the cloud model65–90% fewer (median −82%)N = 9; vs a standard grep-and-read agent
Real-dollar bill~55% lower (median −55%, −58% aggregate)claude-opus-5-5 standard rates, requestId-deduped. Example task: $0.34 → $0.15
Document tokens that never leave your device~93% (92.7–95.5%)N = 3; payload egress down as much as 98%
On-device answer quality9 / 9 correctthe frontier model re-checked 2 of the 9 anyway — the safety net below
Claude Code's Code Mode, stacked on top+3–4% overhead, no compoundingnative parallel tool-calling already batches tool output; see below
The one condition that makes the savings real. The bill goes down on what we call the settled-fact path: when the on-device model returns a correct, high-confidence answer and the frontier model treats it as settled instead of re-deriving it. In our runs the local answer was right 9 out of 9 times — but the saving is contingent on that answer being good. When the local tier is unsure, the frontier model re-checks (it did on 2 of the 9), and on those turns the saving shrinks back toward zero. We would rather tell you that up front than publish a number that only holds on a good day.

How it works: hand the frontier the answer, not the haystack

The mechanism is deliberately boring, which is why it is cheap.

The frontier model still does the hard reasoning and writes the code. It simply stops paying to re-read a haystack your own machine already read. That is where the ~93% egress reduction comes from on document tasks: the document's tokens stay home, and only the answer travels.

Where Code Mode fits — and why it doesn't stack

Claude Code's experimental Code Mode chases the same inefficiency from a different angle: it batches several tool calls into one script and filters the output in-script, so the bulk never lands in context. It is a genuinely good feature, and on big filtering sweeps — "run these twelve searches, give me only the hits" — it wins.

But in our runs, putting Code Mode on top of the on-device tier added only 3–4% overhead and did not compound. The reason is honest and worth stating: both mechanisms save the same way — by keeping bulk tool output out of the cloud context. When they target the same tokens, the second one has less left to save. Claude Code's native parallel tool-calling already batches a lot of what Code Mode scripts, so the marginal gain on top of on-device retrieval was small. We are not going to sell you a stacked "57% plus 65%" number, because the measurement doesn't support one. They are two tools for overlapping jobs; use whichever fits the task.

Reproduce it yourself

None of this needs our word for it. The on-device claim is checkable in about five minutes on your own machine:

armore ask --model your-model.gguf "Summarise the error in ./build.log in one line"

While it runs, watch its sockets from another terminal:

lsof -i -nP | grep -i armore

You will see zero non-local sockets. The answer came off your GPU, not off the internet.

In the desktop app, the same guarantee is a switch: Models → "Sovereign Inference." The command-line tool needs no toggle — it is on-device by construction. Then point your own Claude Code (or Codex, or Cursor, or any MCP client) at the Armore connector, and your agent can start asking questions of your files — read locally, so in the usual path only the answer leaves your laptop, not the file.

Honest caveats

The long-term hypothesis — labelled, because we haven't measured it

Here is the bet we have not yet proven, stated as a bet so you don't mistake it for a result:

As more of a long project's retrieval, reading and extraction moves on-device across many sessions, the per-task savings above could compound into a materially smaller cloud bill and a smaller exposure footprint over the whole life of the project.

That is a hypothesis, and it is unmeasured. Everything with a number in this post is a single-task measurement; we have not measured whole-project trajectories, and we are not going to imply a compounding curve we haven't run. We will measure that next. If it turns out the savings are flat across a long project, we will publish that too.

FAQ

Does using an on-device AI model actually lower my AI bill?

In our nine sealed-key coding runs, yes — about 55% lower in real dollars (median), priced on claude-opus-5-5, when the ticket is routed to the on-device answer-first path. The saving holds when the on-device model returns a correct, high-confidence answer the frontier model can treat as settled. It shrinks when the local answer needs re-checking, and it is ~0% if the agent just greps and reads the files itself.

Why is the token reduction (65–90%) bigger than the dollar reduction (~55%)?

Because most of an agent's context is cheap, cached tokens, billed well below fresh output. Cutting raw token volume by ~82% removes a lot of the cheapest tokens, so the invoice falls by less than the token count does. The dollar figure is the honest one.

Does my source code leave my computer?

On document and file tasks, about 93% of the document's tokens never leave your device (92.7–95.5% measured; payload egress down as much as 98%). The on-device model reads the file locally and hands the cloud model only the short answer. You can verify zero non-local sockets yourself with lsof.

Is this as good as Claude Code's Code Mode?

They solve overlapping problems. Code Mode batches and filters tool output in-script and genuinely wins on big filtering sweeps. Stacked on top of on-device retrieval it added only 3–4% overhead and did not compound, because both work by keeping bulk output out of context. Use whichever fits the task; don't expect the savings to add up.

Does the smaller on-device model make the answers worse?

In this experiment, no — the on-device answers were correct 9 out of 9, and the frontier model independently re-checked 2 of them as a safety net. Quality depends on the task class; we measured coding-retrieval tasks, not every kind of question.

Which on-device model do I need?

Any open-weights GGUF model. We ran Qwen3-8B (Q4_K_M). You bring the model; Armore Edge runs it locally and connects it to your agent.

Is Armore Edge free, and do I need an account?

Yes, and no. It is free, needs no account, and runs on macOS, Windows and Linux. The command-line tool is a ~20 MB download.