We cut a coding agent's cloud bill by about half — by letting a local model read your files first
Private AI is usually a promise about intent. We wanted a cost number that survives a hostile test — so we sealed the keys, ran nine real coding tasks twice, and measured what the cloud model was actually billed for.
Most AI coding agents work the same way: they read your files, paste the contents into the prompt, and ship the whole thing to a frontier model in someone else's data centre. You pay for every one of those tokens, and your source code takes a trip off your machine to get an answer that a much smaller model could often have produced locally.
We build Armore Edge to break that habit. It runs an AI model on your own device and connects to the AI tools you already use, so they can ask questions of your own files and get answers from a model running on your own computer, with only the access you grant. This post is the measurement behind that sentence: what it does to your bill, what it does to how much of your code leaves the machine, and — because the question keeps coming up — how it sits next to Claude Code's experimental Code Mode.
The problem: you pay twice, in dollars and in exposure
Two costs ride along with every agent turn, and most tools only ever talk about the first one:
- Token cost. A coding session is mostly context — the files, logs and diffs the agent reads to figure out what to do. The model is billed for all of it, every turn, even when the useful answer is three lines long.
- Exposure. To read those files, the agent sends them off-device. For a hobby project that is a shrug. For a private codebase — client work, anything under NDA, anything you would not paste into a public box — it is the whole problem. As we put it on the rest of this site: the problem is not the server. It is that nobody told you.
The pitch for keeping work on-device is obvious in the abstract. The honest question is whether it actually costs less once you measure it, or whether it is a privacy story dressed up as a savings story. So we measured it.
The experiment: sealed keys, nine coding runs, priced on the real model
We ran a paired A/B over nine real coding tasks — the kind of ticket you would actually hand an agent: find where a behaviour lives, read the relevant code, propose a fix.
- Baseline arm: a standard grep-and-read coding agent — ordinary file tools (read, grep, glob) that locate and read the files the ticket needs, and nothing else. Not a strawman that reads the whole repo; an agent that greps and then reads what it finds.
- Local-compute arm: the same agent, routed to an Armore Edge on-device tier — it asks the tier for a complete answer to the ticket instead of reading the files into the cloud prompt itself.
The discipline matters more than the result, so here is the method in full:
- Sealed answer keys. The correct answer for each task was fixed and hashed before any run, so we could not grade toward a number we liked.
- Priced on
claude-opus-5-5standard rates, the actual frontier model, not a cheaper stand-in. requestId-deduplicated billing. Retries and cache reads are counted once, the way a real invoice counts them — no double-charging to inflate the baseline.- On-device model: Qwen3-8B (Q4_K_M) — a small, open-weights model running locally, nothing fine-tuned to the test.
Two axes, and why they don't give the same number
The most important thing we learned is that "fewer tokens" and "a smaller bill" are two different measurements, and quoting one as if it were the other is exactly the kind of rounding we refuse to do.
- Tokens reaching the cloud model fell by 65–90% (median −82%, range −65% to −94%). A genuine, large cut in raw volume.
- The real-dollar bill fell by about 55% (median −55%, −58% in aggregate). Smaller than the token cut — and that gap is the honest part.
Why the difference? Because most of what a coding agent sends is cheap, cached context, billed far below fresh output tokens. Cutting raw token volume by 82% does not cut the invoice by 82%, because a lot of those 82% were the cheapest tokens on the bill. The dollar figure is the one you can take to your finance team, so the dollar figure is the headline: about 55% less, measured.
Measured against that standard grep-and-read agent, routing the task through Armore's on-device answer-first path cut tokens ~82% and the dollar bill ~55%. The saving comes from the frontier not re-reading the files once the on-device tier hands it a complete, trusted answer — so it requires using that path, not just installing Armore.
Results
| What we measured | Result | Scope / caveat |
|---|---|---|
| Tokens reaching the cloud model | 65–90% fewer (median −82%) | N = 9; vs a standard grep-and-read agent |
| Real-dollar bill | ~55% lower (median −55%, −58% aggregate) | claude-opus-5-5 standard rates, requestId-deduped. Example task: $0.34 → $0.15 |
| Document tokens that never leave your device | ~93% (92.7–95.5%) | N = 3; payload egress down as much as 98% |
| On-device answer quality | 9 / 9 correct | the frontier model re-checked 2 of the 9 anyway — the safety net below |
| Claude Code's Code Mode, stacked on top | +3–4% overhead, no compounding | native parallel tool-calling already batches tool output; see below |
How it works: hand the frontier the answer, not the haystack
The mechanism is deliberately boring, which is why it is cheap.
- Your agent needs to know something that lives in your files — "where is auth enforced", "what does this 400-line log actually say", "which function sets this flag".
- Instead of pasting the whole file or log into the cloud prompt, the agent asks the on-device model, over a local connection. The big artifact — the source file, the log, the scanned doc — is read on your machine, and in the usual path the cloud model gets the on-device answer, not the file itself.
- The on-device model returns just the answer: the citation, the summary, the extracted field. In the usual path, that short answer is the only thing the frontier model sees.
The frontier model still does the hard reasoning and writes the code. It simply stops paying to re-read a haystack your own machine already read. That is where the ~93% egress reduction comes from on document tasks: the document's tokens stay home, and only the answer travels.
Where Code Mode fits — and why it doesn't stack
Claude Code's experimental Code Mode chases the same inefficiency from a different angle: it batches several tool calls into one script and filters the output in-script, so the bulk never lands in context. It is a genuinely good feature, and on big filtering sweeps — "run these twelve searches, give me only the hits" — it wins.
But in our runs, putting Code Mode on top of the on-device tier added only 3–4% overhead and did not compound. The reason is honest and worth stating: both mechanisms save the same way — by keeping bulk tool output out of the cloud context. When they target the same tokens, the second one has less left to save. Claude Code's native parallel tool-calling already batches a lot of what Code Mode scripts, so the marginal gain on top of on-device retrieval was small. We are not going to sell you a stacked "57% plus 65%" number, because the measurement doesn't support one. They are two tools for overlapping jobs; use whichever fits the task.
Reproduce it yourself
None of this needs our word for it. The on-device claim is checkable in about five minutes on your own machine:
- Download the ~20 MB command-line tool from github.com/uniQ-Ventures/armore-edge-public (v1.0.12, released publicly on 13 Nov 2026). No account.
- Bring any open-weights GGUF model — we used Qwen3-8B (Q4_K_M), but any will do.
- Ask it something about a local file:
armore ask --model your-model.gguf "Summarise the error in ./build.log in one line"
While it runs, watch its sockets from another terminal:
lsof -i -nP | grep -i armore
You will see zero non-local sockets. The answer came off your GPU, not off the internet.
In the desktop app, the same guarantee is a switch: Models → "Sovereign Inference." The command-line tool needs no toggle — it is on-device by construction. Then point your own Claude Code (or Codex, or Cursor, or any MCP client) at the Armore connector, and your agent can start asking questions of your files — read locally, so in the usual path only the answer leaves your laptop, not the file.
Honest caveats
- The savings are for the settled-fact path. They hold when the on-device answer is correct and high-confidence enough for the frontier model to trust, and when the agent is actually routed to that path. A wrong or low-confidence local answer triggers a re-check, and the saving on that turn shrinks. We measured the good case and told you where it breaks.
- This is nine coding tasks, not your codebase. N = 9 for the cost axis, N = 3 for egress. Directionally strong and sealed-key honest, but it is a measurement, not a guarantee about your specific repo.
- "Fewer tokens" ≠ "smaller bill." The 65–90% token cut and the ~55% dollar cut are different quantities; cached context is why. We report both so neither can be mistaken for the other.
- Downloads are not users. Armore Edge has 350+ downloads across three public releases. We have no usage telemetry — downloads include re-downloads, CI and mirrors — so we count downloads and say so.
- On-device is not all-offline. Downloading a model needs the network; syncing between your own devices needs the network. The load-bearing claim is narrower and stronger: the thinking happens on your machine.
The long-term hypothesis — labelled, because we haven't measured it
Here is the bet we have not yet proven, stated as a bet so you don't mistake it for a result:
That is a hypothesis, and it is unmeasured. Everything with a number in this post is a single-task measurement; we have not measured whole-project trajectories, and we are not going to imply a compounding curve we haven't run. We will measure that next. If it turns out the savings are flat across a long project, we will publish that too.
FAQ
Does using an on-device AI model actually lower my AI bill?
In our nine sealed-key coding runs, yes — about 55% lower in real dollars (median), priced on claude-opus-5-5, when the ticket is routed to the on-device answer-first path. The saving holds when the on-device model returns a correct, high-confidence answer the frontier model can treat as settled. It shrinks when the local answer needs re-checking, and it is ~0% if the agent just greps and reads the files itself.
Why is the token reduction (65–90%) bigger than the dollar reduction (~55%)?
Because most of an agent's context is cheap, cached tokens, billed well below fresh output. Cutting raw token volume by ~82% removes a lot of the cheapest tokens, so the invoice falls by less than the token count does. The dollar figure is the honest one.
Does my source code leave my computer?
On document and file tasks, about 93% of the document's tokens never leave your device (92.7–95.5% measured; payload egress down as much as 98%). The on-device model reads the file locally and hands the cloud model only the short answer. You can verify zero non-local sockets yourself with lsof.
Is this as good as Claude Code's Code Mode?
They solve overlapping problems. Code Mode batches and filters tool output in-script and genuinely wins on big filtering sweeps. Stacked on top of on-device retrieval it added only 3–4% overhead and did not compound, because both work by keeping bulk output out of context. Use whichever fits the task; don't expect the savings to add up.
Does the smaller on-device model make the answers worse?
In this experiment, no — the on-device answers were correct 9 out of 9, and the frontier model independently re-checked 2 of them as a safety net. Quality depends on the task class; we measured coding-retrieval tasks, not every kind of question.
Which on-device model do I need?
Any open-weights GGUF model. We ran Qwen3-8B (Q4_K_M). You bring the model; Armore Edge runs it locally and connects it to your agent.
Is Armore Edge free, and do I need an account?
Yes, and no. It is free, needs no account, and runs on macOS, Windows and Linux. The command-line tool is a ~20 MB download.