The next decade of AI: why inference is moving to the edge

Predicting AI a decade out is a fool's errand for capabilities - but infrastructure follows economics, and economics are legible. Follow the cost of a token, the silicon roadmap, and the direction of open-weight releases, and a clear picture emerges: training stays centralized, inference goes everywhere.

The datacenter squeeze

Frontier training runs cost billions and consume gigawatts - costs that must be recovered through metered inference. Meanwhile every consumer chip now ships with an NPU, and unified-memory laptops hold models that needed a server rack three years ago. Vendors are paying datacenter prices to serve workloads your laptop can run. That arbitrage doesn't survive the decade.

What the transition looks like

• 2024–2026: enthusiasts and privacy-conscious teams move daily workloads local. Open models close to within a few points of frontier.
• 2026–2029: default flips for personal and SMB use - AI ships as software you install, not a subscription you meter. Cloud specializes in frontier reasoning and massive context.
• 2029+: hybrid becomes invisible. Your device handles the constant stream of everyday inference; rare hard problems burst to big iron - explicitly, with consent.

Owning your stack matters more every year

As AI mediates more of your work and life, the difference between renting and owning compounds. Owned models don't change behavior after a vendor update you didn't ask for. Owned memory isn't a profile that gets monetized. Gotchi is our bet on that future: open models, local agents, your data on your disk - with an open core you can audit.

Training is a factory. Inference is electricity. Factories centralize; electricity goes everywhere.

The silicon roadmap tells the story

Follow the hardware announcements rather than the model announcements and the direction is unmistakable. Every consumer chip vendor now ships a neural accelerator as standard: Apple's Neural Engine, Qualcomm's Hexagon, Intel and AMD's NPUs. Unified memory ceilings on consumer machines have tripled in four years. Memory bandwidth - the real constraint on inference speed - improves every generation. None of this silicon exists to run cloud software; it's a bet by every chipmaker simultaneously that inference happens on the device. Chip roadmaps are five-year commitments; this one has already been made.

The open-weight flywheel

Open model releases aren't charity - they're strategy, and the strategy is self-reinforcing. Each major open release expands the ecosystem of tools, fine-tunes, and deployments built on open weights, which raises the value of the next release, which pressures every lab holding capable models to publish or become irrelevant to the largest developer community in AI. The result is a ratchet: open capability only moves forward, and the lag behind the frontier keeps shrinking. Betting against local AI means betting this flywheel stops - and it is accelerating.

• 2023: open models were toys - a novelty gap of 20+ benchmark points to frontier.
• 2024: Llama 3 class closed to ~10 points; local coding assistance became viable.
• 2025: DeepSeek-class efficiency broke the assumption that quality requires scale-priced serving.
• 2026: open mid-size models are indistinguishable from cloud subscriptions for the median task.

What incumbents will do

Expect cloud vendors to respond rationally: push modalities local hardware can't yet match (long-horizon video, massive context), bundle AI into productivity suites where the subscription is already sold, and market 'hybrid' architectures that keep billing relationships alive. Some of this will be genuinely useful. But the strategic ground - the everyday inference that constitutes most AI usage - shifts to the edge regardless, because no bundling strategy beats zero marginal cost and zero data exposure for workloads the device can handle.

How to position yourself now

For individuals: build your workflows on models and data you own; skills and setups transfer forward, subscriptions don't. For teams: treat local AI as infrastructure - a shared config, a knowledge base, an agent library - and let cloud APIs be a burst resource, not a foundation. For builders: the opportunity is the layer above inference - the interfaces, agents, and integrations that turn commodity models into daily utility. That's the layer we're building at Gotchi, in the open, on the assumption this essay is right.

The best time to own your AI stack was before it mattered. The second-best time is now.