A laptop disconnected from Wi-Fi used to become noticeably less clever.

Email stopped. Cloud documents complained. Search disappeared. AI tools, especially, became blank boxes waiting for a network connection.

That is changing faster than I expected.

Phones and laptops are increasingly able to run useful AI models directly on the device. Not the biggest models in the world, and not every task. But enough summarising, rewriting, classification, image understanding and small automated actions can now happen locally that the cloud is no longer the automatic destination for every AI request.

I think this will change software in a quieter way than the chatbot boom did.

Offline is a feature again

There is something slightly ridiculous about needing an internet connection to rewrite a paragraph that is already on your laptop.

Cloud AI made sense because large models needed expensive hardware. Sending the request to a data centre was the practical choice. But smaller models have improved, chips now include hardware designed for AI workloads, and software frameworks are getting better at using those resources.

So a note-taking app can summarise a document during a flight. A photo app can classify images without uploading every picture. A developer tool can perform a small code task while the train enters a dead zone.

Offline capability sounds old-fashioned until the hotel Wi-Fi opens a login page that refuses to load.

Then it feels excellent.

Privacy is where local AI earns attention

A lot of people paste private information into AI systems without thinking about where it goes.

Meeting notes. Customer emails. Contracts. Personal photos. A developer copies an error log and only later notices that it contains account names and tokens.

Processing something on the device can remove one important part of that data journey. The source material may never need to leave the phone or laptop.

That does not make the device magically secure. Malware still exists. Files can be stolen. A badly designed app can store output carelessly.

But “the data stayed here” is a useful property, especially for features that work on personal information all day long.

I suspect people will care more about this once AI becomes less of a separate tool and more of a layer inside ordinary apps.

Smaller models can be better when the job is small

The AI industry has spent years training everyone to look at size as a score.

Bigger model. More parameters. Larger context. Higher benchmark.

That matters for difficult reasoning and broad tasks. It matters much less when the job is “turn this voice note into three tasks” or “classify these messages into five known categories.”

A smaller model designed for a narrow job can be faster, cheaper and easier to run locally. It may also produce more predictable output because it has less freedom to wander.

This is not a new idea in computing. We do not run every calculation on the most powerful computer available. We use enough machine for the job.

AI is finally rediscovering that ordinary principle.

The fan will tell you when you’ve gone too far

Local AI has physical limits.

Run something too large and the laptop becomes warm, the battery begins disappearing and the fan develops a strong opinion about your experiment. Phones have even tighter limits because nobody wants an assistant that reduces battery life by lunch.

This is why specialised AI hardware matters. Modern devices increasingly include NPUs or similar accelerators meant to handle machine-learning work efficiently. Developers can send suitable tasks there instead of making the general-purpose processor do everything.

The experience still varies widely by device. A feature that feels instant on a new laptop may be slow on an older one. Developers now have another compatibility question: not only “Can this operating system run my app?” but “Can this hardware run this model well enough to be useful?”

That will create some messy years.

Hybrid will beat the purity contest

People love turning architecture into ideology.

Local AI supporters talk about privacy and independence. Cloud AI supporters point to larger, stronger models and easier updates. Both are right often enough to become annoying.

Most good products will use both.

A phone can handle a quick classification locally, then send a harder request to a cloud model when the user asks for deeper analysis. An app can keep sensitive data on the device while sending a stripped-down version of the task elsewhere. A laptop can continue with basic features offline and become more capable when connected.

The user should not have to think about this every time.

Good software will choose sensibly and explain the boundary when privacy or cost matters.

Developers get a new kind of constraint

Cloud AI made development strangely comfortable. The provider worried about memory, hardware and much of the model serving. The app sent a request and received an answer.

On-device AI gives developers more control and more homework.

Model size matters. Quantisation matters. Battery use matters. Startup time matters. A feature that downloads several gigabytes without warning will not become popular simply because the model is clever.

Then there is versioning. Do you ship a model with the app? Download it later? Use the model built into the operating system? What happens when a user has an older device?

These are ordinary software questions, which is exactly why local AI is becoming real. The conversation is moving from research capability to product trade-offs.

The interesting apps may stop advertising AI

The first generation of generative AI products often placed the AI at the centre.

Open the app. See the chat box. Ask the AI something.

On-device AI fits more naturally into features that do not need a conversation. A camera can find the right photo. A mail app can pull out a date. A travel app can organise screenshots into an itinerary. A keyboard can rewrite a sentence without sending it away.

Nobody needs to know the model name.

That is usually what happens when technology matures. The impressive component becomes part of the plumbing.

I don’t think the cloud is going anywhere. The largest models will still live there, and many applications will need them.

But the assumption that intelligence must live somewhere else is weakening.

Sometimes the useful model will be sitting quietly inside the machine already on your desk.