Running a model on your own machine

Your data pack finished on the twentieth. The AI app will not load. Someone tells you that you can put an AI inside your own phone and use it with no internet at all.

That is a real thing. It is also much smaller and much weaker than the app you were using. This lesson tells you what it is, what it needs, and whether it is worth your time yet.

What open weights actually means

An AI model is a very long list of numbers. Those numbers are everything it learned. Each one is called a weight, and a model has billions of them.

Some companies publish that list. Anyone can download the file and run the model on their own machine. A model like that is called an open-weights model.

Other companies keep the list private. You never touch the model at all. You type a question, it travels to a building full of computers far away, and the answer travels back.

Both kinds are useful. The difference is where the model sits, and who owns the machine it runs on.

Why someone would want to run one

There are three real reasons, and they are good ones.

It works with no internet. Once the file is on your device, everything happens there. No tower, no cable, no data pack. It keeps working when the network drops.

Nothing you type leaves your device. There is no company on the other end to store it. There is no chat history sitting on somebody else’s machine.

There are no limits and no fees. No message count, no daily reset, no plan to buy. You can ask the same thing fifty times.

That last one matters more in Hindi than in English. A Hindi sentence takes three to four times more word pieces than the same sentence in English. So online limits arrive faster when you ask in Hindi. A model on your own device has no limit to reach.

What it really costs

Now the honest side.

A model small enough for a phone is much weaker than the free apps online. The answers are shorter. It makes things up more often. It handles one clear task at a time, and not much more.

A model good enough to feel like the online apps needs a proper computer with a lot of memory. The whole list of numbers must sit in memory while the model runs. Memory is not the same as storage. Storage holds your files and photos. Memory is the smaller working space, and a phone usually has only a few GB of it. Bigger model, more memory needed. There is no way around that.

It will also be slow on ordinary hardware. On a recent phone chip, a small model writes roughly ten to thirty word pieces a second. That is readable. Put a bigger model on an old laptop and one paragraph can take a long time.

The download costs data as well. The file is big, so get it on a connection you are not paying for by the GB. Our lesson on using a small data pack well is about the same problem. Expect the phone to warm up and the battery to drop while the model is working.

Checked on 9 August 2026. Sizes and speeds move fast, so treat this as a picture of today.

One thing here confuses many people. The best open-weights models are not weak at all. Through 2026 they have run only a few months behind the best closed ones. But those big open models are not the ones that fit in your hand. They need the same costly machines as the closed models do. What fits on a phone is the smallest member of the family.

What a phone can actually run today

You do not have to be a programmer to try this. Apps exist that do the whole job for you.

A few are in common use on Android as of 9 August 2026. Google’s AI Edge Gallery runs Gemma models fully offline. PocketPal AI is another. MLC Chat is built for newer Snapdragon chips. A harder route is Termux, where you type commands and run llama.cpp or Ollama yourself.

Check the maker before you install anything. Fake AI apps are common, and fake apps and copied websites shows you how to spot them.

Once it is running, give it a small clear job:

Write four short lines for a WhatsApp message.
I am reminding a customer about a payment that is late.
Simple English. Polite. No difficult words.

A good answer is four plain lines you could actually send. That is the size of job a small offline model does well. Treat it as a small helper that always works, not as a replacement for the online apps.

Now try a second one, to see where it stops:

Which documents do I need for a caste certificate in Uttar Pradesh?

You will get a confident list. Do not use it. A small model has no way to know the current rules, and it cannot check anything for you. For any office or scheme, go to the office or its own website. Our lesson on understanding schemes and forms covers that properly.

Open weights does not mean free for anything

Downloading is one thing. What you are allowed to do with it is another.

Every open-weights model comes with a licence. Some licences are very open. Others add conditions that matter once money is involved.

Two common open licences are Apache 2.0 and MIT. Model families such as Gemma, Qwen and Mistral have been released under Apache 2.0. DeepSeek has shipped models under MIT. Meta’s Llama family uses its own licence instead. That one adds a condition for very large services. The condition starts once a service passes 700 million users a month.

Open weights is also not the same as open source. The numbers are published. The data the model learned from, and the code that trained it, usually are not.

If you ever build a business on a model, check its licence first:

Name the licence this open-weights model is released under.
List in three lines what the licence does not allow.
If you are not sure, say so instead of guessing.

Take that answer as a starting point only. Then read the licence on the model’s own download page. Our lesson on checking sources and links explains why an AI answer about a document is the least safe kind.

The tools that run in a terminal

You will hear names like Claude Code, and other companies have equivalents. These are a different thing from a chat app, and they deserve an honest answer.

A terminal is the plain text screen on a computer where you type commands instead of tapping. These assistants live there. They can read the files on your computer, change them, and write new ones. The assistant sits on your computer, but the model behind it does not. What it reads from your files still travels to the company’s machines. This is not the same as running a model yourself.

For people who write software, they are genuinely powerful. This is not a toy.

They also need three things. A computer, not a phone. Some comfort with a terminal. And usually a paid plan.

That is a real barrier. It is not a personal failing, and it does not mean you are behind. Most people in the world do not have those three things either.

If you want to walk that road, the order matters. Learn the computer first. Files, folders, typing, saving, copying. Then learn programming basics. Then learn the assistant. That order, over months, not over one weekend. Our lesson on learning computer skills with AI is the right first step.

So should you do this yet?

For almost everyone reading this course, the answer today is no. The free online tools are better. They cost nothing, and they run on the phone already in your hand. This lesson exists so you know the option is there, not so you switch to it.

There are two reasons to still try it once. Seeing a model answer you with flight mode switched on teaches you more than ten videos. And if your network and your power are both unreliable, a weak helper that always works beats a strong one that often does not.

Then check again in a year. Small models are improving quickly. Alibaba released a family of very small open-weights models in March 2026, in sizes meant for modest hardware. The answer that is no today may be yes for your next phone. Our lesson on how fast this is changing explains why that is a safe bet.

One reason never goes away. Running the model yourself is the only way to be certain that what you type is stored nowhere else. Everywhere else, your words reach a company’s machines. What happens after that depends on their rules and your settings, and what happens to what you type covers those settings.

Next comes the question people ask straight after this one. What does all of it mean for AI and jobs in India?