Why AI makes things up

You ask an AI app how much money one scheme gives in your block. It answers in one second, with an exact figure and a last date. Next week the clerk at the office tells you no such amount exists.

The app was not playing a trick on you. It was doing the thing it was built to do.

An exam with no negative marking

There is a name for this. When a model invents something and states it plainly, that is called a hallucination. It is an odd word, but it is the one everybody uses.

It is not lying. A lie needs somebody who knows the truth and hides it. The model does not know its answer is wrong. There is no store of facts inside it to check against.

Now think about the exams you know. In JEE, NEET and GATE, a wrong answer cuts your marks. So a careful student leaves a hard question blank.

Picture the opposite exam instead. No negative marking at all. A blank scores zero. A guess might get lucky. Every sensible student writes something in every box.

These models were graded that way while they were built. Their tests scored each answer right or wrong. Saying “I do not know” scored the same as being wrong, which is nothing. So guessing became the habit, and the habit stayed.

You can see the result in how rarely they refuse. In a large study of free assistants in 2025, more than three thousand questions were put to them. They declined to answer only about one time in two hundred.

A made up answer looks exactly like a real one

This is the whole problem, in one line. There is no tone of voice to listen for.

An invented answer arrives in the same calm, tidy English as a correct one. Same speed. Same confidence. Often the same neat formatting.

It can even sound more certain than the truth deserves. A person who half remembers a date says “sometime in autumn”. A model that is guessing says “30 September”. Being exact is not proof of knowing. Sometimes being exact is a sign of guessing.

Researchers tested this in the simplest way possible. They asked one model for the same person’s birthday three separate times. They got three different wrong dates.

None of those dates was looked up anywhere. As it is guessing the next word explains, the machine writes an answer one small piece at a time. A true date and an invented date fit the sentence equally well.

Which of your questions are the risky ones

You cannot check everything you ask. You do not need to. You need to know which questions are dangerous before you send them.

The risky ones are rare, local, specific and one-off. Your village pincode. One scheme’s exact amount this year. The dates in one officer’s career. The fee at a small private college in your district.

Think about why that is. The model learned patterns from an enormous amount of writing. A fact that appears once in all of that leaves no pattern behind. Researchers put a floor on this. Imagine two facts in ten of one kind appear only once in all that writing. Then the model will get at least two in ten of them wrong.

The safe questions are the opposite. How photosynthesis works. What a percentage is. How to write a leave letter to a headmaster. Those have been written about thousands of times, in much the same way, for years.

Indian questions carry extra risk on top of this. Why it is weaker on Indian questions covers that on its own.

Give it the paper instead of asking from memory

You met this idea earlier in the course. It is the single most useful thing in this whole lesson. Do not ask the model to remember a document. Hand it the document.

Paste the notice into the chat. Or take a clear photo of the page. Then ask your question about that text only.

The gap is measured, and it is wide. Give a model a document and tell it to stick to that document. The best ones then went against it in roughly two to five answers in a hundred. That was the picture in May 2026. Working from memory alone, the same kind of model gets things wrong far more often.

Here is the shape of the request.

I am pasting a notice below. Answer only from this text.
If the answer is not in this text, write NOT IN THE TEXT.
Do not use anything you know from outside it.

[paste the notice here]

Question: what is the last date to apply, and who can apply?

Be careful with that low number, though. It is for the easy job of sticking to a page you handed over. It is not a score for how often the app is right in general.

Giving it your own book, notes or photo shows how to do this on a phone.

One line that lets it say it is not sure

The guessing habit came from grading. You can push back against that grading yourself, in one line, on every factual question.

Answer only if you are sure.
If you are not sure, write NOT SURE instead of guessing.
For each fact, tell me whether you are sure or not sure.

A useful answer then looks like this.

Runs under the state government: sure.
Last date this year: NOT SURE. Check the department website.
Exact amount for your block: NOT SURE.

That is a good outcome, not a failure. It tells you which two things to go and confirm, and which one you can carry.

Be honest about what the line does. It changes what the model is trying to score well on. It does not hand it knowledge it never had. The researchers who studied the grading problem suggested exactly this kind of instruction. It costs you one line, so use it every time.

Search helps, but it does not end the job

Many apps can look things up on the web while they answer. Switch that on for factual questions. It genuinely lowers the invention.

It does not finish the job. A hard set of questions was tested in early 2026. Even the strongest setup, with web search on, still made up claims in about three answers out of ten. The links looked correct. The pages did not always say what the answer claimed.

So search moves the work rather than removing it. Your question stops being “is this true?” and becomes “does that page really say this?”. Only you can answer the second one. Checking is a habit, not a setting you switch on once.

One more thing decides how much invention you face, and that is the tool itself. Smaller and older models make things up far more. In one company’s own tests in 2025, its small cheap model was wrong on about eight out of ten short factual questions. Its larger model was wrong on about half of them.

So a very old free app is not a bargain. It is a worse tool. How it searches, sees and uses tools shows what really happens when an app looks something up.

What it costs when nobody checks

This is not a small worry for careless people. In India, trained professionals have been caught filing documents that quoted judgments which do not exist. The filings were thrown out. The people who filed them paid for it, in money and in standing.

If a lawyer can be fooled by an invented reference, so can you. The rule that profession drew from it works for everyone. Never quote something you have not opened and read yourself.

Researchers who track such court cases around the world counted well over a thousand by the middle of 2026. The count keeps climbing as more people use these tools without checking.

Fake references are convincing because they are built to look ordinary. Checking sources, links and quotes deals with that in full.

Where this goes next

You now know why it invents, and which of your questions pull the invention out. That is half the skill.

The other half is fast checking. The next lesson, the two-chat test and three other checks, gives you four checks that take under two minutes.