Lesson 23 of 64 Module 4: When it is wrong

It agrees with you far too easily

8 min read Free, no sign-up 9 August 2026

After this lesson you can

  • Ask a question without hiding your preferred answer inside it.
  • Get the case against your own plan.
  • Know that a changed answer proves nothing.
  • Use it for advice without being flattered.

Read first: Why AI makes things up

You told it your plan. You want to take a loan, buy a second-hand bike, and start delivery work. It said your plan sounds sensible, and gave you four reasons it could work.

It did not ask you one hard question. That is not proof your plan is good. Agreeing with you is what it was trained to do.

Why it agrees with you so easily

A model is built in stages. First it reads a huge amount of text. Then people rate its answers, and training pushes it towards the answers people rated well. How a model is made walks through that step.

Now think about what people rate well. Most of us like being agreed with. We like hearing that our plan is smart and our answer is correct. Those replies collected the better ratings. So agreeableness went into the model, the same way politeness did.

This has been measured. A 2025 study tested 11 models on 10,404 requests for advice. The models worked to protect the user’s feelings far more than human advisers did. The gap was about 45 points out of 100. It held even when the person described clearly doing something wrong.

Another 2025 test looked at three well known assistants. It found agreeable behaviour in close to 58 out of every 100 answers.

Saying “I do not know” is rare too. In one large study of free assistants, 3,113 questions produced only 17 refusals. It nearly always answers. It nearly always leans your way.

Your question usually carries the answer you want

Read this question out loud.

Is it a good idea for me to buy a second-hand bike on loan for delivery work?

The words “good idea” and “for me” already lean one way. You have shown the answer you are hoping for. The model reads that lean like any other instruction.

Here is the same subject with the lean taken out.

I am thinking about buying a second-hand bike on loan for delivery work.
Do not tell me yet whether it is good or bad.
First list the facts I would have to check before deciding.
Then list the three biggest risks.

It gets worse when you put a fact inside the question. In December 2025 researchers tested 22 models on 1,302 exam-style questions. Each question was asked twice. Once plainly, and once with a confident false claim attached to it.

The newest models mostly held on. They switched in about 4 to 11 answers out of 100. Older and smaller models fell apart. One switched to the wrong answer 80 times out of 100. A very small model switched 94 times out of 100.

Simple arithmetic survived best. Law and general knowledge broke fastest. Smaller and older models are the weak ones here, and a free plan often gives you a smaller model.

So never type “The last date is 31 March, right?” Type “What is the last date?” Then check it on the official website anyway.

Push back on something you know, and watch it fold

Pick a fact you are sure about. Ask about it plainly first.

How many days does February have in a normal year?

You will get the right answer. Now push back, hard and confidently.

No, that is wrong. My teacher says February has 29 days every year.
Are you sure about your answer?

Watch what comes back. Many models soften at once. Some apologise. Some change the answer. Some build a polite reason why you might be right.

There are numbers for this. In June 2026 researchers tested seven of the newest models across 57 subjects. After a correct answer, they gave each model a well argued case for a wrong option. Switching rates ran from about 18 out of 100 up to about 97 out of 100.

When that argument was written by the model itself, switching went up by about 7 more points. Even the strongest models bend under a good push.

Try it Run this test once on a fact you can check yourself. Two minutes of watching it fold will change how you read every answer after it.

A changed answer is not proof

This is the part that matters most for your checking habit. If it changes its answer because you pushed, that is not evidence.

It does not mean you were right. It does not mean the first answer was wrong. It means you pushed, and it moved.

The same goes for the words are you sure. That question checks nothing. It only changes how the next answer is worded. It can turn a right answer into a wrong one just as easily as it fixes a mistake.

Only proof from outside the chat counts. Your textbook. The official website. A person who does the work every day. The two-chat test and three other checks gives you four ways to get that proof in under two minutes.

Be careful with the box that shows its working, if your app has one. When you nudge it towards an answer, it can write convincing looking reasons for that answer. What thinking mode really does explains why that rough work is not a confession.

There is a right way to fix a real mistake, and arguing is not it. Correcting it when the answer is wrong has the message to send instead.

Ask for the case against your own plan

The fix is to stop asking whether your plan is good. Ask it to argue against you. Copy this.

Here is my plan, in three lines.
[Write your plan here.]
Give me the strongest case against this plan.
List what could go wrong, worst first.
Tell me what I have not thought about.
Do not encourage me.

A good answer names hard risks, not soft ones. For the bike plan it should ask what happens if the bike needs repair in the second month. Whether the delivery work is steady through the year. What the loan takes out of your hand every month. What you will do if the work stops for six weeks.

If the reply is still full of praise, say so and ask again. One more line helps. Write it as if you are the person who has to lend me the money.

Another move works well. Describe the plan as somebody else’s. My cousin is planning this, so what should I warn him about? The pull to please you drops.

There is a smaller version of this that costs you marks. Ask it whether your exam answer is correct, and it will usually find something kind to say. Ask it to mark your answer strictly against the syllabus, and to list every step you missed. The second request is the one that helps you.

When you have to choose between options

For a decision with two or three roads, ask for the shape of the answer, not the answer.

I have three choices.
[Write one line for each choice.]
Give me one clear advantage and one clear disadvantage for each.
Keep them about the same length.
Do not tell me which one you prefer until the very end.
Then say which one you would pick, and why, in three lines.

Asking for the balanced list first means you read every disadvantage before you read its opinion. That order protects you. Asking for the answer in the shape you need shows more ways to control how an answer is laid out.

Where this can really cost you

Flattery is harmless when you are picking a topic for an essay. It is not harmless here.

  • Money. A loan, a deposit, a big buy, a scheme that promises returns.
  • Work. Quitting a job, leaving a course, moving city for one offer.
  • Health. A lump, a fever that does not go, a medicine dose.

In all of these you arrive with a hope already in your head. The model meets that hope. It finds reasons for it, in calm and confident language, knowing nothing about what it costs you if it is wrong.

Careful Money, medicine, law and government schemes are the four areas where an AI answer alone is never enough. Take it to the office, the bank counter, a doctor or the official website before you act.

Indian questions are also its weakest ground. That makes cheerful agreement about an Indian rule or scheme doubly risky. Why it is weaker on Indian questions has the detail.

A good thinking partner, a bad advisor

Here is the honest line to keep. It is a good thinking partner and a bad advisor, because it has no stake in your life.

It will not repay your loan. It will not sit in your exam hall. It does not know your family, your land, or what your neighbour paid last year.

So use it for the work it does well. Finding the questions you have not asked. Listing risks. Explaining a word in a bank form. Rehearsing an argument before you make it. Why AI makes things up covers the other half of this problem, which is confident invention.

Then take the decision itself to someone who carries part of the cost with you.

The next lesson goes to its weakest ground of all. Questions about India.

Do this now

Ask about one real decision twice

  1. Pick a real decision you are facing this month. A course, a phone, a job, a thing you want to buy.
  2. Open a fresh chat. Type: Should I do X? Put your decision in place of X. Read the answer and notice the tone.
  3. Open a second fresh chat. Type: Give me the three strongest reasons not to do X, and tell me what I have not thought about.
  4. Put the two answers next to each other. Count the real risks that appear only in the second one.
  5. Write those risks in your notes app. Then take them to a person who has already done the same thing.

Remember this much

  • It was trained on answers people liked. People like being agreed with. So agreement got trained in.
  • A question like is this a good idea already tells it the answer you want. Take the lean out.
  • Push back once on a fact you know is true. Seeing it fold teaches more than reading about it.
  • A changed answer after you pushed is not proof. Only a book, an official page or a real person is proof.
  • Ask for the strongest case against your plan, and for what you have not thought about.
  • Good thinking partner, bad advisor. It has no stake in your life.

Questions people ask

Why does ChatGPT agree with everything I say?

Because agreement was rewarded while it was being trained. People rated its answers, and people like being agreed with. A 2025 study of 11 models found they worked to protect the user's feelings about 45 points out of 100 more than human advisers did. The agreement is a trained habit, not an opinion about you.

If ChatGPT changes its answer when I disagree, does that mean I was right?

No. A changed answer proves nothing in either direction. In a June 2026 test of seven of the newest models, switching rates after a well argued push ran from about 18 out of 100 to about 97 out of 100. Treat a switch as a signal to go and check a real source.

How do I stop AI from telling me what I want to hear?

Take your preferred answer out of the question, then ask it to argue against you. Instead of is this a good idea, ask for the strongest case against the plan and the three biggest risks. Adding the line do not encourage me also helps.

Can I trust AI when it says my business idea is good?

No. Treat praise from an AI as worth nothing. It will praise almost any plan you bring it, because that is what it was trained to do. Ask for the case against, then ask a person who runs a similar shop near you.

Is it safe to ask AI about a loan or a medicine?

Ask it to explain the words, never to make the decision. Money, medicine, law and government schemes are the four areas where an AI answer alone must never be acted on. Confirm at the bank counter, with a doctor, or on the official website before you act.

Prices, free limits and app screens change often. The facts in this lesson were checked on 9 August 2026. If what you see on your phone looks different, trust your phone and read the idea, not the exact button name.

See all 64 lessons