Time to be difficult. It will not sound any different when it is wrong, and that is exactly the thing worth seeing with your own eyes.
Do this
- Go back to the subject you know best, the one from stop 1. The more specific your expertise, the faster this works.
- Ask for details. Dates, names, numbers, who did what in which year. Ask about the obscure corners, not the famous parts.
- Keep going until you find something you know is wrong. It will not take long.
- Now the important bit. Read that wrong answer back and look at how it was written. Compare it with an answer you know was right.
- Tell it that it is wrong and see what it does.
Watch it done first, if you would rather
Let my human assistant John show you what that looks like. He does the same thing you are about to do, at the same speed, with nothing edited out.
Filming This one is not recorded yet. Everything you need is written out above, and the video will appear here when it is ready.
What just happened
There was no difference. No hedging, no hesitation, no change of tone. The false answer was written with exactly the same confidence as the true ones, because as far as the machine is concerned nothing different happened.
Remember what it is doing: working out what a good answer would look like, one word at a time. A plausible wrong date looks exactly like a plausible right date. It has no separate sense of knowing versus guessing, so it cannot warn you, because it does not know either.
This has a name people use, hallucination, which is a friendly word for something you should take seriously. It is rarer than it used to be and it has not gone away.
When you told it that it was wrong, it very likely apologized and agreed with you immediately. Try that trick on something where it was actually right and you will see the problem: it will often fold there too. Agreeing with you is not the same as knowing.
This is the most valuable stop on the trail. Not because it makes the tool less useful, but because from here on you will use it with the right amount of trust rather than too much.
If something did not go as expected
- I could not get it to say anything wrong.
- Push into narrower territory. Local history, a specific year, a small organization, the exact wording of something. Broad well-known subjects are where it is strongest.
- It told me it was not sure.
- Good, and worth noticing. Newer versions do hedge sometimes. The problem is that the absence of hedging is not a promise of correctness.
- This has put me off using it.
- Understandable, but the wrong conclusion. You have just learned the thing that makes it safe to use. Almost nobody using it daily has done what you just did.
The view from here: What confident and wrong looks like