What AI can and cannot do (hallucinations and limits)

What AI does reliably

It is at its best working with text you hand it yourself: rewriting, summarising, translating, adjusting tone, brainstorming, explaining and writing first drafts. This is where it is an enormous help and rarely gets things wrong.

Where it makes things up (hallucinations)

AI almost never says "I do not know". When it does not have the answer, it will happily invent one with full confidence - a name, a number, a section of law, a quote, a link. That is called a hallucination and it is the biggest trap for beginners.

Why does it happen? Remember the first lesson - AI predicts likely words, not truth. "Sounds plausible" is not the same thing as "is true" for it.

When to be extra careful

• Specific facts, figures and statistics.
• Law, tax, health - here bad advice can cost you money, or worse.
• Quotes and links - AI will often invent them so they look completely real.
• Current events - the model knows nothing after its training cut-off.
• Calculations - arithmetic is not its strong suit.

How not to get burned (three rules)

1. Give it the source material. When it answers from your own text or document, it barely ever fibs. Asking it "off the top of its head" is the worst case.
2. Check the facts. Numbers, laws and names in particular - always check them against a second source.
3. Take it as a suggestion, not a verdict. On anything important, AI is the first draft, not the last word.

AI is a great first draft and a terrible last word. Anyone who gets that uses it better than 90 % of people.