Why it makes things up, and what you can actually do about it

Every manager-track AI certification tests this, usually filed under "fabrications" or "reliability." It's also the first thing your team will ask you about, so it's worth understanding properly rather than as a slogan.

The one-sentence version

These systems produce the most plausible continuation of the text so far. They are not looking anything up, and they have no internal signal separating "I know this" from "this is the shape an answer would take."

Everything useful follows from that.

Why confident and wrong travel together

A person who doesn't know something usually sounds like it. The sentences get shorter, the hedges come out. We've spent our whole lives reading that signal and it's genuinely informative.

A language model writes fluent, well-structured, confident prose whether or not the content is right, because fluency and accuracy are separate properties of the output. The confidence you're reading is a feature of the writing. It is not evidence about the knowledge behind it.

This is the part that takes real effort to internalize, because it inverts a lifetime of instinct. The tone tells you nothing. A fabricated citation reads exactly like a real one.

Where it's most dangerous

Not where people expect. The risk isn't the obviously hard question, because someone will check that. It's the plausible detail sitting inside an otherwise correct answer:

• a summary of a real contract that gets one clause subtly wrong
• a correct process description with an invented step number
• a genuine regulation cited with the wrong section
• a real person quoted saying something they never said

Each of those sits inside work that is broadly right, which is precisely why nobody re-reads it.

What actually reduces it

Give it the source material. By far the biggest lever. If the answer has to come from a document you supplied, there's much less room to invent one. This is what people mean by "retrieval" or "grounding," and it's what RAG describes: fetch the relevant documents first, then ask the question with those documents attached.

It doesn't eliminate the problem. The system can still misread what you handed it. But it changes the failure from "invented out of nothing" to "misinterpreted something real," and the second is far easier to catch.

Require citations. Ask for the specific passage each claim came from. You get two things out of it: a reviewer can check in seconds rather than minutes, and having to point at a source makes unsupported claims harder to produce in the first place.

Give it a way to say no. A system with no acceptable way to decline will guess, because producing something is what it does. Telling it explicitly that "not enough information" is a valid and expected answer measurably reduces invention. Most deployed systems never bother.

Check the retrieval, not the wording. When a grounded system starts producing confident nonsense, the first thing to look at is which documents it was actually given. Not the prompt. If the wrong material came back, the model answered faithfully from bad input and no amount of prompt editing will fix it. This is the most common diagnostic mistake there is, and teams lose weeks to it.

What doesn't work

Telling it to be accurate. "Only provide factual information" and "do not make anything up" are instructions, and instructions shift behavior without guaranteeing it. Include them by all means. They are nowhere near sufficient.

Waiting for a better model. More capable models do fabricate less. They still fabricate, and when they do it's more convincing. If your process depends on a human spotting the error, a better model can leave you worse off than you started.

Spot-checking once at launch. A check performed once tells you about that day. These systems are sensitive to shifts in the input mix, and a document set refreshed six months from now can quietly degrade the output with nothing erroring anywhere.

The question to ask in a design review

Not "how accurate is it?" You'll get a number with no context and no way to interrogate it.

Ask: when it's wrong, who notices, and how?

If the answer is "the user will spot it," ask whether the user is in any position to. Someone asking a question usually can't evaluate the answer; that's why they asked. If the answer is "we sample and review," ask what the sample rate is and who does the reviewing. If the answer is a pause, you've found the real risk, and it isn't the model.

Where the certifications land on this

Microsoft's AB-731 blueprint puts fabrications, reliability, and bias in its first and heaviest domain, right next to grounding and retrieval. That's the right instinct. It treats knowing why it happens as prerequisite to knowing what to do about it. AWS and Google both cover the same ground under responsible-AI and model-limitation headings.

Expect scenario questions rather than definitions. The usual shape is a system behaving badly and four plausible responses, where the correct one is the least dramatic intervention that addresses the actual cause.