IntelliTechs
Artificial Intelligence

What a language model is, and why it sometimes makes things up

IntelliTechs ยท Oct 04, 2026

What this means for you

Treat chatbots as a fast assistant for drafts, summaries and ideas, but double-check dates, numbers, names and quotes before you rely on them.

What a language model is

A language model calculates the probability of a word (or a piece of a word) showing up after a stretch of text.

Tokens: how the model reads text

Computers work with numbers, not letters. So text is chopped into tokens, pieces that can be a whole word, part of a word or a single character.

What a context window is

The context window is the number of tokens the model can consider at once. It includes your request, the conversation so far and the answer being written.

Why it makes things up

The model does not look facts up in a database. It generates the text that seems most likely. Most of the time, the most likely text is also the correct one.

Getting the most out of it

Think of a language model as a brilliant, fast, extremely well-read intern who never admits they don't know something.

What a language model is, and why it sometimes makes things up

Read the full article

Read the full article