Tokens and Context Windows

KEY TAKEAWAY

Tokens are the pieces of text AI processes. The context window is how much it can remember at once.

💡 EVERYDAY ANALOGY — BEFORE THE TECH

📖

Context window: how many pages the AI can see at one time

Imagine reading a thousand-page book but only able to see 50 pages at once. If the answer requires connecting page 1 and page 800, you cannot do it — they are never in view together. That is the challenge AI faces with context windows. More tokens means more pages visible at once, which means the AI can reason across longer documents.

👁️ Visual Explanation

📖

Text being split into tokens. Context window showing maximum token capacity.

🌍 In Plain English

LLMs process tokens, not words. One token is roughly three-quarters of a word. The context window is the maximum tokens the AI can process at once — its working memory. GPT-4 handles around 128,000 tokens. Claude handles around 200,000.

Real-World Example

One million tokens is roughly 750,000 words — about 10 full novels. Claude can hold all of that in context simultaneously, which is why it can analyse entire books or codebases.

🎓 For Students

If you paste a long research paper into an AI, whether it can read the whole thing depends on the context window. If the paper exceeds the limit, the AI will miss parts.

💼 For Professionals

Legal teams can feed entire contracts into large-context AI models and ask questions — the AI reads the whole thing at once.

🔧 Technical DepthOptional — expand when ready

Tokenisation converts text to integers using byte-pair encoding (BPE). Context window size determines GPU memory requirements. The attention mechanism processes all tokens simultaneously, which is why longer contexts cost more to run.

Key Terms

TokenThe basic unit of text — roughly three-quarters of a word. ChatGPT is approximately 2 tokens.

Context windowThe maximum number of tokens an LLM can process at once — its working memory

TokenisationThe process of splitting text into tokens before feeding it to the model

⚠️ Common Misconception

A bigger context window does not always mean better performance. Models can lose track of information in very long contexts — sometimes called the lost-in-the-middle problem.