Blog - From tokens to answers

Modern LLMs (GPT, Gemini, Claude…) sit on the Transformer architecture from Google’s Attention Is All You Need (2017). The idea: instead of stepping through a sequence like an RNN, compute attention across all tokens at once.

From text to tokens

Text → tokenizer → tokens $t_1,\ldots,t_n$ → embeddings $x_i \in \mathbb{R}^{d}$. We add position (sinusoidal or learned), because attention alone does not know what is “before” or “after.”

Attention: Q, K, V

For each token we build three vectors:

$$Q = X W_Q,\quad K = X W_K,\quad V = X W_V$$

Scaled dot-product attention:

$$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\left(\frac{Q K^\top}{\sqrt{d_k}}\right) V$$

Intuition: $Q$ asks “what am I looking for?”, $K$ says “what do I have”, $V$ is the content to mix. Softmax weights which tokens matter for which.

Multi-head = several attentions in parallel (syntax, references, local context…) then concatenate.

Transformer block

A decoder-only LLM (like GPT) uses a causal mask in every layer: a token sees only itself and the past, not the future — so generation is autoregressive.

Generating an answer

1. Prompt → tokens.
2. Model predicts $p(t_{n+1} \mid t_1,\ldots,t_n)$.
3. Sampling / greedy / nucleus → next token.
4. Repeat until EOS or a limit.

Training is usually next-token cross-entropy (pretraining), then instruction tuning / RLHF.

Why this matters for documents

Long context = attention over many document tokens (classic cost ~ $O(n^2)$ memory). Hence:

• chunking / RAG for large files,
• sparse / linear attention variants,
• KV cache during generation.

Takeaway

An LLM does not “understand” like a human — it is a deep stack of attention + MLP, trained to predict the next token. Google’s Transformer paper is still the skeleton: Q/K/V, multi-head, layers, causal decoding.