Give the model information it wasn't trained on.
Ask a question below. Only the chunks that actually match get retrieved and handed to the model.
- RAG (retrieval-augmented generation)
- A way to give a model information it wasn't trained on, without changing the model itself: split your documents into small chunks, find the chunks most relevant to a question, and place only those into the model's context alongside the question at answer time. The model doesn't learn this permanently, it has to be retrieved again the next time you ask.
- Chunk
- A small piece of a larger document. Documents are split into chunks because a whole document is usually too long to search efficiently or fit in a model's context. This lab's sliders below control exactly how that split happens.
- Retrieval
- The step that scores every chunk against a question and keeps only the most relevant ones. Real systems often use vector similarity, keyword search, or a hybrid of both, sometimes followed by a reranking step. This lab uses simple keyword overlap (explained further down), but the pipeline shape is the same either way.
This demo scores chunks by how many of your question's words appear in them. Real systems often use vector similarity, keyword search, or a hybrid of both (see the Embeddings lab) — vector similarity can catch related meaning even without exact word matches, which keyword overlap alone can't. Try “How does a network change its parameters?” above and check the matched words in the technical drawer (Engineer Mode): the chunk that actually answers this only matches on generic words like “network” and “its”, never on “parameters” itself, because the document only ever says “weights”. Vector similarity would catch that connection; keyword overlap can't.
Context sent to the model
Context: - Attention is the mechanism that lets a transformer decide, for every token it's processing, how much weight to give every other token in the sequence. Concretely, each token is turned into three vectors: a query, a key, and a value. - at an entire sequence of tokens at once and uses an attention mechanism to decide which other tokens in that sequence are relevant to each one. This lets it capture long-range relationships in text far more efficiently than earlier architectures - in text far more efficiently than earlier architectures could. Transformers were introduced in a 2017 paper called 'Attention Is All You Need', and the core idea has since become the foundation for GPT, Claude, and nearly every other large language Question: How does attention work?