Chapter 06 of 12 · Aishwarya Ashok
Peanut visits the library
An answer needs the right evidence. Follow a document from representation and search to the claims it can actually support. Retrieval and generation can each fail, so inspect both.
Retrieval-augmented generation: retrieving relevant information and supplying it to a model to help generate a response.
RAG adds retrieved material to the context used for generation. The application searches an external collection, selects results, and supplies them to the model. Retrieval might use keywords, vectors, or another method. The quality of the answer depends both on finding useful evidence and on using it accurately, so evaluate those steps separately.
Learned numerical representations of data that allow software to compare items by relevant features.
An embedding model turns data into a vector, a list of numbers encoding learned features. Software can compare these representations to find related text, images, or other items. The illustrated map compresses many dimensions into two. What proximity means depends on the embedding model and the similarity measure used.
Retrieving items by comparing their vector representations with a query vector.
Vector search compares a query representation with stored representations and retrieves nearby matches. Similarity measures such as cosine similarity define the comparison. Larger collections often use approximate search to keep retrieval fast. Judge the results against the actual query: a close vector is evidence of similarity, rather than proof that a document is relevant or true.
Combining different retrieval methods, commonly keyword search and vector search, into one search process.
Exact identifiers and broader meaning can both matter in a search. Hybrid retrieval combines methods, commonly keyword and vector search, so each can contribute candidates. The system then merges their results or combines scores. How those signals are balanced influences which documents reach the model, making that combination worth evaluating on real queries.
Reordering an initial set of search results using an additional relevance assessment.
Retrieval produces candidates; reranking assesses their relevance again to improve the order. A reranker often reads the query alongside each candidate's content, spending more effort on a smaller set. This makes the initial shortlist important. A relevant document excluded at retrieval has no chance to move up during reranking.
Model-generated content that is presented as factual but is incorrect or unsupported by the available evidence.
Fluent output can contain invented facts, nonexistent citations, or claims that contradict the source. This is the failure usually described as hallucination. Check factual claims against appropriate evidence, particularly when the output will drive another action. Supplying reference material helps only when the generated answer actually follows what that material supports.
Connecting a model's response to relevant source information or observations that support its claims.
Grounding connects an answer to information that supports it. That information might come from a document, a database, or an observation returned by a tool. Check the relationship between claim and evidence, including whether the source is reliable. A citation attached to an unrelated passage provides a link without providing support.