Agentic Retrieval

Agentic retrieval is the architectural pattern in which an LLM-based agent decides what to search for and when to stop across multiple tool calls, replacing the single-shot query of a traditional RAG pipeline with an iterative loop of planning, retrieval, and re-planning. The pattern crystallised as the dominant RAG-v2 shape in 2025 as multi-hop evaluation suites began to penalise one-shot retrieval: agents that retrieved once per turn routinely failed on compositional or incomplete-information questions, while agents that decomposed the question, ran multiple narrow searches, scored their own partial evidence, and re-queried based on what they had started to learn raised answer quality materially. Production systems typically pair agentic retrieval with a reranker, a per-tool-result memory buffer, and an explicit stop-rule (evidence-converged, max-budget-reached, or confidence-threshold met).

The cost shape is the lever and the trap. Each retrieval turn adds an LLM call, a reranker call, and a search call; a typical multi-turn agentic-retrieval request takes >10 seconds and ~$0.03 end-to-end on a frontier model — about an order of magnitude more than a single-shot RAG on the same corpus. The cost scales with the model's tendency to over-search, so systems instrument per-run call counts and trim the loop with evidence-thresholds or hard budgets. The same dynamic is why small open-models, once post-trained on the search tool, have started to beat frontier APIs on retrieval eval specifically: at 100× cheaper inference, the cost-multiple of a multi-hop loop collapses back into budget. The pattern is sometimes called adaptive retrieval (academic literature, Anthropic contextual retrieval) or iterative RAG (LlamaIndex era); agentic retrieval has now won as the production name because the dominant formulation is an agent explicitly planning, not a fixed pipeline re-running.

The closest existing vocab entries: RAG (Retrieval-Augmented Generation) — the one-shot parent pattern that agentic retrieval is the iterative successor of, contextual retrieval — Anthropic's 2024 enhancement of the retrieval side independent of the agent loop, reranking — the second-pass relevance scoring step that the agent almost always calls between retrievals, agent harness — the runtime that hosts the loop and the tool calls, tool call — the primitive the loop is built on. The architectural cousins to know apart: agentic search (broader — includes web search workflows, of which retrieval is one subtype), active retrieval (term occasionally used in academic settings to mean retrieval with a learned evidence-decision function — narrower and older), and iterative RAG (legacy term from 2023 — same shape, pre-agent framing). The benchmark for measuring the pattern's value is the same as for any agent — answer quality on multi-hop QA against a fixed corpus, minus call-budget or latency penalty.

Tradeoffs and limits worth flagging. The pattern is more capable than one-shot RAG on the question types that previously failed (multi-hop, partial-information, contradiction-resolution) but strictly worse on the question types one-shot was already good at (single-fact lookup, high-precision short answer) — the extra loops are pure overhead in those cases. Production teams that ship agentic retrieval without a query-routing layer pay the multi-hop cost on every question including the easy ones. The pattern is also exposure-prone: each retrieval turn is a fresh chance for a poisoned source, a model that follows a malicious search result, or an agent that loses the thread of the original question across a long loop. The mitigations — query routers, evidence-trust scoring, per-step tool-call logging — are themselves additional infrastructure, which is why agentic retrieval is disproportionately useful at the enterprise scale where the corpus is large and the question volume is high, and disproportionately expensive at the small-team scale where a one-shot RAG is already saturating answer quality.