Iterative Retrieval

A multi-stage retrieval strategy in which a large corpus is progressively narrowed down through successive filtering passes — for example, reducing a trillion-token dataset to millions of relevant tokens — rather than attempting to process the entire corpus at once.

Core Argument: Why Staged Retrieval Is Necessary

Both Jeff Dean and Kuba Rogut (Turbopuffer) argue that iterative retrieval is essential regardless of how large context windows become. The key insight, as Rogut frames it, is that "you don't need a trillion at once, you need the right million" — meaning the goal is precision of selection, not brute-force inclusion. (source)

Dean's position, as conveyed by Rogut, is that "what you really need is stage retrieval, like a lightweight mechanism to narrow down these trillion tokens into essentially millions at a time." (source) This framing positions iterative retrieval as a practical engineering necessity: feeding a trillion tokens into any model in a single pass is computationally infeasible, so lightweight early-stage filters must do coarse selection before more expensive downstream processing.

Relationship to the "RAG Is Dead" Debate

This concept emerges in the context of debates about whether RAG pipelines become obsolete as context windows grow. The staged-retrieval argument is a direct rebuttal: even an arbitrarily large context window doesn't eliminate the need for retrieval, because the problem is not window size but the ratio of useful to useless tokens. Iterative retrieval addresses this ratio problem at each stage, making it complementary to — rather than replaced by — large context models.

Points of Disagreement

No speaker in the available material directly disputes iterative retrieval. The advocacy from both Dean and Rogut is presented without a recorded counterargument in this context.