Dual-Encoder

A dual-encoder (also called a bi-encoder) is a retrieval architecture in which two encoders — typically neural networks sharing weights, or two separate encoders — are trained to map queries and documents into a shared vector space. At indexing time, every document is independently encoded into a vector. At query time, the query is encoded into the same space, and relevance is computed as a cheap similarity (most often cosine or inner product) between the query vector and every document vector. The architecture is the dominant first-stage retrieval pattern in modern search: it is the basis of dense passage retrieval (DPR), Sentence-BERT, most embedding-based RAG systems, and the modern embedding APIs offered by OpenAI, Cohere, Voyage, and others. The defining feature is that query and document never attend to each other — all the work is in producing good independent vectors.

Mechanically, a dual-encoder is trained to place relevant (query, document) pairs close together in vector space and irrelevant pairs far apart. The standard training objective is contrastive: given a query and a set of candidate documents (one or more positive, several negatives), the encoder is trained to maximize similarity with positives and minimize similarity with negatives. Negative selection is the load-bearing trick: in-batch negatives, hard negatives (documents that look relevant but are not), and mined negatives from the index are the three main sources. The encoders are typically BERT-family or a domain-specific transformer; the vector dimension is in the hundreds to low thousands; and the similarity metric is normalized so that cosine and inner product are equivalent. At inference, the dominant cost is the document encoding (one pass per document at indexing time) plus a fast nearest-neighbor search over the resulting vectors using an approximate nearest neighbor index.

The advantage of the dual-encoder frame is that it explains why dense retrieval scales: once vectors are indexed, retrieval is a nearest-neighbor problem that scales to billions of vectors with mature ANN infrastructure. The frame also explains the cost — query and document cannot interact during encoding, which means fine-grained relevance signals (does this specific document answer this specific question?) are weaker than what a cross-encoder can capture. The cost of the frame is that 'dual-encoder' is sometimes used loosely for architectures that have more structure (multi-vector, late-interaction, sparse learned vectors), and the line between 'true' dual-encoder and 'dual with interaction' is fuzzy. The frame also understates how much engineering goes into the negative set — choosing negatives is often the difference between a mediocre and a great dual-encoder, and that work is not visible in the architecture name.

Open questions include how to combine dual-encoder speed with cross-encoder accuracy without the cost tradeoffs of late-interaction, how to train dual-encoders without expensive human relevance labels (synthetic supervision from LLMs is one approach), and how to handle long documents when encoder input length is limited (chunking strategies are the practical answer, but they introduce their own artifacts). The deeper question — what the right interaction pattern is between query and document in a world where vector search is also increasingly used for recommendation, image search, code search, and other modalities — is genuinely open.