bi-encoder

A **bi-encoder** is a neural retrieval architecture that uses **separate encoder networks** to independently encode the **query** and **document** into dense vector representations. Similarity is then computed by comparing these vectors, typically using **cosine similarity** or **dot product**. **How Bi-Encoders Work** - **Document Encoding (Offline)**: All documents in the corpus are pre-encoded into vectors and stored in an **index** (typically a vector database). This is a one-time cost. - **Query Encoding (Online)**: At search time, the query is encoded into a vector using the query encoder. - **Retrieval**: The query vector is compared against all document vectors using **approximate nearest neighbor (ANN)** search, returning the most similar documents in milliseconds. **Advantages** - **Speed**: Since documents are pre-encoded, retrieval only requires encoding the query and performing a fast vector lookup — **sub-millisecond** latency for millions of documents. - **Scalability**: Works efficiently with corpora of **billions of documents** using ANN indexes like **HNSW** or **IVF**. - **Independence**: Query and document encoders can be based on different model architectures if needed. **Bi-Encoder vs. Cross-Encoder** - **Bi-Encoder**: Fast but less accurate — query and document never "see" each other during encoding, so fine-grained token-level interactions are missed. - **Cross-Encoder**: Processes query+document together through a single model, capturing rich interactions, but is **100–1000× slower** since every candidate must be scored individually. - **Common Pattern**: Use a bi-encoder for **first-stage retrieval** (fast, broad recall) followed by a cross-encoder for **reranking** the top results (slow, high precision). **Popular Bi-Encoder Models** - **Sentence-BERT (SBERT)** - **E5** and **BGE** families - **GTE** (General Text Embeddings) - **Cohere Embed** and **OpenAI text-embedding-3** Bi-encoders are the backbone of modern **semantic search** and **RAG retrieval** systems.