poly-encoder

**Poly-Encoder** is the retrieval model that uses multiple context vectors per query enabling efficient approximate query-document interactions — Poly-Encoders balance the efficiency of dual-encoders with the interaction capacity of cross-encoders through multiple learnable query context vectors, enabling both scalable retrieval and richer semantic matching than pure dual-encoder systems. --- ## 🔬 Core Concept Poly-Encoder addresses a core trade-off between dual-encoders and cross-encoders: dual-encoders are efficient but capture limited query-document interactions, while cross-encoders model rich interactions but are slow. Poly-Encoders use multiple learned context vectors per query, enabling interaction approximation that's more rich than dual-encoders but faster than cross-encoders. | Aspect | Detail | |--------|--------| | **Type** | Poly-Encoder is a retrieval model | | **Key Innovation** | Multiple context vectors for approximate interactions | | **Primary Use** | Balanced efficiency and interaction modeling | --- ## ⚡ Key Characteristics **Combines Scalability with Interaction Richness**: Poly-Encoders balance the efficiency of dual-encoders with the interaction capacity of cross-encoders through multiple learnable query context vectors, enabling both scalable retrieval and richer semantic matching. Instead of one averaged query representation, Poly-Encoders learn multiple query representations capturing different aspects of the information need, then compute interactions between each and document representations. --- ## 🔬 Technical Architecture Poly-Encoders use BERT for encoding queries and documents separately. The innovation is learning multiple context vectors from the query encoding that represent different aspects of the information need. During ranking, each context vector is scored against document representations, and scores are aggregated. | Component | Feature | |-----------|--------| | **Query Encoding** | BERT encoder producing sequence of tokens | | **Context Vectors** | Learned aggregate representations of query aspects | | **Document Encoding** | Independent BERT encoder | | **Interaction** | Multiple context-document interactions | --- ## 🎯 Use Cases **Enterprise Applications**: - Balanced efficiency-quality retrieval - Large-scale ranking systems - Conversational search with multi-axis queries **Research Domains**: - Approximating cross-encoder quality with dual-encoder efficiency - Multi-aspect query representation - Scalable ranking methodologies --- ## 🚀 Impact & Future Directions Poly-Encoders demonstrate a middle ground between pure efficiency and pure interaction quality. Emerging research explores learned context vector selection and deeper integration with dense retrieval.