Poly-Encoder is the retrieval model that uses multiple context vectors per query enabling efficient approximate query-document interactions — Poly-Encoders balance the efficiency of dual-encoders with the interaction capacity of cross-encoders through multiple learnable query context vectors, enabling both scalable retrieval and richer semantic matching than pure dual-encoder systems.

🔬 Core Concept

Poly-Encoder addresses a core trade-off between dual-encoders and cross-encoders: dual-encoders are efficient but capture limited query-document interactions, while cross-encoders model rich interactions but are slow. Poly-Encoders use multiple learned context vectors per query, enabling interaction approximation that's more rich than dual-encoders but faster than cross-encoders.

⚡ Key Characteristics

Combines Scalability with Interaction Richness: Poly-Encoders balance the efficiency of dual-encoders with the interaction capacity of cross-encoders through multiple learnable query context vectors, enabling both scalable retrieval and richer semantic matching.

Instead of one averaged query representation, Poly-Encoders learn multiple query representations capturing different aspects of the information need, then compute interactions between each and document representations.

🔬 Technical Architecture

Poly-Encoders use BERT for encoding queries and documents separately. The innovation is learning multiple context vectors from the query encoding that represent different aspects of the information need. During ranking, each context vector is scored against document representations, and scores are aggregated.

🎯 Use Cases

Enterprise Applications:

• Balanced efficiency-quality retrieval
• Large-scale ranking systems
• Conversational search with multi-axis queries

Research Domains:

• Approximating cross-encoder quality with dual-encoder efficiency
• Multi-aspect query representation
• Scalable ranking methodologies

🚀 Impact & Future Directions

Poly-Encoders demonstrate a middle ground between pure efficiency and pure interaction quality. Emerging research explores learned context vector selection and deeper integration with dense retrieval.