mixture of agents (moa)

Mixture of Agents (MoA) routes queries to specialized agents based on task type, combining expert capabilities. **Architecture**: Router/gate model classifies query → selects appropriate specialist(s) → aggregates responses. **Similarity to MoE**: Like Mixture of Experts but at agent level rather than neural network layer. **Routing strategies**: Hard routing (one agent), soft routing (weighted combination), top-k (multiple specialists), learned routing function. **Specialist types**: Domain experts (coding, writing, analysis), task experts (search, calculation, planning), format experts (JSON, markdown, code). **Router training**: Classification on task types, learned from interaction data, or rule-based heuristics. **Benefits**: Specialized agents outperform generalists, efficient resource use, modular updates. **Implementation**: Query embedding → router model → agent selection → execution → response merging. **Aggregation**: Single response pass-through, synthesis across specialists, quality-based selection. **Frameworks**: LangChain routers, custom MoA implementations. **Challenges**: Routing accuracy, handling ambiguous queries, load balancing, maintaining consistency. **Optimization**: Cache routing decisions, batch similar queries, precompute agent capabilities.