transductive learning

**Transductive learning** in few-shot learning allows the model to leverage information about the **structure of the entire query (test) set** during prediction, rather than classifying each query example independently. It exploits the **distributional properties** of the test batch for improved accuracy. **Inductive vs. Transductive** - **Inductive**: Process each query example **independently** — prediction for one query doesn't depend on other queries. Standard approach. - **Transductive**: Process all query examples **jointly** — the model can use relationships, clusters, and distributions within the query batch to inform predictions. **Why Transductive Helps** - **Cluster Structure**: Query examples from the same class tend to cluster in feature space. The model can identify these clusters even without labels. - **Distribution Information**: The query set reveals the marginal distribution of test data — useful for calibrating decision boundaries. - **Mutual Information**: One query example's classification can inform others — if two queries are very similar, they likely share a class. - **Typical Accuracy Improvement**: **2–5%** over inductive methods on standard benchmarks. **Transductive Approaches** - **Label Propagation**: Construct a **graph** connecting support and query examples by feature similarity. Propagate labels from support nodes to query nodes through the graph using iterative message passing. - **Transductive Fine-Tuning**: Adapt model parameters using **both** labeled support AND unlabeled query examples. Use entropy minimization on query predictions as an unsupervised signal. - **Sinkhorn-Based Methods**: Enforce **balanced class assignments** across the query set — if there are 5 classes and 75 queries, encourage roughly 15 assignments per class using the Sinkhorn-Knopp algorithm. - **Expectation-Maximization (EM)**: Iteratively assign soft labels to query examples (E-step) and update class representations (M-step) — alternating until convergence. - **Transductive Prototype Refinement**: Start with prototypes from support examples, then iteratively **refine prototypes** using high-confidence query assignments. **Graph-Based Methods** - **GNN for Few-Shot**: Build a graph with support and query examples as nodes. Use **Graph Neural Networks** to propagate information — node features are updated based on neighbors, allowing label information to flow from support to query nodes. - **Edge-Labeling GNNs**: Predict edge labels (same-class or different-class) for all pairs of nodes in the graph. **Assumptions and Limitations** - **Batch Availability**: Requires access to the full query batch at once — doesn't work for **streaming/online** scenarios where examples arrive one at a time. - **Class Coverage**: Assumes query set contains examples from **all support classes** — if a class is absent from the query batch, methods like Sinkhorn can malfunction. - **Equal Representation**: Some methods assume roughly equal class distribution in queries — violated in imbalanced test scenarios. - **Computational Cost**: Joint processing of all queries is more expensive than independent classification. Transductive learning is a **powerful technique** for few-shot learning when the full test batch is available — it extracts additional signal from the unlabeled test data that purely inductive methods waste.