Ranking Learning to Rank

# Ranking & Learning to Rank

## Introduction & Motivation

Learning to Rank: optimize ranking of items. Pointwise, pairwise, listwise losses. Applications: information retrieval, recommendation, e-commerce.

Motivation: Learn ranking functions from labeled data.

Applications: Search results ranking, product ranking, recommendation ranking.

---

## Core Concepts & Theory

### Pointwise Approach

Predict relevance scores.

### Pairwise Approach

Optimize preference pairs.

### Listwise Approach

Optimize ranking metrics directly.

### Ranking Metrics

NDCG, MRR, MAP evaluation.

---

## Mathematical Formulation

Pointwise Loss:
$$L = \sum_i \ell(f(x_i), y_i)$$

Pairwise Loss:
$$L = \sum_{(i,j)} \ell(f(x_i) - f(x_j), y_i > y_j)$$

NDCG:
$$ ext{NDCG}@k = \frac{1}{Z_k} \sum_{i=1}^k \frac{2^{rel_i} - 1}{\log_2(i+1)}$$

---

## Advanced Theory & Extensions

### LambdaRank

Gradient approximation for ranking metrics.

### Neural Ranking

Deep neural networks for ranking.

### Cross-Encoder Architecture

Joint relevance scoring.

---

## Computational Considerations

Pointwise: O(n).

Pairwise: O(n²).

Listwise: O(n log n).

---

## Practical Implementation Strategies

### Batch Sampling

Efficient pair/list sampling.

### Metric Approximation

Differentiable metric surrogates.

### Position Bias

Correct for position bias in labels.

---

## Benchmark Datasets & Evaluation

MS MARCO: Large-scale relevance dataset.

TREC: Standard IR evaluation.

WebTrack: TREC web rankings.

---

## Key Challenges & Limitations

### Metric Optimization

NDCG non-differentiability.

### Position Bias

Training data bias.

### Cold Start

Ranking for new items.

---

## Hyperparameter Tuning

Learning rate: 0.001-0.1.

Batch size: 32-256.

Network depth: 2-4 layers.

---

## Real-World Applications & Case Studies

Web Search: Search result ranking.

E-commerce: Product ranking.

Recommendation: Ranked recommendations.

---

## Integration with Other Methods

Ranking + embedding for dense retrieval; + feature engineering for sparse signals.

---

## Summary & Key Takeaways

Learning to Rank optimizes ranking functions for information retrieval.

Principles:
1. Pointwise: Relevance scoring.
2. Pairwise: Preference optimization.
3. Listwise: Metric optimization.
4. NDCG: Position-aware metrics.
5. Bias correction: Label debiasing.

---

## Appendix: Practical Labs

### Lab 1: NDCG Computation

import numpy as np

def compute_ndcg(relevances, k=10):
 """Compute NDCG@k"""
 ideal_relevances = sorted(relevances, reverse=True)[:k]
 sorted_relevances = sorted(relevances, reverse=True)[:k]
 
 dcg = sum((2**rel - 1) / np.log2(i+2) for i, rel in enumerate(sorted_relevances))
 idcg = sum((2**rel - 1) / np.log2(i+2) for i, rel in enumerate(ideal_relevances))
 
 ndcg = dcg / (idcg + 1e-8)
 return ndcg

relevances = [3, 2, 3, 0, 1, 2, 0, 0, 2]
ndcg = compute_ndcg(relevances, k=5)
assert 0 <= ndcg <= 1, "NDCG in valid range"
print("✓ NDCG computation working")

### Lab 2: MRR Computation

import numpy as np

def compute_mrr(relevances, threshold=1):
 """Compute Mean Reciprocal Rank"""
 for i, rel in enumerate(relevances):
 if rel >= threshold:
 return 1.0 / (i + 1)
 return 0.0

relevances = [0, 0, 2, 1, 3]
mrr = compute_mrr(relevances, threshold=1)
assert 0 <= mrr <= 1, "MRR in valid range"
print("✓ MRR computation working")

### Lab 3: Pairwise Loss

import numpy as np

def pairwise_ranking_loss(scores, labels):
 """Compute pairwise ranking loss"""
 loss = 0
 count = 0
 for i in range(len(scores)):
 for j in range(len(scores)):
 if labels[i] > labels[j]:
 loss += max(0, 1 + scores[j] - scores[i])
 count += 1
 return loss / (count + 1e-8)

np.random.seed(42)
scores = np.array([0.5, 0.3, 0.8])
labels = np.array([3, 1, 2])
loss = pairwise_ranking_loss(scores, labels)
assert loss >= 0, "Loss non-negative"
print("✓ Pairwise ranking loss working")

### Lab 4: MAP Computation

import numpy as np

def compute_map(relevances, k=10):
 """Compute Mean Average Precision"""
 sorted_rel = sorted(relevances, reverse=True)[:k]
 precisions = []
 relevant_count = 0
 
 for i, rel in enumerate(sorted_rel):
 if rel >= 1:
 relevant_count += 1
 precision = relevant_count / (i + 1)
 precisions.append(precision)
 
 map_score = sum(precisions) / (relevant_count + 1e-8)
 return map_score

relevances = [1, 0, 1, 1, 0, 1, 0]
map_score = compute_map(relevances)
assert 0 <= map_score <= 1, "MAP in valid range"
print("✓ MAP computation working")

---