learning to rank ranking systems information retrieval
# Learning to Rank: Ranking Systems & Information Retrieval
## Introduction & Motivation
Learning to Rank: predict item order. Pointwise, pairwise, listwise approaches. Applications: search results, recommendations, recommendations.
Motivation: Optimize ranking quality; relevance.
Applications: Search, recommendations, IR.
---
## Core Concepts & Theory
### Pointwise
Predict relevance score per item.
### Pairwise
Learn item preference order.
### Listwise
Optimize full ranking quality.
---
## Mathematical Formulation
Pointwise loss:
$$L = \sum_i (y_i - f(x_i))^2$$
Pairwise loss:
$$L = \sum_{i > j} \max(0, 1 + f(x_j) - f(x_i))^{y_i > y_j}$$
NDCG:
$$ ext{NDCG} = \frac{ ext{DCG}}{ ext{Ideal DCG}}$$
---
## Advanced Theory & Extensions
### LambdaMART
Gradient boosting for ranking.
### RankNet
Neural ranking; pairwise.
### ListNet
Listwise neural approach.
---
## Computational Considerations
Pointwise: O(N).
Pairwise: O(N²).
Listwise: O(N log N).
---
## Practical Implementation Strategies
### Feature Engineering
Query-document relevance.
### Metric Optimization
Optimize NDCG directly.
### Online Learning
Update from user feedback.
---
## Benchmark Datasets & Evaluation
LETOR: Learning to rank benchmark.
MSLR: Large-scale ranking.
Yahoo Learning to Rank: Competition data.
---
## Key Challenges & Limitations
### Computational Cost
Pairwise quadratic.
### Ranking Cascade
Positions affect relevance.
### Position Bias
User click bias.
---
## Hyperparameter Tuning
Learning rate: 0.01-0.1.
Tree depth: 5-10.
Sample weight: Balanced.
---
## Summary & Key Takeaways
Learning to Rank via pointwise, pairwise, listwise enables information retrieval optimization through relevance prediction.
Principles:
1. Ranking task: order optimization.
2. Pointwise: score per item.
3. Pairwise: preference learning.
4. Listwise: full ranking.
5. NDCG: ranking metric.
---
---
## Appendix: Practical Labs
### Lab 1: Relevance Scoring
import numpy as np
def pointwise_ranking_score(features, weights):
"""Compute relevance scores"""
scores = features @ weights
return scores
# Test
np.random.seed(42)
features = np.random.randn(100, 20)
weights = np.random.randn(20)
scores = pointwise_ranking_score(features, weights)
assert scores.shape == (100,), "Score shape"
print("✓ Pointwise scoring working")
if __name__ == "__main__":
print("Lab 1: PointwiseScoring - PASSED")### Lab 2: Pairwise Loss
import numpy as np
def pairwise_loss(scores, labels):
"""Pairwise ranking loss"""
loss = 0
count = 0
for i in range(len(scores)):
for j in range(i + 1, len(scores)):
if labels[i] > labels[j]:
loss += np.log(1 + np.exp(-(scores[i] - scores[j])))
count += 1
return loss / (count + 1e-8)
# Test
np.random.seed(42)
scores = np.random.randn(10)
labels = np.random.rand(10)
loss = pairwise_loss(scores, labels)
assert np.isfinite(loss), "Loss finite"
print("✓ Pairwise loss working")
if __name__ == "__main__":
print("Lab 2: PairwiseLoss - PASSED")### Lab 3: NDCG Metric
import numpy as np
def compute_ndcg(predicted_ranks, true_labels, k=10):
"""Compute Normalized Discounted Cumulative Gain"""
# DCG
top_k = np.argsort(-predicted_ranks)[:k]
dcg = 0
for i, idx in enumerate(top_k):
dcg += true_labels[idx] / np.log2(i + 2)
# Ideal DCG
ideal_labels = np.sort(-true_labels)[:k]
idcg = 0
for i, label in enumerate(ideal_labels):
idcg += label / np.log2(i + 2)
ndcg = dcg / (idcg + 1e-8)
return ndcg
# Test
np.random.seed(42)
scores = np.random.rand(100)
labels = np.random.rand(100)
ndcg = compute_ndcg(scores, labels, k=10)
assert 0 <= ndcg <= 1, "NDCG in [0,1]"
print("✓ NDCG working")
if __name__ == "__main__":
print("Lab 3: NDCG - PASSED")### Lab 4: Ranking Evaluation
import numpy as np
def mean_reciprocal_rank(predictions, true_relevant, k=10):
"""Compute Mean Reciprocal Rank"""
ranked_indices = np.argsort(-predictions)[:k]
for i, idx in enumerate(ranked_indices):
if true_relevant[idx]:
return 1 / (i + 1)
return 0
# Test
np.random.seed(42)
scores = np.random.rand(100)
relevant = np.random.rand(100) > 0.8
mrr = mean_reciprocal_rank(scores, relevant)
assert 0 <= mrr <= 1, "MRR in [0,1]"
print("✓ MRR working")
if __name__ == "__main__":
print("Lab 4: MRR - PASSED")