Recommendation Systems Collaborative Filtering Matrix Factorization
# Recommendation Systems: Collaborative Filtering & Matrix Factorization
## Introduction & Motivation
Recommendation systems: predict user preferences. Collaborative filtering: leverage user-item interactions. Matrix factorization: low-rank decomposition. Content-based: item features. Hybrid: combine approaches. Applications: e-commerce, streaming, social media.
Motivation: Users don't know all items; personalized recommendations needed.
Applications: E-commerce, streaming, social platforms.
---
## Core Concepts & Theory
### Collaborative Filtering
User-user, item-item similarity.
### Matrix Factorization
Factorize user-item matrix; latent factors.
### Content-Based Filtering
Item feature similarity.
---
## Mathematical Formulation
Matrix factorization:
$$R \approx U imes V^T$$
where U = user factors, V = item factors.
Prediction:
$$\hat{r}_{ui} = u_i^T v_u$$
Loss function (weighted squared error):
$$L = \sum_{(u,i) \in \mathcal{D}} w_{ui}(r_{ui} - u_i^T v_u)^2 + \lambda(||U||^2 + ||V||^2)$$
---
## Advanced Theory & Extensions
### Deep Learning RS
Neural collaborative filtering; embeddings.
### Factorization Machines
Higher-order interactions; FM.
### Temporal Dynamics
Time-aware recommendations; drifting preferences.
---
## Computational Considerations
Collaborative filtering: O(U·I·K) where K = latent factors.
Matrix factorization: O(|D|·K) SGD per iteration.
Deep learning: O(model_size).
---
## Practical Implementation Strategies
### Negative Sampling
Balance positive and negative examples.
### Cold Start
New users; content-based initialization.
### Evaluation Protocols
K-fold on temporal split; no future leakage.
---
## Benchmark Datasets & Evaluation
MovieLens: Standard benchmark; rating prediction.
Netflix: Large-scale; matrix completion.
LastFM: Music recommendations; implicit feedback.
---
## Key Challenges & Limitations
### Cold Start
New users lack history; limited data.
### Data Sparsity
Most user-item pairs unknown.
### Popularity Bias
Popular items over-recommended.
---
## Hyperparameter Tuning
Latent dimension K: 10-100; tradeoff.
Learning rate: 0.01-0.1; convergence.
Regularization λ: 0.01-0.1; overfitting.
---
## Real-World Applications & Case Studies
Netflix: Movie recommendations; matrix factorization.
Amazon: Product recommendations; collaborative.
Spotify: Music recommendations; hybrid.
---
## Integration with Other Methods
RS + Deep Learning → neural RS.
RS + Content → hybrid systems.
---
## Summary & Key Takeaways
Recommendation systems via collaborative filtering and matrix factorization enable personalized predictions through user-item interaction factorization.
Principles:
1. Collaborative: user-item interactions.
2. Matrix factorization: low-rank decomposition.
3. Content-based: item features.
4. Hybrid: combine approaches.
5. Cold start: address via content.
---
---
## Appendix: Practical Labs
### Lab 1: Collaborative Filtering
import numpy as np
def collaborative_filtering_predict(user_idx, item_idx, R_factorized):
"""Predict rating via user-item similarity"""
U, V = R_factorized
user_vec = U[user_idx]
item_vec = V[item_idx]
# Dot product
prediction = np.dot(user_vec, item_vec)
return prediction
# Test
np.random.seed(42)
U = np.random.randn(100, 20)
V = np.random.randn(500, 20)
pred = collaborative_filtering_predict(5, 10, (U, V))
assert np.isfinite(pred), "Prediction finite"
print("✓ Collaborative filtering working")
if __name__ == "__main__":
print("Lab 1: CollaborativeFiltering - PASSED")### Lab 2: Matrix Factorization
import numpy as np
def matrix_factorization_update(R, U, V, learning_rate=0.01, reg=0.01):
"""SGD update for matrix factorization"""
# Sample one (user, item) pair
u_idx, i_idx = np.random.randint(0, R.shape[0]), np.random.randint(0, R.shape[1])
if R[u_idx, i_idx] == 0: # Skip unobserved
return U, V
# Prediction error
error = R[u_idx, i_idx] - np.dot(U[u_idx], V[i_idx])
# Update
grad_u = -2 * error * V[i_idx] + 2 * reg * U[u_idx]
grad_v = -2 * error * U[u_idx] + 2 * reg * V[i_idx]
U[u_idx] -= learning_rate * grad_u
V[i_idx] -= learning_rate * grad_v
return U, V
# Test
np.random.seed(42)
R = np.random.rand(100, 500)
U = np.random.randn(100, 20)
V = np.random.randn(500, 20)
U, V = matrix_factorization_update(R, U, V)
assert U.shape == (100, 20), "U shape"
assert V.shape == (500, 20), "V shape"
print("✓ Matrix factorization update working")
if __name__ == "__main__":
print("Lab 2: MatrixFactorization - PASSED")### Lab 3: Cold Start Problem
import numpy as np
def cold_start_recommendation(item_features, new_user_profile, k=5):
"""Content-based recommendation for new users"""
# Compute similarity
similarities = np.dot(item_features, new_user_profile)
# Top-k
top_indices = np.argsort(-similarities)[:k]
top_scores = similarities[top_indices]
return top_indices, top_scores
# Test
np.random.seed(42)
item_features = np.random.randn(500, 50)
user_profile = np.random.randn(50)
items, scores = cold_start_recommendation(item_features, user_profile, k=5)
assert len(items) == 5, "Top-5 items"
assert len(scores) == 5, "Scores for each"
print("✓ Cold start recommendation working")
if __name__ == "__main__":
print("Lab 3: ColdStart - PASSED")### Lab 4: Recommendation Metrics
import numpy as np
def compute_precision_recall_k(predictions, ground_truth, k=10):
"""Compute precision@k and recall@k"""
pred_set = set(predictions[:k])
truth_set = set(ground_truth)
hits = len(pred_set & truth_set)
precision = hits / k if k > 0 else 0
recall = hits / len(truth_set) if len(truth_set) > 0 else 0
return precision, recall
# Test
np.random.seed(42)
predictions = np.random.randint(0, 100, 20)
ground_truth = np.random.randint(0, 100, 15)
prec, rec = compute_precision_recall_k(predictions, ground_truth, k=10)
assert 0 <= prec <= 1, "Precision in [0,1]"
assert 0 <= rec <= 1, "Recall in [0,1]"
print("✓ Recommendation metrics working")
if __name__ == "__main__":
print("Lab 4: RecommendationMetrics - PASSED")