text classification sentiment analysis
# Text Classification & Sentiment Analysis
## Introduction & Motivation
Text Classification: assign documents to categories. Sentiment Analysis: classify emotional tone. Applications: spam detection, review classification, customer feedback, topic modeling.
Motivation: Automated text understanding; business intelligence.
Applications: Sentiment, topic, spam, intent.
---
## Core Concepts & Theory
### Document Representation
Bag-of-words, TF-IDF, embeddings.
### Feature Extraction
Unigrams, bigrams, character n-grams.
### Classifier
Naive Bayes, SVM, neural networks.
---
## Mathematical Formulation
TF-IDF:
$$ ext{TF-IDF}(t, d) = ext{TF}(t, d) imes \log\frac{N}{N_t}$$
Naive Bayes:
$$P(c | d) \propto P(c) \prod_w P(w | c)$$
Cross-entropy loss:
$$L = -\sum_c y_c \log \hat{y}_c$$
---
## Advanced Theory & Extensions
### Aspect-Based Sentiment
Target-specific sentiments.
### Multi-label Classification
Multiple categories per document.
### Zero-shot Classification
Classify without training examples.
---
## Computational Considerations
Vectorization: O(|V|·|D|) where V=vocabulary, D=docs.
Classification: O(|V|) per document (linear).
Deep Learning: O(batch·seq_len·model_size).
---
## Practical Implementation Strategies
### Preprocessing
Lowercasing, tokenization, stemming.
### Feature Selection
TF-IDF weighting, feature importance.
### Class Balancing
Weighted loss, oversampling.
---
## Benchmark Datasets & Evaluation
Movie Reviews (Stanford Sentiment Treebank): Binary sentiment.
Amazon Reviews: Multi-class.
News Classification: Topic classification.
---
## Key Challenges & Limitations
### Context Sensitivity
Sarcasm, irony; meaning context-dependent.
### Domain Adaptation
Models overfit to training domain.
### Imbalanced Classes
Skewed class distributions.
---
## Hyperparameter Tuning
Learning rate: 1e-3 to 1e-4 for deep models.
Dropout: 0.1-0.3; regularization.
Batch size: 32-128; gradient stability.
---
## Real-World Applications & Case Studies
Product Reviews: E-commerce sentiment.
Social Media: Public opinion analysis.
Customer Service: Feedback categorization.
---
## Integration with Other Methods
Classification + Attention → interpretability.
Classification + Ensemble → robustness.
---
## Summary & Key Takeaways
Text Classification via document representation and sentiment analysis enables automated content categorization through feature extraction and neural classification.
Principles:
1. Representation: embeddings vs. features.
2. Features: n-grams, TF-IDF.
3. Classification: neural vs. classical.
4. Evaluation: accuracy, F1, AUC.
5. Applications: sentiment, topic, spam.
---
---
## Appendix: Practical Labs
### Lab 1: TF-IDF Vectorization
import numpy as np
def tf_idf_vectorize(documents, vocab_size=1000):
"""Compute TF-IDF vectors"""
N = len(documents)
tf_idf = np.zeros((N, vocab_size))
# Term frequencies
for i, doc in enumerate(documents):
words = doc.split()
word_counts = {}
for word in words:
word_id = hash(word) % vocab_size
word_counts[word_id] = word_counts.get(word_id, 0) + 1
# TF
for word_id, count in word_counts.items():
tf = count / max(len(words), 1)
# IDF (simplified: document frequency)
idf = np.log(N / (1 + len([d for d in documents if str(word_id) in d])))
tf_idf[i, word_id] = tf * idf
return tf_idf
# Test
documents = ["the cat sat", "dog ran fast", "cat and dog play"]
tfidf = tf_idf_vectorize(documents, vocab_size=100)
assert tfidf.shape == (3, 100), "TF-IDF shape"
assert np.all(tfidf >= 0), "TF-IDF non-negative"
print("✓ TF-IDF working")
if __name__ == "__main__":
print("Lab 1: TFIDF - PASSED")### Lab 2: Naive Bayes Classifier
import numpy as np
def naive_bayes_sentiment(documents, labels, test_doc):
"""Simple Naive Bayes for sentiment"""
vocab_size = 1000
# Prior probabilities
p_pos = np.sum(labels) / len(labels)
p_neg = 1 - p_pos
# Word probabilities given class
words_pos = np.ones(vocab_size) # Laplace smoothing
words_neg = np.ones(vocab_size)
for doc, label in zip(documents, labels):
words = doc.split()
for word in words:
word_id = hash(word) % vocab_size
if label == 1:
words_pos[word_id] += 1
else:
words_neg[word_id] += 1
# Normalize
words_pos = words_pos / words_pos.sum()
words_neg = words_neg / words_neg.sum()
# Predict
log_prob_pos = np.log(p_pos)
log_prob_neg = np.log(p_neg)
test_words = test_doc.split()
for word in test_words:
word_id = hash(word) % vocab_size
log_prob_pos += np.log(words_pos[word_id])
log_prob_neg += np.log(words_neg[word_id])
return 1 if log_prob_pos > log_prob_neg else 0
# Test
documents = ["great movie", "bad film", "wonderful show", "terrible"]
labels = [1, 0, 1, 0]
test = "great film"
pred = naive_bayes_sentiment(documents, labels, test)
assert pred in [0, 1], "Valid prediction"
print("✓ Naive Bayes working")
if __name__ == "__main__":
print("Lab 2: NaiveBayes - PASSED")### Lab 3: Sentiment Score
import numpy as np
def sentiment_score(logits):
"""Convert logits to sentiment scores"""
# Softmax
exp_logits = np.exp(logits - np.max(logits, axis=1, keepdims=True))
probs = exp_logits / exp_logits.sum(axis=1, keepdims=True)
# Confidence: max probability
confidence = np.max(probs, axis=1)
# Sentiment: probability of positive class
sentiment = probs[:, 1] # Assuming class 1 is positive
return sentiment, confidence
# Test
np.random.seed(42)
logits = np.random.randn(10, 2) # 10 samples, 2 classes
sentiment, confidence = sentiment_score(logits)
assert sentiment.shape == (10,), "Sentiment shape"
assert confidence.shape == (10,), "Confidence shape"
assert np.all((sentiment >= 0) & (sentiment <= 1)), "Valid sentiment"
print("✓ Sentiment score working")
if __name__ == "__main__":
print("Lab 3: SentimentScore - PASSED")### Lab 4: Classification Metrics
import numpy as np
def classification_metrics(y_true, y_pred):
"""Compute precision, recall, F1"""
# Confusion matrix
tp = np.sum((y_pred == 1) & (y_true == 1))
fp = np.sum((y_pred == 1) & (y_true == 0))
tn = np.sum((y_pred == 0) & (y_true == 0))
fn = np.sum((y_pred == 0) & (y_true == 1))
# Metrics
precision = tp / (tp + fp + 1e-8)
recall = tp / (tp + fn + 1e-8)
f1 = 2 * precision * recall / (precision + recall + 1e-8)
accuracy = (tp + tn) / len(y_true)
return accuracy, precision, recall, f1
# Test
y_true = np.array([0, 1, 1, 0, 1, 1, 0, 0])
y_pred = np.array([0, 1, 1, 0, 0, 1, 1, 0])
acc, prec, rec, f1 = classification_metrics(y_true, y_pred)
assert 0 <= acc <= 1, "Accuracy in [0,1]"
assert 0 <= f1 <= 1, "F1 in [0,1]"
print("✓ Classification metrics working")
if __name__ == "__main__":
print("Lab 4: ClassificationMetrics - PASSED")