Skip to content

Commit 55694f1

Browse files
committed
chore: clean up repository
1 parent d891fe7 commit 55694f1

5 files changed

Lines changed: 1321 additions & 5 deletions

File tree

.gitignore

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -64,10 +64,5 @@ personal/
6464
# Test scripts in root
6565
test_*.sh
6666

67-
# Generated results (metrics in README)
68-
results/
69-
7067
# Dev-only scripts
71-
scripts/reset.sh
7268
scripts/cleanup.sh
73-
scripts/generate_report.sh
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Sentiment Analysis: Final Comprehensive Evaluation Report
2+
3+
**Project**: Cross-Domain Sentiment Classification
4+
**Date**: January 30, 2026
5+
**Author**: Victoria Alabi
6+
**Generated**: Auto-generated from model metadata (do not edit manually)
7+
8+
---
9+
10+
## Executive Summary
11+
12+
This report summarizes the training and evaluation of sentiment analysis models across multiple algorithms and domains.
13+
14+
### Production Model
15+
16+
**Algorithm**: SVM
17+
**Training Dataset**: amazon_polarity (40000 samples)
18+
**Model Size**: 15.9 MB
19+
20+
| Metric | Value |
21+
|--------|-------|
22+
| Test Accuracy | 89.6% |
23+
| Test F1 | 0.896 |
24+
| Test Precision | 0.896 |
25+
| Test Recall | 0.896 |
26+
| ROC-AUC | 0.957 |
27+
| Cross-Domain Avg | 88.0% |
28+
29+
### Production Model Confusion Matrix
30+
31+
| | Predicted Negative | Predicted Positive |
32+
|---|---|---|
33+
| **Actual Negative** | 4478 (TN) | 525 (FP) |
34+
| **Actual Positive** | 519 (FN) | 4478 (TP) |
35+
36+
### Best Generalizing Model
37+
38+
**Model**: svm-amazon_polarity
39+
**Cross-Domain Average Accuracy**: 88.0%
40+
41+
---
42+
43+
## Part 1: Model Comparison (All 12 Experiments)
44+
45+
| Algorithm | Dataset | Accuracy | F1 | Precision | Recall | Training Time |
46+
|-----------|---------|----------|-----|-----------|--------|---------------|
47+
| SVM | amazon_polarity | 89.2% | 0.892 | 0.893 | 0.892 | 97m 22s |
48+
| SVM | imdb_50k | 89.1% | 0.891 | 0.891 | 0.891 | 245m 56s |
49+
| SVM | yelp | 94.0% | 0.940 | 0.940 | 0.940 | 22m 32s |
50+
| LOGISTIC_REGRESSION | amazon_polarity | 84.1% | 0.841 | 0.841 | 0.841 | 53m 28s |
51+
| LOGISTIC_REGRESSION | imdb_50k | 85.5% | 0.855 | 0.855 | 0.855 | 87m 57s |
52+
| LOGISTIC_REGRESSION | yelp | 92.6% | 0.926 | 0.926 | 0.926 | 24m 48s |
53+
| RANDOM_FOREST | amazon_polarity | 86.6% | 0.866 | 0.866 | 0.866 | 101m 40s |
54+
| RANDOM_FOREST | imdb_50k | 86.6% | 0.866 | 0.866 | 0.866 | 223m 51s |
55+
| RANDOM_FOREST | yelp | 92.0% | 0.920 | 0.921 | 0.920 | 37m 47s |
56+
| NAIVE_BAYES | amazon_polarity | 79.2% | 0.792 | 0.796 | 0.792 | 59m 38s |
57+
| NAIVE_BAYES | imdb_50k | 82.9% | 0.829 | 0.830 | 0.829 | 115m 9s |
58+
| NAIVE_BAYES | yelp | 81.3% | 0.813 | 0.816 | 0.813 | 10m 39s |
59+
60+
---
61+
62+
## Part 2: Cross-Domain Evaluation
63+
64+
Each model was evaluated on all three test domains. Asterisk (*) indicates in-domain evaluation.
65+
66+
#### SVM
67+
68+
| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg |
69+
|--------------|-----------|-------------|-----------|------------------|
70+
| imdb 50k | 89.1% * | 81.9% | 84.9% | 83.4% |
71+
| amazon polarity | 85.2% | 89.2% * | 90.9% | 88.0% |
72+
| yelp | 78.5% | 82.0% | 94.0% * | 80.3% |
73+
74+
#### LOGISTIC REGRESSION
75+
76+
| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg |
77+
|--------------|-----------|-------------|-----------|------------------|
78+
| imdb 50k | 85.5% * | 76.8% | 79.3% | 78.1% |
79+
| amazon polarity | 80.4% | 84.1% * | 84.7% | 82.6% |
80+
| yelp | 75.8% | 77.8% | 92.6% * | 76.8% |
81+
82+
#### RANDOM FOREST
83+
84+
| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg |
85+
|--------------|-----------|-------------|-----------|------------------|
86+
| imdb 50k | 86.6% * | 80.2% | 83.1% | 81.6% |
87+
| amazon polarity | 79.6% | 86.6% * | 89.6% | 84.6% |
88+
| yelp | 71.5% | 81.3% | 92.0% * | 76.4% |
89+
90+
#### NAIVE BAYES
91+
92+
| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg |
93+
|--------------|-----------|-------------|-----------|------------------|
94+
| imdb 50k | 82.9% * | 73.5% | 79.1% | 76.3% |
95+
| amazon polarity | 69.4% | 79.2% * | 81.3% | 75.4% |
96+
| yelp | 59.8% | 70.9% | 81.3% * | 65.3% |
97+
98+
**Legend**: * = in-domain evaluation
99+
100+
---
101+
102+
## Part 3: Reproducibility
103+
104+
All results can be reproduced via:
105+
106+
```bash
107+
# Prepare immutable data splits (run once)
108+
./scripts/prepare_data.sh
109+
110+
# Train all models
111+
./scripts/train_all_models.sh
112+
113+
# Cross-domain evaluation
114+
./scripts/evaluate_cross_domain.sh
115+
116+
# Regenerate this report
117+
./scripts/generate_report.sh
118+
```
119+
120+
### Model Artifacts
121+
122+
| File | Algorithm | Dataset | Size |
123+
|------|-----------|---------|------|
124+
| amazon_polarity_logistic_regression_model.ser | LOGISTIC_REGRESSION | amazon_polarity | 14.4 MB |
125+
| imdb_50k_logistic_regression_model.ser | LOGISTIC_REGRESSION | imdb_50k | 30.1 MB |
126+
| yelp_logistic_regression_model.ser | LOGISTIC_REGRESSION | yelp | 9.7 MB |
127+
| amazon_polarity_naive_bayes_model.ser | NAIVE_BAYES | amazon_polarity | 14.5 MB |
128+
| imdb_50k_naive_bayes_model.ser | NAIVE_BAYES | imdb_50k | 30.3 MB |
129+
| yelp_naive_bayes_model.ser | NAIVE_BAYES | yelp | 9.9 MB |
130+
| sentiment_model.ser | SVM | amazon_polarity | 15.9 MB |
131+
| amazon_polarity_random_forest_model.ser | RANDOM_FOREST | amazon_polarity | 194.6 MB |
132+
| imdb_50k_random_forest_model.ser | RANDOM_FOREST | imdb_50k | 193.7 MB |
133+
| yelp_random_forest_model.ser | RANDOM_FOREST | yelp | 91.9 MB |
134+
| amazon_polarity_svm_model.ser | SVM | amazon_polarity | 15.9 MB |
135+
| imdb_50k_svm_model.ser | SVM | imdb_50k | 31.6 MB |
136+
| yelp_svm_model.ser | SVM | yelp | 10.3 MB |
137+
138+
---
139+
140+
## Metadata
141+
142+
- **Report Generated**: 2026-01-30T20:40:18Z
143+
- **Git Commit**: 6d7f5e5
144+
- **Java Version**: 24.0.1
145+

0 commit comments

Comments
 (0)