|
| 1 | +# Sentiment Analysis: Final Comprehensive Evaluation Report |
| 2 | + |
| 3 | +**Project**: Cross-Domain Sentiment Classification |
| 4 | +**Date**: January 30, 2026 |
| 5 | +**Author**: Victoria Alabi |
| 6 | +**Generated**: Auto-generated from model metadata (do not edit manually) |
| 7 | + |
| 8 | +--- |
| 9 | + |
| 10 | +## Executive Summary |
| 11 | + |
| 12 | +This report summarizes the training and evaluation of sentiment analysis models across multiple algorithms and domains. |
| 13 | + |
| 14 | +### Production Model |
| 15 | + |
| 16 | +**Algorithm**: SVM |
| 17 | +**Training Dataset**: amazon_polarity (40000 samples) |
| 18 | +**Model Size**: 15.9 MB |
| 19 | + |
| 20 | +| Metric | Value | |
| 21 | +|--------|-------| |
| 22 | +| Test Accuracy | 89.6% | |
| 23 | +| Test F1 | 0.896 | |
| 24 | +| Test Precision | 0.896 | |
| 25 | +| Test Recall | 0.896 | |
| 26 | +| ROC-AUC | 0.957 | |
| 27 | +| Cross-Domain Avg | 88.0% | |
| 28 | + |
| 29 | +### Production Model Confusion Matrix |
| 30 | + |
| 31 | +| | Predicted Negative | Predicted Positive | |
| 32 | +|---|---|---| |
| 33 | +| **Actual Negative** | 4478 (TN) | 525 (FP) | |
| 34 | +| **Actual Positive** | 519 (FN) | 4478 (TP) | |
| 35 | + |
| 36 | +### Best Generalizing Model |
| 37 | + |
| 38 | +**Model**: svm-amazon_polarity |
| 39 | +**Cross-Domain Average Accuracy**: 88.0% |
| 40 | + |
| 41 | +--- |
| 42 | + |
| 43 | +## Part 1: Model Comparison (All 12 Experiments) |
| 44 | + |
| 45 | +| Algorithm | Dataset | Accuracy | F1 | Precision | Recall | Training Time | |
| 46 | +|-----------|---------|----------|-----|-----------|--------|---------------| |
| 47 | +| SVM | amazon_polarity | 89.2% | 0.892 | 0.893 | 0.892 | 97m 22s | |
| 48 | +| SVM | imdb_50k | 89.1% | 0.891 | 0.891 | 0.891 | 245m 56s | |
| 49 | +| SVM | yelp | 94.0% | 0.940 | 0.940 | 0.940 | 22m 32s | |
| 50 | +| LOGISTIC_REGRESSION | amazon_polarity | 84.1% | 0.841 | 0.841 | 0.841 | 53m 28s | |
| 51 | +| LOGISTIC_REGRESSION | imdb_50k | 85.5% | 0.855 | 0.855 | 0.855 | 87m 57s | |
| 52 | +| LOGISTIC_REGRESSION | yelp | 92.6% | 0.926 | 0.926 | 0.926 | 24m 48s | |
| 53 | +| RANDOM_FOREST | amazon_polarity | 86.6% | 0.866 | 0.866 | 0.866 | 101m 40s | |
| 54 | +| RANDOM_FOREST | imdb_50k | 86.6% | 0.866 | 0.866 | 0.866 | 223m 51s | |
| 55 | +| RANDOM_FOREST | yelp | 92.0% | 0.920 | 0.921 | 0.920 | 37m 47s | |
| 56 | +| NAIVE_BAYES | amazon_polarity | 79.2% | 0.792 | 0.796 | 0.792 | 59m 38s | |
| 57 | +| NAIVE_BAYES | imdb_50k | 82.9% | 0.829 | 0.830 | 0.829 | 115m 9s | |
| 58 | +| NAIVE_BAYES | yelp | 81.3% | 0.813 | 0.816 | 0.813 | 10m 39s | |
| 59 | + |
| 60 | +--- |
| 61 | + |
| 62 | +## Part 2: Cross-Domain Evaluation |
| 63 | + |
| 64 | +Each model was evaluated on all three test domains. Asterisk (*) indicates in-domain evaluation. |
| 65 | + |
| 66 | +#### SVM |
| 67 | + |
| 68 | +| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg | |
| 69 | +|--------------|-----------|-------------|-----------|------------------| |
| 70 | +| imdb 50k | 89.1% * | 81.9% | 84.9% | 83.4% | |
| 71 | +| amazon polarity | 85.2% | 89.2% * | 90.9% | 88.0% | |
| 72 | +| yelp | 78.5% | 82.0% | 94.0% * | 80.3% | |
| 73 | + |
| 74 | +#### LOGISTIC REGRESSION |
| 75 | + |
| 76 | +| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg | |
| 77 | +|--------------|-----------|-------------|-----------|------------------| |
| 78 | +| imdb 50k | 85.5% * | 76.8% | 79.3% | 78.1% | |
| 79 | +| amazon polarity | 80.4% | 84.1% * | 84.7% | 82.6% | |
| 80 | +| yelp | 75.8% | 77.8% | 92.6% * | 76.8% | |
| 81 | + |
| 82 | +#### RANDOM FOREST |
| 83 | + |
| 84 | +| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg | |
| 85 | +|--------------|-----------|-------------|-----------|------------------| |
| 86 | +| imdb 50k | 86.6% * | 80.2% | 83.1% | 81.6% | |
| 87 | +| amazon polarity | 79.6% | 86.6% * | 89.6% | 84.6% | |
| 88 | +| yelp | 71.5% | 81.3% | 92.0% * | 76.4% | |
| 89 | + |
| 90 | +#### NAIVE BAYES |
| 91 | + |
| 92 | +| Train Domain | IMDB Test | Amazon Test | Yelp Test | Cross-Domain Avg | |
| 93 | +|--------------|-----------|-------------|-----------|------------------| |
| 94 | +| imdb 50k | 82.9% * | 73.5% | 79.1% | 76.3% | |
| 95 | +| amazon polarity | 69.4% | 79.2% * | 81.3% | 75.4% | |
| 96 | +| yelp | 59.8% | 70.9% | 81.3% * | 65.3% | |
| 97 | + |
| 98 | +**Legend**: * = in-domain evaluation |
| 99 | + |
| 100 | +--- |
| 101 | + |
| 102 | +## Part 3: Reproducibility |
| 103 | + |
| 104 | +All results can be reproduced via: |
| 105 | + |
| 106 | +```bash |
| 107 | +# Prepare immutable data splits (run once) |
| 108 | +./scripts/prepare_data.sh |
| 109 | + |
| 110 | +# Train all models |
| 111 | +./scripts/train_all_models.sh |
| 112 | + |
| 113 | +# Cross-domain evaluation |
| 114 | +./scripts/evaluate_cross_domain.sh |
| 115 | + |
| 116 | +# Regenerate this report |
| 117 | +./scripts/generate_report.sh |
| 118 | +``` |
| 119 | + |
| 120 | +### Model Artifacts |
| 121 | + |
| 122 | +| File | Algorithm | Dataset | Size | |
| 123 | +|------|-----------|---------|------| |
| 124 | +| amazon_polarity_logistic_regression_model.ser | LOGISTIC_REGRESSION | amazon_polarity | 14.4 MB | |
| 125 | +| imdb_50k_logistic_regression_model.ser | LOGISTIC_REGRESSION | imdb_50k | 30.1 MB | |
| 126 | +| yelp_logistic_regression_model.ser | LOGISTIC_REGRESSION | yelp | 9.7 MB | |
| 127 | +| amazon_polarity_naive_bayes_model.ser | NAIVE_BAYES | amazon_polarity | 14.5 MB | |
| 128 | +| imdb_50k_naive_bayes_model.ser | NAIVE_BAYES | imdb_50k | 30.3 MB | |
| 129 | +| yelp_naive_bayes_model.ser | NAIVE_BAYES | yelp | 9.9 MB | |
| 130 | +| sentiment_model.ser | SVM | amazon_polarity | 15.9 MB | |
| 131 | +| amazon_polarity_random_forest_model.ser | RANDOM_FOREST | amazon_polarity | 194.6 MB | |
| 132 | +| imdb_50k_random_forest_model.ser | RANDOM_FOREST | imdb_50k | 193.7 MB | |
| 133 | +| yelp_random_forest_model.ser | RANDOM_FOREST | yelp | 91.9 MB | |
| 134 | +| amazon_polarity_svm_model.ser | SVM | amazon_polarity | 15.9 MB | |
| 135 | +| imdb_50k_svm_model.ser | SVM | imdb_50k | 31.6 MB | |
| 136 | +| yelp_svm_model.ser | SVM | yelp | 10.3 MB | |
| 137 | + |
| 138 | +--- |
| 139 | + |
| 140 | +## Metadata |
| 141 | + |
| 142 | +- **Report Generated**: 2026-01-30T20:40:18Z |
| 143 | +- **Git Commit**: 6d7f5e5 |
| 144 | +- **Java Version**: 24.0.1 |
| 145 | + |
0 commit comments