docs: radically simplify EVALUATION.md and RANKING.md

[timduly4]

Summary

Radically simplified two major documentation files by removing extensive documentation for unimplemented features and keeping only practical, working content.

Files Changed

1. EVALUATION.md: 982 → 218 lines (78% reduction)

2. RANKING.md: 843 → 338 lines (60% reduction)

Total reduction: 1,825 lines → 556 lines (69% reduction, 1,269 lines removed)

---

Problem

Both documentation files extensively documented features that don't exist:

EVALUATION.md Issues

• Scripts that don't exist (evaluate_ranking.py, generate_test_queries.py)
• Modules that don't exist (src.ranking.evaluation, EvaluationService, ABTest)
• Infrastructure not implemented (A/B testing, continuous monitoring, Grafana dashboards)
• Automated evaluation pipelines not yet built

RANKING.md Issues

• Non-existent modules (BM25Scorer, FeatureExtractor)
• 40+ ranking features documented but not implemented
• Parameter tuning infrastructure not built
• Query expansion, ML reranking not implemented
• Advanced optimization code (ANN, batch processing) not built

This created confusion for users trying to follow the documentation.

---

Changes Made

EVALUATION.md (982 → 218 lines)

Removed (764 lines):

• ❌ Unimplemented evaluation scripts and tools
• ❌ Non-existent Python modules and classes
• ❌ A/B testing methodology (200+ lines)
• ❌ Automated evaluation pipeline documentation
• ❌ Continuous monitoring and Grafana integration
• ❌ Relevance judgment collection tools
• ❌ Query set generation and validation
• ❌ Statistical significance testing code

Kept (218 lines):

• ✅ Manual search testing with working commands
• ✅ Brief explanation of key metrics (MRR, NDCG, P@k, R@k)
• ✅ Comparison of semantic/BM25/hybrid strategies
• ✅ Practical evaluation checklist
• ✅ Links to external resources for theory
• ✅ "Future Evaluation Plans" section for aspirational features

RANKING.md (843 → 338 lines)

Removed (505 lines):

• ❌ Non-existent modules (BM25Scorer, FeatureExtractor, ABTest)
• ❌ 40+ ranking features documentation (not implemented)
• ❌ Parameter tuning infrastructure (not implemented)
• ❌ Query expansion and reformulation (not implemented)
• ❌ Advanced optimization code (ANN, batch processing)
• ❌ A/B testing framework (not implemented)
• ❌ ML-based reranking features

Kept (338 lines):

• ✅ Four actual strategies: semantic, bm25, hybrid_rrf, hybrid_weighted
• ✅ Clear decision guide for when to use each
• ✅ Working API examples (all curl commands tested)
• ✅ Algorithm explanations with concrete examples
• ✅ Understanding sections for BM25, RRF, vector similarity
• ✅ "Future Enhancements" section (clearly marked as planned)

---

Key Improvements

Both Files

1. Added disclaimers at top:

   • EVALUATION.md: "Automated evaluation pipelines... not yet implemented"
   • RANKING.md: "Advanced features like ML-based ranking... not yet implemented"

2. All code examples work: Every curl command and script uses actual implemented endpoints

3. Clear structure:

   • What you can do now (implemented features)
   • Future plans (clearly separated and marked)
   • External resources for deeper learning

4. Status footers:

   • EVALUATION.md: "Manual testing only; automated evaluation planned"
   • RANKING.md: "Basic ranking strategies implemented; advanced features planned"

---

Impact

File	Before	After	Reduction	Usability
EVALUATION.md	982 lines	218 lines	78%	✅ 100% working
RANKING.md	843 lines	338 lines	60%	✅ 100% working
Total	1,825 lines	556 lines	69%	✅ All examples work

---

Testing

EVALUATION.md

• ✅ All curl commands verified to work with current API
• ✅ Comparison script tested and produces expected output
• ✅ All links to external resources checked

RANKING.md

• ✅ All strategy examples tested (semantic, bm25, hybrid_rrf, hybrid_weighted)
• ✅ Comparison script tested with actual data
• ✅ Algorithm explanations verified against implementation
• ✅ All academic paper links checked

---

Related PRs

Part of post-Milestone 3 documentation cleanup:

• PR docs: cleanup and fixes after Milestone 3 completion #42: Fixed DEMO.md bugs and collection UUID issue
• PR fix(docs): correct DEMO.md commands and improve usability #43: Fixed DEMO.md command syntax
• PR docs: remove unimplemented API endpoints from API_EXAMPLES.md #44: Cleaned up API_EXAMPLES.md
• PR docs: remove QUICKSTART.md (redundant with DEMO.md) #45: Removed redundant QUICKSTART.md
• PR docs: remove milestone references from TESTING.md #46: Removed milestone references from TESTING.md

---

Philosophy

Better to have accurate, minimal documentation than comprehensive but misleading documentation.

We can expand these files when the infrastructure is actually implemented. For now, users get:

• Clear understanding of what exists
• Working examples they can copy-paste
• Honest roadmap of future features
• No confusion from non-existent code references