Skip to content

Add comprehensive test coverage for broken_wikicode check#161

Open
xenacode-art wants to merge 1 commit into
Wikimedia-Suomi:mainfrom
xenacode-art:test/broken-wikicode-coverage
Open

Add comprehensive test coverage for broken_wikicode check#161
xenacode-art wants to merge 1 commit into
Wikimedia-Suomi:mainfrom
xenacode-art:test/broken-wikicode-coverage

Conversation

@xenacode-art

Copy link
Copy Markdown
Contributor

Hi @zache-fi @ad-an-26 @ademolaomosanya ,
This commit adds complete test coverage for the broken_wikicode detection system, which previously had zero tests despite being a critical feature for auto-review decisions.

Added tests cover:

  1. Visible text extraction (9 tests)

    • Basic text extraction from HTML
    • Removal of script, style, code, pre, tt, syntaxhighlight tags
    • Ensures code examples don't trigger false positives
    • Edge cases: empty and None inputs
  2. Math article detection (7 tests)

    • Detection via math class attributes
    • Detection via math tags
    • Detection via LaTeX backslash syntax
    • Detection via dollar sign inline math
    • False negatives for regular articles
    • Edge cases: empty and None inputs
  3. Localized media keywords (5 tests)

    • English, Finnish, German, French keywords
    • Unknown language defaults to English
    • Ensures internationalization works correctly
  4. Indicator detection (14 tests)

    • Template syntax {{}}
    • Internal link syntax [[]]
    • Broken ref, div, span tags (with HTML escaping)
    • File/Category syntax with localization
    • Section headers (== ignored in math articles)
    • Code block exclusion to prevent false positives
    • Multiple simultaneous indicators
    • Empty input edge cases
  5. Threshold logic and parent comparison (9 tests)

    • Clean HTML returns no broken wikicode
    • Single indicator low count (<3) ignored as noise
    • Single indicator high count (>=3) detected
    • Multiple indicator types (2+) always detected
    • NEW indicators compared to parent HTML
    • Pre-existing indicators not flagged
    • Details message format verification
    • Edge cases for threshold boundaries
    • Localized keyword integration

Test results: All 44 tests pass successfully

This brings test coverage from 0% to comprehensive coverage of all major code paths and edge cases in the broken_wikicode detection system.

This commit adds complete test coverage for the broken_wikicode detection
system, which previously had zero tests despite being a critical feature for
auto-review decisions.

Added tests cover:

1. Visible text extraction (9 tests)
   - Basic text extraction from HTML
   - Removal of script, style, code, pre, tt, syntaxhighlight tags
   - Ensures code examples don't trigger false positives
   - Edge cases: empty and None inputs

2. Math article detection (7 tests)
   - Detection via math class attributes
   - Detection via math tags
   - Detection via LaTeX backslash syntax
   - Detection via dollar sign inline math
   - False negatives for regular articles
   - Edge cases: empty and None inputs

3. Localized media keywords (5 tests)
   - English, Finnish, German, French keywords
   - Unknown language defaults to English
   - Ensures internationalization works correctly

4. Indicator detection (14 tests)
   - Template syntax {{}}
   - Internal link syntax [[]]
   - Broken ref, div, span tags (with HTML escaping)
   - File/Category syntax with localization
   - Section headers (== ignored in math articles)
   - Code block exclusion to prevent false positives
   - Multiple simultaneous indicators
   - Empty input edge cases

5. Threshold logic and parent comparison (9 tests)
   - Clean HTML returns no broken wikicode
   - Single indicator low count (<3) ignored as noise
   - Single indicator high count (>=3) detected
   - Multiple indicator types (2+) always detected
   - NEW indicators compared to parent HTML
   - Pre-existing indicators not flagged
   - Details message format verification
   - Edge cases for threshold boundaries
   - Localized keyword integration

Test results: All 44 tests pass successfully

This brings test coverage from 0% to comprehensive coverage of all
major code paths and edge cases in the broken_wikicode detection system.
@xenacode-art

Copy link
Copy Markdown
Contributor Author

Why threshold logic matters: Without it, a single stray {{ could block legitimate edits. The system
intelligently filters noise from real problems.

Test Results

cd app
python manage.py test reviews.tests.autoreview.test_broken_wikicode
# Ran 44 tests in 0.041s - OK ✅

Impact

- Before: 0% test coverage on critical feature
- After: Comprehensive coverage of all code paths
- Benefits:
  - Safe to refactor and improve detection logic
  - Prevents regressions when making changes
  - Documents expected behavior for future contributors
  - Shows thorough understanding of internationalization needs

Key Testing Insights

1. HTML Entity Escaping: Tests use &lt; and &gt; because BeautifulSoup parses actual HTML
2. Math Article Special Handling: Tests verify == is ignored in math contexts
3. Threshold Edge Cases: Tests verify boundaries (count=2 vs 3, types=1 vs 2)
4. Localization: Tests cover multiple languages to ensure international wikis work
5. False Positive Prevention: Tests verify code blocks don't trigger detection

Checklist

- All 44 tests pass
- Tests cover all major code paths
- Tests cover edge cases (None, empty, boundaries)
- Tests document complex threshold logic
- Tests verify internationalization works
- No changes to production code (pure test addition)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant