Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hocr: only add space if boxwidth is positive #1446

Merged
merged 1 commit into from
Jan 2, 2025

Conversation

pajowu
Copy link
Contributor

@pajowu pajowu commented Dec 11, 2024

If the HOCR contains overlapping words in the same line or the words are not ordered in reading-direction, space_box.width might be negative. This leads to more issues with the calculation of font width etc, producing text boxes that are way larger than the actual bbox set within the hocr-file

Copy link

codecov bot commented Dec 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.20%. Comparing base (02d85ff) to head (74253e5).
Report is 10 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1446      +/-   ##
==========================================
+ Coverage   90.18%   90.20%   +0.01%     
==========================================
  Files          95       95              
  Lines        7073     7073              
  Branches      722      722              
==========================================
+ Hits         6379     6380       +1     
  Misses        491      491              
+ Partials      203      202       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbarlow83
Copy link
Collaborator

This is great work - I just want to take some time to review it.

@jbarlow83 jbarlow83 merged commit d87db6c into ocrmypdf:main Jan 2, 2025
28 checks passed
@QuLogic
Copy link
Contributor

QuLogic commented Feb 9, 2025

This seems to have caused a test failure for me, but I'm not entirely sure why:

_____________________ test_hocrtransform_matches_sandwich ______________________
[gw1] linux -- Python 3.13.2 /usr/bin/python3
resources = PosixPath('/builddir/build/BUILD/ocrmypdf-16.8.0-build/ocrmypdf-16.8.0/tests/resources')
outdir = PosixPath('/tmp/pytest-of-mockbuild/pytest-0/popen-gw1/test_hocrtransform_matches_san0')
    @pytest.mark.slow
    def test_hocrtransform_matches_sandwich(resources, outdir):
        check_ocrmypdf(resources / 'ccitt.pdf', outdir / 'hocr.pdf', '--pdf-renderer=hocr')
        check_ocrmypdf(
            resources / 'ccitt.pdf', outdir / 'tess.pdf', '--pdf-renderer=sandwich'
        )
    
        # Slight differences in spacing and word order can appear, so at least ensure
        # that we get all of the same words...
        def clean(s):
            s = re.sub(r'\s+', ' ', s)
            words = s.split(' ')
            return '\n'.join(sorted(words))
    
        hocr_txt = clean(text_from_pdf(outdir / 'hocr.pdf'))
        tess_txt = clean(text_from_pdf(outdir / 'tess.pdf'))
    
        # from pathlib import Path
        # Path('hocr.txt').write_text(hocr_txt)
        # Path('tess.txt').write_text(tess_txt)
    
>       assert hocr_txt == tess_txt
E       AssertionError: assert '\n#298949\n(...ing’\n“chain”' == '\n#298949\n(...ing’\n“chain”'
E         
E         Skipping 2456 identical leading characters in diff, use -v to show
E           let
E           linn
E         - ll
E           location
E           location...
E         
E         ...Full output truncated (333 lines hidden), use '-vv' to show
tests/test_hocrtransform.py:96: AssertionError
----------------------------- Captured stderr call -----------------------------
Scanning contents     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
OCR                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
PDF/A conversion      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
Linearizing           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 100/100 0:00:00
Recompressing JPEGs   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
Deflating JPEGs       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
JBIG2                 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
Scanning contents     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
OCR                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
PDF/A conversion      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
Linearizing           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 100/100 0:00:00
Recompressing JPEGs   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
Deflating JPEGs       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
JBIG2                 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--

@QuLogic
Copy link
Contributor

QuLogic commented Feb 9, 2025

Ah, it is one of the slow tests, so I think it's not being run in CI?

@jbarlow83
Copy link
Collaborator

It should pass. At some point some difference emerged in how those two rendering modes handle straight and smart quotes which appears to be issue. I changed the test to be tolerant of a few minor differences in commit 32322a9f.

@QuLogic
Copy link
Contributor

QuLogic commented Feb 11, 2025

I don't see that commit; is it not pushed out, or a typo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants