Code

Release 1.1.0#1614

Code

Merged

myhloli merged 77 commits intomasteropendatalab/MinerU:masterfrom release-1.1.0opendatalab/MinerU:release-1.1.0Copy head branch name to clipboardJan 23, 2025

+542-271

Commits

Commits on Jan 14, 2025

• Update pdf_parse_union_core_v2.py

  myhloli

  authored
  9f12c39

  Copy full SHA for 9f12c39

• Merge pull request #1534 from myhloli/dev

  Show description for 1b0ef29
  myhloli

  authored
  1b0ef29

  Copy full SHA for 1b0ef29

• build(deps): add upper version limit for PyMuPDF

  Show description for ee9340e
  myhloli

  committed
  ee9340e

  Copy full SHA for ee9340e

• Merge pull request #1535 from myhloli/doclayoutyolo-fix

  Show description for bbe0276
  myhloli

  authored
  bbe0276

  Copy full SHA for bbe0276

• feat(layout): improve title block handling and layout detection

  Show description for c20e9a1
  myhloli

  committed
  c20e9a1

  Copy full SHA for c20e9a1

• refactor(BatchAnalyze): comment out image rotation logic in doclayout_yolo

  myhloli

  committed
  902dcd2

  Copy full SHA for 902dcd2

• Merge pull request #1537 from myhloli/doclayoutyolo-fix

  Show description for 206fcb3
  myhloli

  authored
  206fcb3

  Copy full SHA for 206fcb3

• feat(post_proc): enhance title block processing with average line height

  Show description for bbd8695
  myhloli

  committed
  bbd8695

  Copy full SHA for bbd8695

• Merge pull request #1538 from myhloli/doclayoutyolo-fix

  Show description for 60054fe
  myhloli

  authored
  60054fe

  Copy full SHA for 60054fe

Commits on Jan 15, 2025

• refactor(pre_proc): adjust IOU threshold for character overlap detection

  Show description for f37b14b
  myhloli

  committed
  f37b14b

  Copy full SHA for f37b14b

• Merge pull request #1542 from myhloli/dev

  Show description for e139e5b
  myhloli

  authored
  e139e5b

  Copy full SHA for e139e5b

• docs(magic_pdf): update llm_aided.py prompt for title list optimization

  Show description for 916ced9
  myhloli

  committed
  916ced9

  Copy full SHA for 916ced9

• Merge pull request #1543 from myhloli/dev

  Show description for 1954573
  myhloli

  authored
  1954573

  Copy full SHA for 1954573

• fix(language): remove invalid UTF-16 surrogate pairs from input text

  Show description for 1a549a0
  myhloli

  committed
  1a549a0

  Copy full SHA for 1a549a0

• Merge pull request #1546 from myhloli/dev

  Show description for fbc7611
  myhloli

  authored
  fbc7611

  Copy full SHA for fbc7611

• update logo

  myhloli

  committed
  75a61fc

  Copy full SHA for 75a61fc

• Merge pull request #1547 from myhloli/dev

  Show description for 316f3b9
  myhloli

  authored
  316f3b9

  Copy full SHA for 316f3b9

• build(docker): update doclayout-yolo dependency

  Show description for 84f808f
  myhloli

  committed
  84f808f

  Copy full SHA for 84f808f

• Merge pull request #1549 from myhloli/dev

  Show description for f405cc2
  myhloli

  authored
  f405cc2

  Copy full SHA for f405cc2

• feat(model): improve batch analysis logic and support npu

  Show description for f350222
  myhloli

  committed
  f350222

  Copy full SHA for f350222

• Merge pull request #1550 from myhloli/dev

  Show description for 852ae37
  myhloli

  authored
  852ae37

  Copy full SHA for 852ae37

• refactor(magic_pdf): improve title block merging logic

  Show description for 8570e00
  myhloli

  committed
  8570e00

  Copy full SHA for 8570e00

• Merge pull request #1551 from myhloli/dev

  Show description for 2aea5d6
  myhloli

  authored
  2aea5d6

  Copy full SHA for 2aea5d6

Commits on Jan 16, 2025

• fix(magic_pdf): correct end page index and improve error handling

  Show description for f209dde
  myhloli

  committed
  f209dde

  Copy full SHA for f209dde

• Merge pull request #1553 from myhloli/dev

  Show description for d08fe27
  myhloli

  authored
  d08fe27

  Copy full SHA for d08fe27

• docs(README): update demo badges

  Show description for cebaffb
  myhloli

  committed
  cebaffb

  Copy full SHA for cebaffb

• docs(README): update demo badges

  Show description for 443966d
  myhloli

  committed
  443966d

  Copy full SHA for 443966d

• Merge branch 'opendatalab:dev' into dev

  myhloli

  authored
  46ce94e

  Copy full SHA for 46ce94e

• Merge pull request #1554 from myhloli/dev

  Show description for 63c267f
  myhloli

  authored
  63c267f

  Copy full SHA for 63c267f

• feat(table): upgrade RapidTable to1.0.3 and add sub-model support

  Show description for 79c8a5c
  myhloli

  committed
  79c8a5c

  Copy full SHA for 79c8a5c

• build(docker): update rapid-table dependency

  Show description for 61f75b3
  myhloli

  committed
  61f75b3

  Copy full SHA for 61f75b3

• refactor(model): update batch analyze logic for rapid table model

  Show description for 452a9c0
  myhloli

  committed
  452a9c0

  Copy full SHA for 452a9c0

• Merge pull request #1556 from myhloli/dev

  Show description for 230191c
  myhloli

  authored
  230191c

  Copy full SHA for 230191c

• docs(README): update WeChat group link

  Show description for 48c2051
  myhloli

  committed
  48c2051

  Copy full SHA for 48c2051

• Merge pull request #1557 from myhloli/dev

  Show description for fd5427a
  myhloli

  authored
  fd5427a

  Copy full SHA for fd5427a

Commits on Jan 17, 2025

• refactor(table): add device configuration for Unitable model

  Show description for e64d4fe
  myhloli

  committed
  e64d4fe

  Copy full SHA for e64d4fe

• Merge pull request #1567 from myhloli/dev

  Show description for af3ec55
  myhloli

  authored
  af3ec55

  Copy full SHA for af3ec55

• refactor(model): update config version check to 1.1.1

  Show description for 59502e5
  myhloli

  committed
  59502e5

  Copy full SHA for 59502e5

• Merge pull request #1569 from myhloli/dev

  Show description for 9da857d
  myhloli

  authored
  9da857d

  Copy full SHA for 9da857d

• fix(magic_pdf): limit batch ratio for GPU memory

  Show description for db8be97
  myhloli

  committed
  db8be97

  Copy full SHA for db8be97

• Merge pull request #1570 from myhloli/dev

  Show description for b894b78
  myhloli

  authored
  b894b78

  Copy full SHA for b894b78

• feat(llm_aided): add reasonability check and fine-tuning guidelines

  Show description for d986e39
  myhloli

  committed
  d986e39

  Copy full SHA for d986e39

• Merge pull request #1571 from myhloli/dev

  Show description for 48a4337
  myhloli

  authored
  48a4337

  Copy full SHA for 48a4337

Commits on Jan 20, 2025

• Fix ocr utills

  moria97

  committed
  fbf1c4b

  Copy full SHA for fbf1c4b

• Merge pull request #1578 from moria97/fix-ocr

  Show description for 96d5bc0
  myhloli

  authored
  96d5bc0

  Copy full SHA for 96d5bc0

• feat(pdf_parse): remove tilted lines for better text extraction

  Show description for ba6c17a
  myhloli

  committed
  ba6c17a

  Copy full SHA for ba6c17a

• Merge pull request #1580 from myhloli/dev

  Show description for f473028
  myhloli

  authored
  f473028

  Copy full SHA for f473028

• fix(ocr): improve ONNX model initialization and error handling

  Show description for b3d60b9
  myhloli

  committed
  b3d60b9

  Copy full SHA for b3d60b9

• Merge pull request #1582 from myhloli/dev

  Show description for 58d28c9
  myhloli

  authored
  58d28c9

  Copy full SHA for 58d28c9

Commits on Jan 21, 2025

• fix(models): update unimernet_small model path

  Show description for 2a3a006
  myhloli

  committed
  2a3a006

  Copy full SHA for 2a3a006

• Merge pull request #1591 from myhloli/dev

  Show description for 6be3277
  myhloli

  authored
  6be3277

  Copy full SHA for 6be3277

• perf(model): adjust batch size for layout and formula detection

  Show description for 49d140c
  myhloli

  committed
  49d140c

  Copy full SHA for 49d140c

• perf(magic_pdf): optimize batch ratio calculation for GPU

  Show description for 052a4d7
  myhloli

  committed
  052a4d7

  Copy full SHA for 052a4d7

• refactor(magic_pdf): adjust VRAM allocation and MFR batch size- Update VRAM allocation logic to use 'VIRTUAL_VRAM_SIZE' environment variable

  Show description for e74a296
  myhloli

  committed
  e74a296

  Copy full SHA for e74a296

• Merge pull request #1593 from myhloli/dev

  Show description for 636d78a
  myhloli

  authored
  636d78a

  Copy full SHA for 636d78a

• perf(magic_pdf): adjust batch ratio calculation for GPU memory

  Show description for 037736f
  myhloli

  committed
  037736f

  Copy full SHA for 037736f

• Merge pull request #1594 from myhloli/dev

  Show description for 08a9558
  myhloli

  authored
  08a9558

  Copy full SHA for 08a9558

• perf(magic_pdf): optimize batch processing for GPU

  Show description for 55447c8
  myhloli

  committed
  55447c8

  Copy full SHA for 55447c8

• fix(magic_pdf): correct batch ratio conditions for GPU memory

  Show description for b6710b9
  myhloli

  committed
  b6710b9

  Copy full SHA for b6710b9

• Merge pull request #1595 from myhloli/dev

  Show description for 98c0568
  myhloli

  authored
  98c0568

  Copy full SHA for 98c0568

Commits on Jan 22, 2025

• refactor(pdf_parse): uncomment char bbox validation logic

  Show description for 1d08865
  myhloli

  committed
  1d08865

  Copy full SHA for 1d08865

• fix(boxbase): handle cases where bounding box area is zero

  Show description for c38060d
  myhloli

  committed
  c38060d

  Copy full SHA for c38060d

• Merge pull request #1601 from myhloli/dev

  Show description for c7a3a68
  myhloli

  authored
  c7a3a68

  Copy full SHA for c7a3a68

• feat(pdf_parse_union_core_v2): add timing log for LLM aided processes

  Show description for 10e848b
  myhloli

  committed
  10e848b

  Copy full SHA for 10e848b

• docs(readme):update readme for 1.1.0

  myhloli

  committed
  4fe89d5

  Copy full SHA for 4fe89d5

• Merge pull request #1602 from myhloli/dev

  Show description for 5115d00
  myhloli

  authored
  5115d00

  Copy full SHA for 5115d00

• docs(url): update Miners links in header

  Show description for af4b209
  myhloli

  committed
  af4b209

  Copy full SHA for af4b209

• Merge pull request #1606 from myhloli/dev

  Show description for dc41636
  myhloli

  authored
  dc41636

  Copy full SHA for dc41636

• Merge pull request #1607 from opendatalab/dev

  Show description for 2eb50da
  myhloli

  authored
  2eb50da

  Copy full SHA for 2eb50da

Commits on Jan 23, 2025

• feat(table-config): add sub_model configuration for rapid_table

  Show description for 6ff18b1
  myhloli

  committed
  6ff18b1

  Copy full SHA for 6ff18b1

• Merge pull request #1612 from myhloli/dev

  Show description for f101826
  myhloli

  authored
  f101826

  Copy full SHA for f101826

• Merge pull request #1613 from opendatalab/dev

  Show description for 235f341
  myhloli

  authored
  235f341

  Copy full SHA for 235f341

• docs(readme): update changelog for v1.1.0 release- Update model capabilities: upgrade to latest doclayout_yolo(2501) and unimernet(2501) models

  Show description for ab263aa
  myhloli

  committed
  ab263aa

  Copy full SHA for ab263aa

• Merge pull request #1616 from myhloli/dev

  Show description for 5c4c79e
  myhloli

  authored
  5c4c79e

  Copy full SHA for 5c4c79e

• docs(README): update online demo links and enhance documentation readability

  Show description for 30ac4d0
  myhloli

  committed
  30ac4d0

  Copy full SHA for 30ac4d0

• Merge pull request #1617 from myhloli/dev

  Show description for 24f352f
  myhloli

  authored
  24f352f

  Copy full SHA for 24f352f

• Merge pull request #1618 from opendatalab/dev

  Show description for adcace4
  myhloli

  authored
  adcace4

  Copy full SHA for adcace4