Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ocr开启cuda加速后,无法解析pdf #894

Closed
huangyao1021 opened this issue Nov 7, 2024 · 5 comments
Closed

ocr开启cuda加速后,无法解析pdf #894

huangyao1021 opened this issue Nov 7, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@huangyao1021
Copy link

Description of the bug | 错误描述

[11/07 16:23:16 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from /root/.cache/modelscope/hub/opendatalab/PDF-Extract-Kit-1___0/models/Layout/LayoutLMv3/model_final.pth ...
[11/07 16:23:16 fvcore.common.checkpoint]: [Checkpointer] Loading from /root/.cache/modelscope/hub/opendatalab/PDF-Extract-Kit-1___0/models/Layout/LayoutLMv3/model_final.pth ...
[HAMI-core Msg(53365:140341461254656:memory.c:511)]: orig free=40618885120 total=42505273344 limit=42505076736 usage=1883242496
2024-11-07 16:23:22.830 | INFO | magic_pdf.model.pdf_extract_kit:init:302 - DocAnalysis init done!
2024-11-07 16:23:22.830 | INFO | magic_pdf.model.doc_analyze_by_custom_model:custom_model_init:131 - model init cost: 23.24226951599121


C++ Traceback (most recent call last):

0 at::_ops::conv2d::call(at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::SymInt)
1 at::native::conv2d_symint(at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::SymInt)
2 at::_ops::convolution::call(at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, bool, c10::ArrayRefc10::SymInt, c10::SymInt)
3 at::_ops::convolution::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, bool, c10::ArrayRefc10::SymInt, c10::SymInt)
4 at::native::convolution(at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRef, c10::ArrayRef, c10::ArrayRef, bool, c10::ArrayRef, long)
5 at::_ops::_convolution::call(at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, bool, c10::ArrayRefc10::SymInt, c10::SymInt, bool, bool, bool, bool)
6 at::native::_convolution(at::Tensor const&, at::Tensor const&, std::optionalat::Tensor const&, c10::ArrayRef, c10::ArrayRef, c10::ArrayRef, bool, c10::ArrayRef, long, bool, bool, bool, bool)
7 at::_ops::cudnn_convolution::call(at::Tensor const&, at::Tensor const&, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::SymInt, bool, bool, bool)
8 at::native::cudnn_convolution(at::Tensor const&, at::Tensor const&, c10::ArrayRef, c10::ArrayRef, c10::ArrayRef, long, bool, bool, bool)
9 cuLaunchKernel


Error Message Summary:

FatalError: Segmentation fault is detected by the operating system.
[TimeInfo: *** Aborted at 1730967804 (unix time) try "date -d @1730967804" if you are using GNU date ***]
[SignalInfo: *** SIGSEGV (@0x20000002ef4) received by PID 53365 (TID 0x7fa3caf20200) from PID 12020 ***]

段错误 (核心已转储)

How to reproduce the bug | 如何复现

absl-py 2.1.0
accelerate 1.1.1
aiohappyeyeballs 2.4.3
aiohttp 3.10.10
aiosignal 1.3.1
albucore 0.0.20
albumentations 1.4.21
annotated-types 0.7.0
antlr4-python3-runtime 4.9.3
anyio 4.6.2.post1
astor 0.8.1
async-timeout 4.0.3
attrdict 2.0.1
attrs 24.2.0
babel 2.16.0
bce-python-sdk 0.9.23
beautifulsoup4 4.12.3
black 24.10.0
blinker 1.8.2
boto3 1.35.55
botocore 1.35.55
braceexpand 0.1.7
Brotli 1.1.0
cachetools 5.5.0
certifi 2024.8.30
cffi 1.17.1
charset-normalizer 3.4.0
click 8.1.7
cloudpickle 3.1.0
colorlog 6.9.0
contourpy 1.3.0
cryptography 43.0.3
cssselect 1.2.0
cssutils 2.11.1
cycler 0.12.1
Cython 3.0.11
datasets 3.1.0
decorator 5.1.1
detectron2 0.6
dill 0.3.8
doclayout_yolo 0.0.2
einops 0.8.0
et_xmlfile 2.0.0
eva-decord 0.6.1
eval_type_backport 0.2.0
evaluate 0.4.3
exceptiongroup 1.2.2
fairscale 0.4.13
fast-langdetect 0.2.0
fasttext-wheel 0.9.2
filelock 3.16.1
fire 0.7.0
Flask 3.0.3
flask-babel 4.0.0
fonttools 4.54.1
frozenlist 1.5.0
fsspec 2024.9.0
ftfy 6.3.1
future 1.0.0
fvcore 0.1.5.post20221221
grpcio 1.67.1
h11 0.14.0
httpcore 1.0.6
httpx 0.27.2
huggingface-hub 0.26.2
hydra-core 1.3.2
idna 3.10
imageio 2.36.0
imgaug 0.4.0
iopath 0.1.9
itsdangerous 2.2.0
Jinja2 3.1.4
jmespath 1.0.1
joblib 1.4.2
kiwisolver 1.4.7
langdetect 1.0.9
lazy_loader 0.4
lmdb 1.5.1
loguru 0.7.2
lxml 5.3.0
magic-pdf 0.9.2
Markdown 3.7
MarkupSafe 3.0.2
matplotlib 3.9.2
modelscope 1.19.2
more-itertools 10.5.0
mpmath 1.3.0
multidict 6.1.0
multiprocess 0.70.16
mypy-extensions 1.0.0
networkx 3.4.2
numpy 1.26.4
nvidia-cublas-cu11 11.11.3.6
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu11 11.8.87
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu11 11.8.89
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu11 11.8.89
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu11 8.7.0.84
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu11 10.9.0.58
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu11 10.3.0.86
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu11 11.4.1.48
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu11 11.7.5.86
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu11 2.19.3
nvidia-nccl-cu12 2.20.5
nvidia-nvjitlink-cu12 12.6.77
nvidia-nvtx-cu11 11.8.86
nvidia-nvtx-cu12 12.1.105
omegaconf 2.3.0
opencv-contrib-python 4.6.0.66
opencv-python 4.6.0.66
opencv-python-headless 4.10.0.84
openpyxl 3.1.5
opt-einsum 3.3.0
packaging 24.1
paddleocr 2.7.3
paddlepaddle 3.0.0b1
paddlepaddle-gpu 3.0.0b1
pandas 2.2.3
pathspec 0.12.1
pdf2docx 0.5.8
pdfminer.six 20231228
pillow 11.0.0
pip 24.2
platformdirs 4.3.6
portalocker 2.10.1
premailer 3.10.0
propcache 0.2.0
protobuf 5.28.3
psutil 6.1.0
py-cpuinfo 9.0.0
pyarrow 18.0.0
pybind11 2.13.6
pyclipper 1.3.0.post6
pycocotools 2.0.8
pycparser 2.22
pycryptodome 3.21.0
pydantic 2.7.4
pydantic_core 2.18.4
PyMuPDF 1.24.13
pyparsing 3.2.0
python-dateutil 2.9.0.post0
python-docx 1.1.2
pytz 2024.2
PyYAML 6.0.2
RapidFuzz 3.10.1
rarfile 4.2
regex 2024.11.6
requests 2.32.3
robust-downloader 0.0.2
s3transfer 0.10.3
safetensors 0.4.5
scikit-image 0.24.0
scikit-learn 1.5.2
scipy 1.14.1
seaborn 0.13.2
setuptools 75.1.0
shapely 2.0.6
simsimd 6.0.1
six 1.16.0
sniffio 1.3.1
soupsieve 2.6
stringzilla 3.10.8
struct-eqtable 0.3.2
sympy 1.13.3
tabulate 0.9.0
tensorboard 2.18.0
tensorboard-data-server 0.7.2
termcolor 2.5.0
thop 0.1.1.post2209072238
threadpoolctl 3.5.0
tifffile 2024.9.20
timm 0.9.16
tokenizers 0.19.1
tomli 2.0.2
torch 2.3.1
torchtext 0.18.0
torchvision 0.18.1
tqdm 4.67.0
transformers 4.42.4
triton 2.3.1
typing_extensions 4.12.2
tzdata 2024.2
ultralytics 8.3.28
ultralytics-thop 2.0.10
unimernet 0.2.1
urllib3 2.2.3
visualdl 2.5.3
Wand 0.6.13
wcwidth 0.2.13
webdataset 0.2.100
Werkzeug 3.1.2
wheel 0.44.0
xxhash 3.5.0
yacs 0.1.8
yarl 1.17.1

没开启cuda加速前,也就是没有执行这个命令前:python -m pip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/ 一切正常
安装之后,没有任何包的冲突,安装之后重新执行命令就出现了解析失败的问题

Operating system | 操作系统

Linux

Python version | Python 版本

3.10

Software version | 软件版本 (magic-pdf --version)

0.9.x

Device mode | 设备模式

cuda

@huangyao1021 huangyao1021 added the bug Something isn't working label Nov 7, 2024
@myhloli
Copy link
Collaborator

myhloli commented Nov 8, 2024

有可能是cu118的paddle不兼容,您是H系列的显卡吗?

@huangyao1021
Copy link
Author

有可能是cu118的paddle不兼容,您是H系列的显卡吗?

我的卡是A100的,有影响吗

@myhloli
Copy link
Collaborator

myhloli commented Nov 8, 2024

有可能是cu118的paddle不兼容,您是H系列的显卡吗?

我的卡是A100的,有影响吗

A100应该是没问题的

@lrybbbccc
Copy link

会不会是同时装了cuda11和12驱动的问题?看到你的pip list里面有
截屏2024-11-25 19 52 44

@userjiangxin
Copy link

I also encountered the same problem. The graphics card is RTX4090. Is the problem solved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants