Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast-rcnn training error: Check failed: data_ *** Check failure stack trace: *** Aborted (core dumped) when training a VGG16 network #182

Open
Truthuleave opened this issue Sep 20, 2018 · 1 comment

Comments

@Truthuleave
Copy link

Truthuleave commented Sep 20, 2018

When I trained a vgg16 network using ./tools/train_net.py --gpu 0 --solver models/VGG16/solver.prototxt --weights data/imagenet_models/VGG16.v2.caffemodel, I got an error almost at the end.
Error is as follows:

I0920 10:43:24.275528 14792 solver.cpp:60] Solver scaffolding done.
Loading pretrained model weights from data/imagenet_models/VGG16.v2.caffemodel
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:537] Reading dangerously large protocol message. If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 553432430
I0920 10:43:24.639192 14792 net.cpp:816] Ignoring source layer pool5
I0920 10:43:24.724637 14792 net.cpp:816] Ignoring source layer fc8
I0920 10:43:24.724663 14792 net.cpp:816] Ignoring source layer prob
Solving...
F0920 10:43:25.510537 14792 blob.cpp:95] Check failed: data_
*** Check failure stack trace: ***
Aborted (core dumped)

The snapshot:
qq 20180920162705

Anyone has encountered this before,please?

@Truthuleave
Copy link
Author

GTX960M 4G memory + cuda 8.0+cudnn 7.1.4 for cuda 8

@Truthuleave Truthuleave changed the title Check failed: data_ *** Check failure stack trace: *** Aborted (core dumped) when training a VGG16 network Fast-rcnn training error: Check failed: data_ *** Check failure stack trace: *** Aborted (core dumped) when training a VGG16 network Sep 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant