Help | error loading model: unexpectedly reached end of file

[kpshukla]

Running llama.cpp on intel MacBook Pro. I get the following error while running
./examples/alpaca.sh

I saved the model file in ./models

Error:
llama.cpp: loading model from ./models/ggml-alpaca-7b-q4.bin
error loading model: unexpectedly reached end of file
llama_init_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model './models/ggml-alpaca-7b-q4.bin'
main: error: unable to load model

What could be the problem?

[jyviko]

Same here:

main: build = 635 (5c64a09)
main: seed  = 1686146476
llama.cpp: loading model from ./models/7B/ggml-model-q4_0.bin
error loading model: unexpectedly reached end of file
llama_init_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model './models/7B/ggml-model-q4_0.bin'
main: error: unable to load model

System: MacBookPro15,2, Intel Core i7

[jspenaq]

Same error (MacBook M1 Pro, Ventura 13.3):

 ./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -n 512
main: build = 635 (5c64a09)
main: seed  = 1686150489
llama.cpp: loading model from ./models/7B/ggml-model-q4_0.bin
error loading model: unexpectedly reached end of file
llama_init_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model './models/7B/ggml-model-q4_0.bin'
main: error: unable to load model

[jspenaq]

I fix it by deleting ggml-model-f16.bin and ggml-model-q4_0.bin, then convert and quantize again.

[snwfdhmp]

can you provide a bash script ?

[kpshukla]

I am still getting the error...replaced the bin file

llama.cpp: loading model from ./models/ggml-alpaca-7b-q4.bin
error loading model: unexpectedly reached end of file
llama_init_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model './models/ggml-alpaca-7b-q4.bin'
main: error: unable to load model

[jyviko]

@kpshukla I think it's the model. Quantized my own model and it worked

[bolorundurovj]

@jyviko how'd you quantize the model?

[xelibrion]

@bolorundurovj get LLaMA weights and follow instructions here https://github.com/tatsu-lab/stanford_alpaca to get a complete representation of Alpaca model. Then follow instructions from README of this repo to quantize the model (it's written for LLaMA, but works for Alpaca as well).