Drastically improved load speed of llama when low_resource=False#370

Open

cmdr64 wants to merge 1 commit intoVision-CAIR:mainfrom cmdr64:load-llama-faster

+1

Commits on Oct 11, 2023

For non low_resource llama loading code, I added an argument low_cpu_mem_usage=True, drastically speeding up load time (cuts more than a minute off of load)

Cory
authored and
Cory
committed