Llama 3.3 on llama.cpp

[obriensystems]

• https://huggingface.co/lmstudio-community/Llama-3.3-70B-Instruct-GGUF/tree/main

[obriensystems]

On windows - 14900K - c
clone the repo or the fork
https://github.com/ObrienlabsDev/llama.cpp

download/install cmake (will not interfere with your visual studio 2022 CUDA setup
https://cmake.org/download/

Build for CPU access first

   3 cd wse_github/obrienlabsdev
   4 cd .\llama.cpp\
   5 cmake -B build
   6 cmake --build build --config Release

download models
https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/tree/main
or
https://huggingface.co/lmstudio-community/Llama-3.3-70B-Instruct-GGUF/tree/main

place the model in llama-cpp/models

Run the model

PS C:\wse_github\obrienlabsdev\llama.cpp\build\bin\Release> .\llama-cli.exe -m ..\..\..\models\Llama-3.3-70B-Instruct-Q4_K_M.gguf -p "factorial function in java 21" -n 400 -e
llama_perf_sampler_print:    sampling time =       1.88 ms /    48 runs   (    0.04 ms per token, 25545.50 tokens per second)
llama_perf_context_print:        load time =    8654.92 ms
llama_perf_context_print: prompt eval time =   66842.50 ms /    25 tokens ( 2673.70 ms per token,     0.37 tokens per second)
llama_perf_context_print:        eval time =  352175.99 ms /   435 runs   (  809.60 ms per token,     1.24 tokens per second)
llama_perf_context_print:       total time =  422667.14 ms /   460 tokens
Interrupted by user

build for GPU access