Llama 3.3 redownloading in every new session and few other problems

[VistritPandey]

For reference, I am using 2 Mac Mini with M4 Pro and 48GB RAM each. Both are connected with Thunderbolt 5 cable. However, I am facing multiple issues:

1. Llama 3.3-70b is working at 4~5 tokens/sec which is the same number of tokens or less I am getting when using only 1 machine and running the model with Ollama.
2. After pulling the latest version of the repository, or whenever I restart the machines, it redownloads some of the models
3. Llama 3.1-8b does not work at all
4. Getting gRPC error in the logs
   
5. It is not able to connect to Hugging Face to download the model, which it was able to do before I installed the latest version of MacOS (15.2 (24C101)). If it has something to do with the access token to hugging face, it is already there, and I can use it for other tasks
   
6. It is stuck at a number beyond which it is not able to download the Llama3.3 that it downloaded before. Maybe it's corrupted somehow?

[AFDudley]

My guess is that's not enough RAM. I have the same problem. https://www.substratus.ai/blog/calculating-gpu-memory-for-llm I'm still trying to figure out how to benchmark this system. when using llama 3.2 1B adding a node 32GB M2 Mac Mini to a single 64GB M4 mac mini varies performance up or down about 10%.

[VistritPandey]

Ohh that maybe a good answer. Thanks! I was under the assumption that it takes all the RAM by default to be considered by the GPU

[rolliver]

Re: It is not able to connect to Hugging Face to download the model

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer
certificate (_ssl.c:1000)

I ran

/Applications/Python\ 3.12/Install\ Certificates.command
and
pip install --upgrade certifi

and it seemed to fix it

[VistritPandey]

Re: It is not able to connect to Hugging Face to download the model

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

I ran

/Applications/Python\ 3.12/Install\ Certificates.command and pip install --upgrade certifi

and it seemed to fix it

This actually fixed the issue for me! Thank you!

Although when I restart my machine, I had to re-run it

[ThaddeusJiang]

Re: It is not able to connect to Hugging Face to download the model

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

I ran

/Applications/Python\ 3.12/Install\ Certificates.command and pip install --upgrade certifi

and it seemed to fix it

Thank you @rolliver

Fixed