NO_REPLY from agent small model with llama.cpp running on Proxmox LXC

[variety4me]

My setup:

Host(localai): Proxmox LXC: 4 cores, 8GB RAM
LLM Server: llama.cpp
Model: llama-3.2-3b-q4.gguf

This is the setup i was using for n8n AI automations. Works flawlessly

Installed localgpt on this LXC (ocalai) but getting no reply from llm

$ curl http://192.168.150.16:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.2-3b-q4.gguf",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user", "content": "How do i create a git branch?" }
    ]
  }'
{"choices":[{"finish_reason":"stop","index":0,"message":{"role":"assistant","content":"To create a new Git branch, you can use the following commands:\n\n### Method 1: Using the `git branch` command\n\n```bash\ngit branch <branch-name>\n```\n\nReplace `<branch-name>` with the name of the new branch you want to create. For example:\n\n```bash\ngit branch feature/new-feature\n```\n\nThis will create a new branch named `feature/new-feature`.\n\n### Method 2: Using the `git checkout` command\n\nIf you want to create a new branch and switch to it immediately, you can use the following command:\n\n```bash\ngit checkout -b <branch-name>\n```\n\nFor example:\n\n```bash\ngit checkout -b feature/new-feature\n```\n\nThis will create a new branch named `feature/new-feature` and switch to it.\n\n### Method 3: Using the `git checkout` command with a branch name\n\nAlternatively, you can use the following command to create a new branch and switch to it:\n\n```bash\ngit checkout -t origin/<branch-name>\n```\n\nFor example:\n\n```bash\ngit checkout -t origin/feature/new-feature\n```\n\nThis will create a new branch named `<branch-name>` based on the remote branch `origin/<branch-name>`."}}],"created":1771845690,"model":"llama-3.2-3b-q4.gguf","system_fingerprint":"b8125-e877ad8bd","object":"chat.completion","usage":{"completion_tokens":252,"prompt_tokens":49,"total_tokens":301},"id":"chatcmpl-kj7g6QK2HXZBg5fqVXEAZe6dTmzwQjYg","timings":{"cache_n":0,"prompt_n":49,"prompt_ms":1479.253,"prompt_per_token_ms":30.188836734693876,"prompt_per_second":33.124827193184665,"predicted_n":252,"predicted_ms":23491.553,"p

$ localgpt ask "How do I create a new git branch?"
2026-02-23T11:21:38.008326Z  INFO localgpt::memory: Using local embedding provider: all-MiniLM-L6-v2
2026-02-23T11:21:38.104067Z  WARN localgpt::security::policy: LocalGPT.md exists but is not signed. Run `localgpt md sign` to activate.
2026-02-23T11:21:38.104758Z  INFO localgpt::agent: LocalGPT.md not signed. Run `localgpt md sign` to activate.
2026-02-23T11:21:38.106152Z  INFO localgpt::agent: Created new session: 6c8cb4a2-1521-43a5-9878-625386f19f7e
NO_REPLY

[yiwang]

@variety4me do you install from cargo install?
which version are you on?

[variety4me]

I installed with cargo install localgpt

localgpt --version
localgpt 0.2.0```

I see a lot of value here if this can handle smaller models, dont need the agent to do too much work, just orchestrate pre-written bash scripts to run maintenance, log analysis and alerts on a production server

[yiwang]

@variety4me thanks, I haven't got a chance to run smaller model yet. will give it a try

[yiwang]

@variety4me do you have a simple example for how you setup n8n prompts, I have a theory that the input is too big for small models.

[variety4me]

you could take a look here https://n8n.io/workflows/categories/ai/

[variety4me]

@variety4me do you have a simple example for how you setup n8n prompts, I have a theory that the input is too big for small models.

let me try to give an example, "update docker containerX on server2"

localgpt (planning) → llama.cpp running LFM2.5-Thinking (JSON tool call) → n8n/xyOps webhook → SSH "docker pull + docker compose up -d"