WARN llama-server <chat> exited with status code 1

[Mte90]

Describe the bug
I am trying to run tabby but I get:

WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:98: llama-server <chat> exited with status code 1, args: `Command { std: "//tabby_x86_64-manylinux2014-cuda122/llama-server" "-m" "/home/mte90/.tabby/models/TabbyML/Mistral-7B/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30892" "-np" "1" "--log-disable" "--ctx-size" "4096" "-ngl" "9999" "--chat-template" "<s>{% for message in messages %}{% if (message[\'role\'] == \'user\') != (loop.index0 % 2 == 0) %}{{ raise_exception(\'Conversation roles must alternate user/assistant/user/assistant/...\') }}{% endif %}{% if message[\'role\'] == \'user\' %}{{ \'[INST] \' + message[\'content\'] + \' [/INST]\' }}{% elif message[\'role\'] == \'assistant\' %}{{ message[\'content\'] + \'</s> \' }}{% else %}{{ raise_exception(\'Only user and assistant roles are supported!\') }}{% endif %}{% endfor %}", kill_on_drop: true }`

Information about your version
0.21

Ideally when there is this output tabby should exit and not still trying to start and generate the command to try with the same parameter or atleast a log output so it is possible to investigate as copy and paste that output doesn't work as it is escaped.

[gtozzi]

Same here. Also tried 0.20

[JamesNewton]

Same. Version 0.23.

[wsxiaoys]

it's likely the CPU of the host doesn't support avx2 - thus cause the issue. We shall at least provide proper error log for such case.

Filing #3694

[JamesNewton]

it's likely the CPU of the host doesn't support avx2 - thus cause the issue. We shall at least provide proper error log for such case.

My system has Intel© Core™ i7-4700HQ which Intel says does have the avx2 feature:
https://www.intel.com/content/www/us/en/products/sku/75116/intel-core-i74700hq-processor-6m-cache-up-to-3-40-ghz/specifications.html

Note that this error only appears for me if I try to use the -device vulkan option. It works fine without that; on the CPU only.

[zwpaper]

Hi @JamesNewton, you might already be aware, but just to ensure clarity, running Tabby with Vulkan requires a Vulkan setup on your system. Can you confirm that this is in place?

If so, could you please provide more details about your system, such as the GPU, OS, and other relevant specifications?