Skip to content

Conversation

@ykhrustalev
Copy link
Contributor

@ykhrustalev ykhrustalev commented Oct 24, 2025

LFM2 model has specific tool calling handling, it only produces python like code as of Oct 2025.

The LFM2 requires proper system prompt formatting to trigger the tool calling functionality.

Changes:

  • Add proper system message formatting to render the tools in the form the model could understand them.
  • Add additional, but optional, JSON schema following. Due to the Python calling, the model is not generally good at JSON, but when the JSON schema constraint is applied it can follow the JSON generation instructions. To trigger that behavior, add force json schema. in the system prompt. By defaults, the output for tools invocation is not forced.

Attaching testing scripts in case anyone wants to try it out.

Upstream LFM2 models chat template will be updated next week.

Test:

cmake --preset arm64-apple-clang-release
cmake --build build-arm64-apple-clang-release -j10
./build-arm64-apple-clang-release/bin/llama-server \
  -hfr LiquidAI/LFM2-1.2B-GGUF \
  -hff LFM2-1.2B-Q4_0.gguf \
  --chat-template-file models/templates/llama-cpp-lfm2.jinja  \
  --jinja

@github-actions github-actions bot added the testing Everything test related label Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant