Skip to content

[Feature request] Tool calling support #21

Description

@binarybana

First time user of llm with the mlx backend. Really amazing UX and workflow you've got working here. Thank you!

I've downloaded several models that I'd expect to support tool calling (eg Qwen3) but it looks like tool calling is not supported with any of them, so I'm assuming it's a limitation of the plugin to expose it?

❯ llm models --options
...
MlxModel: mlx-community/Mistral-Small-24B-Instruct-2501-4bit (aliases: ms)
  Options:
    max_tokens: int
    unlimited: boolean
    temperature: float
    top_p: float
    min_p: float
    min_tokens_to_keep: int
    seed: int
  Features:
  - streaming
MlxModel: mlx-community/Qwen3-30B-A3B-4bit-DWQ-053125 (aliases: moe)
  Options:
    max_tokens: int
    unlimited: boolean
    temperature: float
    top_p: float
    min_p: float
    min_tokens_to_keep: int
    seed: int
  Features:
  - streaming
MlxModel: mlx-community/Qwen3-1.7B-4bit (aliases: smol)
  Options:
    max_tokens: int
    unlimited: boolean
    temperature: float
    top_p: float
    min_p: float
    min_tokens_to_keep: int
    seed: int
  Features:
  - streaming

Expected to see: tools in Features list above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions