llm mlx remove-model command

[maelp]

I'm not sure if I'm missing something but I can install models with llm mlx etc, but there doesn't seem to be a command to delete them, or find where they are stored on disk to remove them manually?

[simonw]

Yeah, llm-mlx could definitely do with a command for this. Currently:

llm mlx --help

Usage: llm mlx [OPTIONS] COMMAND [ARGS]...

  Commands for working with MLX models

Options:
  --help  Show this message and exit.

Commands:
  download-model  Download and register a MLX model
  import-models   Import existing MLX models from the Hugging Face cache
  models          List registered MLX models
  models-file     Display the path to the llm-mlx.json file

remove-model feels right to me.

[maelp]

Agreed. Although it might be more suited to do llm models rm? it's slightly confusing that there's a llm models and llm mlx

[simonw]

You can remove manually right now by editing the ~/Library/Application Support/io.datasette.llm/llm-mlx.json file and then deleting the model itself by deleting the directory called ~/.cache/huggingface/hub/models--mlx-community--Mistral-Small-3.1-Text-24B-Instruct-2503-8bit.

If I do a llm models remove I'll have to promote that to a full plugin hook concept so that different plugins (which store different models in different places) can all work. I'm not against doing that in the long run but it's a bigger job than just adding a llm mlx remove-model command for the moment.

[simonw]

So this would need to both update the llm-mlx.json file AND delete the Hugging Face cached model. I wonder if Hugging Face have a library method for that?

[maelp]

Ah I see! Indeed it might be a bit more complicated, then perhaps at least in llm mlx --help we could show where the models are stored so people could remove them if they need

[simonw]

Relevant code from https://github.com/ml-explore/mlx-lm/blob/3b9b0ed05260dfa6f9c660adb8dd4398bc990b1a/mlx_lm/manage.py#L91-L135

    if args.delete:
        print(f'Deleting models matching pattern "{args.pattern}"')
        hf_cache_info = scan_cache_dir()

        repos = [
            repo
            for repo in sorted(hf_cache_info.repos, key=lambda repo: repo.repo_path)
            if args.pattern in repo.repo_id
        ]
        if repos:
            print("\nFound the following models:")
            print(
                tabulate(
                    rows=[
                        [
                            repo.repo_id,
                            repo.size_on_disk_str,  # Added size information
                            str(repo.repo_path),
                        ]
                        for repo in repos
                    ],
                    headers=[
                        "REPO ID",
                        "SIZE",  # Added size header
                        "LOCAL PATH",
                    ],
                )
            )

            confirmed = ask_for_confirmation(
                "\nAre you sure you want to delete these models?"
            )
            if confirmed:
                for model_info in repos:
                    print(f"\nDeleting {model_info.repo_id}...")
                    for revision in sorted(
                        model_info.revisions, key=lambda revision: revision.commit_hash
                    ):
                        strategy = hf_cache_info.delete_revisions(revision.commit_hash)
                        strategy.execute()
                print("\nModel(s) deleted successfully.")
            else:
                print("\nDeletion cancelled - no changes made.")
        else:
            print(f'No models found matching pattern "{args.pattern}"')

[simonw]

Researched a bit with:

llm -m openai/o4-mini \
  -o web_search_preview 1 \
  -f issue:https://github.com/simonw/llm-mlx/issues/14 \
  -f github:simonw/llm-mlx \
  -s 'implement the new feature, look up the Hugging Face relevant delete model stuff first' -o search_context_size high

Output here: https://gist.github.com/simonw/7ba828c6e9db46cb3cb0b28b896bc85d