Skip to content

Remove all remaining torch calls#95

Merged
Blaizzy merged 5 commits into
Blaizzy:mainfrom
lucasnewman:remove-torch-dependency
Apr 26, 2025
Merged

Remove all remaining torch calls#95
Blaizzy merged 5 commits into
Blaizzy:mainfrom
lucasnewman:remove-torch-dependency

Conversation

@lucasnewman

Copy link
Copy Markdown
Collaborator

The only remaining caller of torch was the Kokoro voice loader, and we can replicate that with a small pickle-based loader to load the single tensor .pt files.

@Blaizzy

Blaizzy commented Apr 26, 2025

Copy link
Copy Markdown
Owner

Perfect, indeed it was the only missing dependencies!

However, users will still need torch so transformers can return torch tensors inputs for LLM based TTS models.

So we before we remove it we need to replace return_tensors=mlx or return_tensors=np and test it works.

Ideally with a fallback to torch, such as this:
https://github.com/Blaizzy/mlx-vlm/blob/ef7d6bad980b809a5d4d1e2b778dd736adb865f2/mlx_vlm/utils.py#L851

@Blaizzy

Blaizzy commented Apr 26, 2025

Copy link
Copy Markdown
Owner

Seems like all models are fine and return “mlx” specified, expect Sesame:

text_tokens = self._text_tokenizer.encode(f"[{speaker}]{text}")

@lucasnewman

Copy link
Copy Markdown
Collaborator Author

Thanks. I also pulled in the load_config() call from mlx-vlm since it's pretty small and avoids some implicit dependencies on mlx-vlm -> transformers -> torch.hub.

Comment thread mlx_audio/tts/utils.py Outdated

@Blaizzy Blaizzy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Blaizzy Blaizzy merged commit 021586f into Blaizzy:main Apr 26, 2025
1 check passed
@lucasnewman lucasnewman deleted the remove-torch-dependency branch May 2, 2025 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants