You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context
Currently, the mlx-lm server relies on argparse.Namespace to pass configuration around. While this works for CLI usage, it presents some limitations:
Python 3.14 Compatibility: Newer versions of Python are stricter with dataclass validation, leading to ValueError or AttributeError when handling mutable defaults or missing attributes in certain environments.
Programmatic Access: It is currently difficult to launch the server as a library within other Python scripts without manually mocking argparse objects.
Robustness: Accessing aliases like .temp or .num_tokens directly on the config object can lead to runtime crashes if the object isn't perfectly initialized.
Proposal
I have developed a refactor that introduces a structured MLXServerConfig dataclass. This change:
Implements default_factory: Properly handles mutable types (lists/dicts) to ensure compatibility with modern Python versions and test runners.
Adds run_server(config): A new entry point that allows launching the server by passing a config object, making mlx-lm easier to integrate into existing pipelines.
Benefits
Full compatibility with Python 3.14.
Improved testability (cleaner mocking of server settings).
Better developer experience for those using MLX programmatically.
I have already implemented these changes along with a new test suite in tests/test_server.py. I would love to hear the maintainers' thoughts on this direction before opening a formal Pull Request.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Context
Currently, the mlx-lm server relies on argparse.Namespace to pass configuration around. While this works for CLI usage, it presents some limitations:
Python 3.14 Compatibility: Newer versions of Python are stricter with dataclass validation, leading to ValueError or AttributeError when handling mutable defaults or missing attributes in certain environments.
Programmatic Access: It is currently difficult to launch the server as a library within other Python scripts without manually mocking argparse objects.
Robustness: Accessing aliases like .temp or .num_tokens directly on the config object can lead to runtime crashes if the object isn't perfectly initialized.
Proposal
I have developed a refactor that introduces a structured MLXServerConfig dataclass. This change:
Implements default_factory: Properly handles mutable types (lists/dicts) to ensure compatibility with modern Python versions and test runners.
Adds run_server(config): A new entry point that allows launching the server by passing a config object, making mlx-lm easier to integrate into existing pipelines.
Benefits
Full compatibility with Python 3.14.
Improved testability (cleaner mocking of server settings).
Better developer experience for those using MLX programmatically.
I have already implemented these changes along with a new test suite in tests/test_server.py. I would love to hear the maintainers' thoughts on this direction before opening a formal Pull Request.
Looking forward to your feedback!
Beta Was this translation helpful? Give feedback.
All reactions