Configurable LLM Client Settings

[hmshuv]

Configurable LLM Client Settings

Problem Description

Currently, client_lib.py has hardcoded values for retry logic (tries=10, delay=10) and model parameters. This lack of flexibility makes it difficult for users to:

• Adjust timeouts for slower/faster connections
• Change the default model without modifying the code
• Customize retry behavior based on their specific environment

Proposed Solution

Implement a configuration system to control these parameters through multiple sources.

1. Configuration Sources

Support loading configuration from:

• Environment variables: SWM_RETRY_COUNT, SWM_RETRY_DELAY, SWM_MODEL_NAME
• Configuration file: ~/.swm/config.yaml or pyproject.toml

2. Configurable Client Library

Update client_lib.py to use these configured values instead of hardcoded constants.

Implementation Details

Step 1: Create Config Module

Create a config.py module to handle loading settings:

from pydantic import BaseSettings

class Settings(BaseSettings):
    retry_count: int = 10
    retry_delay: int = 10
    model_name: str = "default-model"
    
    class Config:
        env_prefix = "SWM_"
        case_sensitive = False

Step 2: Update Client Library

• Update generate_with_retry decorator to accept dynamic arguments from config
• Load settings at module initialization
• Ensure backward compatibility with existing code

Step 3: Configuration File Support

• Support reading from ~/.swm/config.yaml or project-level pyproject.toml
• Priority order: Environment variables > Config file > Defaults

Acceptance Criteria

• Users can override default retry count via SWM_RETRY_COUNT environment variable
• Users can override retry delay via SWM_RETRY_DELAY environment variable
• Users can specify default model via SWM_MODEL_NAME environment variable
• Configuration can be loaded from ~/.swm/config.yaml or pyproject.toml
• Existing behavior remains the default if no config is provided (backward compatible)
• Tests verify configuration loading from all sources
• Documentation updated with configuration examples

Example Usage

# Via environment variables
export SWM_RETRY_COUNT=5
export SWM_RETRY_DELAY=15
export SWM_MODEL_NAME=gpt-4

# Via config file (~/.swm/config.yaml)
retry_count: 5
retry_delay: 15
model_name: gpt-4

---

Labels: enhancement, good-first-issue
Difficulty: Medium
Estimated Effort: 4-6 hours