Skip to content

Conversation

@igordayen
Copy link
Contributor

@igordayen igordayen commented Nov 3, 2025

Multi-Ollama Instance Support

Please refer to issue: #886

Summary

Enables applications to configure and use multiple Ollama instances with explicit node-aware
model selection while maintaining full backward compatibility.

Configuration Options

Legacy Single Instance (Unchanged)

# Existing configuration continues to work unchanged
spring.ai.ollama.base-url=http://localhost:11434

Multi-Node Configuration

# Configure multiple Ollama instances
spring.ai.ollama.nodes[0].name=main
spring.ai.ollama.nodes[0].base-url=http://localhost:11434
spring.ai.ollama.nodes[1].name=gpu-server
spring.ai.ollama.nodes[1].base-url=http://localhost:11435

Hybrid Configuration

# Use both default instance and additional nodes
spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.nodes[0].name=main
spring.ai.ollama.nodes[0].base-url=http://localhost:11434
spring.ai.ollama.nodes[1].name=gpu-server  
spring.ai.ollama.nodes[1].base-url=http://localhost:11435

Application Model Selection

Backward Compatible (Legacy)

# Uses any available instance (default behavior)
embabel.models.default-llm=gemma3:latest

Node-Specific Selection

# Target specific Ollama instances by node name
embabel.models.default-llm=main-qwen3:latest
embabel.models.llms.cheapest=main-qwen3:latest  
embabel.models.llms.best=gpu-server-llama3:latest

Multi-LLM Agent Configuration

examples:
  book-writer:
    researcher-llm:
      model: main-gemma3:latest        # Fast model on CPU node
    writer-llm:
      model: gpu-server-llama3:latest  # Powerful model on GPU node

@igordayen
Copy link
Contributor Author

addition of Unit tests for Olama Configuration:

image

@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 4, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants