Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# LLaMEA – Claude Code Guide

## Repository overview

LLaMEA is a Python framework that drives an evolutionary loop powered by an LLM.
It generates, evaluates, and refines code (e.g. optimisation algorithms, ML pipelines)
by prompting the LLM to mutate the current best solution.

Key source files under `llamea/`:
- `llamea.py` – main `LLaMEA` class; evolutionary loop, prompt construction, default prompts
- `llm.py` – LLM wrappers (`OpenAI_LLM`, `Gemini_LLM`, `Ollama_LLM`, `Dummy_LLM`, …)
- `solution.py` – `Solution` dataclass holding code, description, score, feedback
- `diffmodemanager.py` – applies SEARCH/REPLACE diffs returned by the LLM
- `feature_guidance.py` – complexity-based mutation guidance
- `loggers.py` / `utils.py` – experiment logging and helper utilities

Examples live in `examples/`, benchmarks in `benchmarks/`.

## Development workflow

### Install dependencies (always do this first)
```bash
uv sync --dev
```

### Run import sorter (run before black)
```bash
uv run isort llamea/
```

### Run formatter
```bash
uv run black llamea/
```

### Run tests
```bash
uv run pytest
# with coverage:
uv run pytest --cov=llamea
```

### Typical pre-commit sequence
```bash
uv sync --dev
uv run isort llamea/
uv run black llamea/
uv run pytest
```

## Code conventions

- **Formatter**: `black` (line length default, ~88 chars)
- **Import order**: `isort` (stdlib → third-party → local)
- **Python**: ≥ 3.11
- **Multi-line prompt strings**: wrap with `textwrap.dedent()` to avoid sending
accidental leading whitespace to the LLM, e.g.:
```python
import textwrap
prompt = textwrap.dedent("""
Your instructions here.
More lines.
""")
```

## Prompt architecture

`LLaMEA.__init__` accepts four prompt parameters (all have defaults):
| Parameter | Purpose |
|---|---|
| `role_prompt` | LLM system persona (neutral, task-independent) |
| `task_prompt` | Problem description; what the LLM should produce |
| `example_prompt` | Illustrative code example |
| `output_format_prompt` | Required response format |

A fifth prompt, `diff_output_format_prompt`, is built-in and controls the
SEARCH/REPLACE diff mode.

The default `role_prompt` is intentionally generic ("You are a highly skilled
computer scientist and Python expert.") so it works across domains without
users needing to override it.

## Git branch convention

Feature branches follow the pattern `feature/<short-description>`.
21 changes: 11 additions & 10 deletions examples/black-box-opt-with-HPO.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

import os
import re
import textwrap
import time
from itertools import product

Expand Down Expand Up @@ -170,7 +171,7 @@ def get_bbob_performance(config: Configuration, instance: str, seed: int = 0):
return solution

role_prompt = "You are a highly skilled computer scientist in the field of natural computing. Your task is to design novel metaheuristic algorithms to solve black box optimization problems."
task_prompt = """
task_prompt = textwrap.dedent("""
The optimization algorithm should handle a wide range of tasks, which is evaluated on the BBOB test suite of 24 noiseless functions. Your task is to write the optimization algorithm in Python code. The code should contain an `__init__(self, budget, dim)` function with optional additional arguments and the function `def __call__(self, func)`, which should optimize the black box function `func` using `self.budget` function evaluations.
The func() can only be called as many times as the budget allows, not more. Each of the optimization functions has a search space between -5.0 (lower bound) and 5.0 (upper bound). The dimensionality can be varied.

Expand All @@ -180,22 +181,22 @@ def get_bbob_performance(config: Configuration, instance: str, seed: int = 0):
```python
{
"float_parameter": (0.1, 1.5),
"int_parameter": (2, 10),
"int_parameter": (2, 10),
"categoral_parameter": ["mouse", "cat", "dog"]
}
```

Give an excellent and novel heuristic algorithm including its configuration space to solve this task and also give it a one-line description, describing the main idea.
"""
Give an excellent and novel heuristic algorithm including its configuration space to solve this task and also give it a one-line description, describing the main idea.
""")

format_prompt = """
format_prompt = textwrap.dedent("""
Give the response in the format:
# Description: <short-description>
# Code: <code>
# Space: <configuration_space>
"""
""")

example_prompt = """
example_prompt = textwrap.dedent("""
An example of such code (a simple random search), is as follows:
```python
import numpy as np
Expand All @@ -210,15 +211,15 @@ def __call__(self, func):
self.x_opt = None
for i in range(self.budget):
x = np.random.uniform(func.bounds.lb, func.bounds.ub)

f = func(x)
if f < self.f_opt:
self.f_opt = f
self.x_opt = x

return self.f_opt, self.x_opt
```
"""
""")

feedback_prompts = [
f"Either refine or redesign to improve the solution (and give it a distinct one-line description)."
Expand Down
5 changes: 3 additions & 2 deletions examples/black-box-optimization-diff-mode.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
# code replacement, without making new request for that iteration.

import os
import textwrap

import numpy as np
from ioh import get_problem, logger
Expand Down Expand Up @@ -85,11 +86,11 @@ def evaluateBBOB(solution, explogger=None):
return solution

# The task prompt describes the problem to be solved by the LLaMEA algorithm.
task_prompt = """
task_prompt = textwrap.dedent("""
The optimization algorithm should handle a wide range of tasks, which is evaluated on the BBOB test suite of 24 noiseless functions. Your task is to write the optimization algorithm in Python code. The code should contain an `__init__(self, budget, dim)` function and the function `def __call__(self, func)`, which should optimize the black box function `func` using `self.budget` function evaluations.
The func() can only be called as many times as the budget allows, not more. Each of the optimization functions has a search space between -5.0 (lower bound) and 5.0 (upper bound). The dimensionality can be varied.
Give an excellent and novel heuristic algorithm to solve this task and also give it a one-line description with the main idea.
"""
""")

for experiment_i in [1]:
# A 1+1 strategy
Expand Down
5 changes: 3 additions & 2 deletions examples/black-box-optimization.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import os
import pickle
import textwrap

import numpy as np
from ioh import get_problem, logger
Expand Down Expand Up @@ -80,11 +81,11 @@ def evaluateBBOB(solution, explogger=None):
return solution

# The task prompt describes the problem to be solved by the LLaMEA algorithm.
task_prompt = """
task_prompt = textwrap.dedent("""
The optimization algorithm should handle a wide range of tasks, which is evaluated on the BBOB test suite of 24 noiseless functions. Your task is to write the optimization algorithm in Python code. The code should contain an `__init__(self, budget, dim)` function and the function `def __call__(self, func)`, which should optimize the black box function `func` using `self.budget` function evaluations.
The func() can only be called as many times as the budget allows, not more. Each of the optimization functions has a search space between -5.0 (lower bound) and 5.0 (upper bound). The dimensionality can be varied.
Give an excellent and novel heuristic algorithm to solve this task and also give it a one-line description with the main idea.
"""
""")

for experiment_i in [1]:
# A 1+1 strategy
Expand Down
5 changes: 3 additions & 2 deletions examples/minimum_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import os
import re
import textwrap

import numpy as np

Expand Down Expand Up @@ -36,11 +37,11 @@ def evaluate(solution, explogger=None):
return solution

# The task prompt describes the problem to be solved by the LLaMEA algorithm.
task_prompt = """
task_prompt = textwrap.dedent("""
The optimization algorithm should handle a wide range of tasks, which is evaluated on the BBOB test suite of 24 noiseless functions. Your task is to write the optimization algorithm in Python code. The code should contain an `__init__(self, budget, dim)` function and the function `def __call__(self, func)`, which should optimize the black box function `func` using `self.budget` function evaluations.
The func() can only be called as many times as the budget allows, not more. Each of the optimization functions has a search space between -5.0 (lower bound) and 5.0 (upper bound). The dimensionality can be varied.
Give an excellent and novel heuristic algorithm to solve this task and also give it a one-line description with the main idea. You can only use the numpy (1.26) library, no other libraries are allowed.
"""
""")
es = LLaMEA(
evaluate,
n_parents=1,
Expand Down
5 changes: 3 additions & 2 deletions examples/warm-start-tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import os
import re
import pickle
import textwrap

import numpy as np
from ioh import get_problem, logger
Expand Down Expand Up @@ -68,11 +69,11 @@ def evaluateBBOB(solution, explogger=None):
return solution

# The task prompt describes the problem to be solved by the LLaMEA algorithm.
task_prompt = """
task_prompt = textwrap.dedent("""
The optimization algorithm should handle a wide range of tasks, which is evaluated on the BBOB test suite of 24 noiseless functions. Your task is to write the optimization algorithm in Python code. The code should contain an `__init__(self, budget, dim)` function and the function `def __call__(self, func)`, which should optimize the black box function `func` using `self.budget` function evaluations.
The func() can only be called as many times as the budget allows, not more. Each of the optimization functions has a search space between -5.0 (lower bound) and 5.0 (upper bound). The dimensionality can be varied.
Give an excellent and novel heuristic algorithm to solve this task and also give it a one-line description with the main idea.
"""
""")

for experiment_i in [1]:
# A 1+1 strategy
Expand Down
Loading
Loading