docs: Add comprehensive instruction_proposer documentation and examples for GEPA #8775

andressrg · 2025-09-06T07:59:40Z

Summary

Resolves #8767 - Adds comprehensive documentation and working examples for GEPA's instruction_proposer parameter.

What's Changed

Documentation Enhancements

Added complete custom instruction_proposer section to docs/docs/api/optimizers/GEPA.md covering what
instruction_proposer is, when to use custom implementations, available options, and step-by-step implementation
guidance

Working Examples Added

Basic Word Limit Proposer - Demonstrates constraint satisfaction using dspy.Refine
RAG-Enhanced Instruction Proposer - Advanced example showing documentation retrieval with pattern analysis
Best practices section - Comprehensive guidelines for creating effective custom proposers

Key Features

✅ All examples use native DSPy components (dspy.Module, dspy.Signature, dspy.ChainOfThought)
✅ Proper handling of multimodal inputs (dspy.Image)
✅ Examples show batch feedback analysis advantages over individual metric evaluation
✅ Updated to use current LM configuration patterns (dspy.LM)
✅ Accurate reflective_dataset structure documentation

Testing

All code examples validated for syntactic correctness
Examples align with existing GEPA implementation patterns
Documentation follows established DSPy style guidelines

Files Modified

docs/docs/api/optimizers/GEPA.md - Added comprehensive instruction_proposer section (~200 lines)

andressrg · 2025-09-06T08:00:17Z

@LakshyAAAgrawal wip

will add the docstring updates in another commit

please LMK if you have feedback so far in the docs

LakshyAAAgrawal · 2025-09-06T17:42:14Z

docs/docs/api/optimizers/GEPA.md

+
+### Default Implementation
+
+By default, GEPA uses the built-in instruction proposer from the [GEPA library](https://github.com/gepa-ai/gepa), which implements the `InstructionProposalSignature`. This default proposer:


Link to InstructionProposalSignature

LakshyAAAgrawal · 2025-09-06T17:43:13Z

docs/docs/api/optimizers/GEPA.md

+
+- **Multi-modal handling**: Process images (dspy.Image) alongside textual information in your inputs
+- **Nuanced control on limits and length constraints**: Have more fine-grained control over instruction length, format, and structural requirements
+- **Domain-specific information**: Inject specialized knowledge, terminology, or context that the default proposer lacks


and cannot be provided via feedback_func. Highlight that this is an advanced feature, and most users shouldn't need to use this.

LakshyAAAgrawal · 2025-09-06T17:43:48Z

docs/docs/api/optimizers/GEPA.md

+- **Nuanced control on limits and length constraints**: Have more fine-grained control over instruction length, format, and structural requirements
+- **Domain-specific information**: Inject specialized knowledge, terminology, or context that the default proposer lacks
+- **Provider-specific prompting guides**: Optimize instructions for specific LLM providers (OpenAI, Anthropic, etc.) with their unique formatting preferences
+- **Coupled component updates**: Handle situations where 2 or more components need to be updated together in a coordinated manner, rather than optimizing each component independently


Add "(Refer to section X for component selection)"

Sure. Will fix this after rebasing with #8765

LakshyAAAgrawal · 2025-09-06T17:46:27Z

docs/docs/api/optimizers/GEPA.md

+)
+```
+
+We invite community contributions of new instruction proposers for specialized domains as the [GEPA library](https://github.com/gepa-ai/gepa) continues to grow. The GEPA repository contains multiple adapters (like `DefaultAdapter`, `DSPyFullProgramAdapter`, `TerminalBenchAdapter`, etc.) that, while not instruction proposers themselves, provide good examples and inspiration for implementing your own instruction proposer.


Let's not bring in adapters here.

LakshyAAAgrawal · 2025-09-06T17:47:29Z

docs/docs/api/optimizers/GEPA.md

+```python
+def __call__(
+    self,
+    candidate: dict[str, str],           # Current component name -> instruction mapping


Drop "Current". Or maybe say candidate to be updated in this round.

LakshyAAAgrawal · 2025-09-06T17:47:58Z

docs/docs/api/optimizers/GEPA.md

+def __call__(
+    self,
+    candidate: dict[str, str],           # Current component name -> instruction mapping
+    reflective_dataset: dict[str, list[dict[str, Any]]],  # Component -> failed examples  


These aren't just failed examples. These could be good examples too! In fact, contrasting failed and good examples is how GEPA finds the right balance.

Also include the key values (Inputs, Generated Outputs, Feedback) that are present in reflective_dataset.

Good point

Took the liberty to create a TypedDict:

class ReflectiveExample(TypedDict): """ Structure of individual examples in the reflective dataset. Each example contains the predictor inputs, generated outputs, and feedback from evaluation. """ Inputs: dict[str, Any] # Predictor inputs (may include str, dspy.Image, etc.) Generated_Outputs: dict[str, Any] | str # Success: dict with output fields, Failure: error message string Feedback: str # Always a string - from metric function or parsing error message

I believe this makes it more clear. But please LMK if you prefer me to revert

LakshyAAAgrawal · 2025-09-06T17:49:15Z

docs/docs/api/optimizers/GEPA.md

+    self,
+    candidate: dict[str, str],           # Current component name -> instruction mapping
+    reflective_dataset: dict[str, list[dict[str, Any]]],  # Component -> failed examples  
+    components_to_update: list[str]      # Which components to improve


The function should return new prompts, only for these components.

LakshyAAAgrawal · 2025-09-06T17:49:57Z

docs/docs/api/optimizers/GEPA.md

+    """Given a current instruction and feedback examples, generate an improved instruction with word limit constraints."""
+
+    current_instruction = dspy.InputField(desc="The current instruction that needs improvement")
+    issues_found = dspy.InputField(desc="Feedback and issues identified from failed examples")


Similar, this isn't issues_found. There are 3 keys in "reflective_dataset" (Inputs, Generated Outputs, Feedback)

LakshyAAAgrawal · 2025-09-06T17:50:59Z

docs/docs/api/optimizers/GEPA.md

+
+    improved_instruction = dspy.OutputField(desc="A new instruction that fixes the issues while staying under the word limit")
+
+class WordLimitedInstructionImprover(dspy.Module):


This is great, but seems too complicated for docs. Let's move this to instruction proposal library, but here, let's just have dspy.CoT(signature)

good point.

Simplified.

LakshyAAAgrawal · 2025-09-06T17:52:13Z

docs/docs/api/optimizers/GEPA.md

+
+**Best Practices:**
+- **Use the full power of DSPy**: Leverage DSPy components like `dspy.Module`, `dspy.Signature`, and `dspy.Predict` to create your instruction proposer rather than direct LM calls. Consider `dspy.Refine` for constraint satisfaction, `dspy.ChainOfThought` for complex reasoning tasks, and compose multiple modules for sophisticated instruction improvement workflows
+- **Validate component existence**: Always validate that required components exist in `candidate` and `reflective_dataset`  


They do always exist. No need for asserts

LakshyAAAgrawal · 2025-09-06T17:53:03Z

docs/docs/api/optimizers/GEPA.md

+- **Validate component existence**: Always validate that required components exist in `candidate` and `reflective_dataset`  
+- **Enable holistic feedback analysis**: While dspy.GEPA's `GEPAFeedbackMetric` processes one (gold, prediction) pair at a time, instruction proposers receive all examples for a component in batch, enabling cross-example pattern detection and systematic issue identification.
+- **Mind data serialization**: Serializing everything to strings might not be ideal - handle complex input types (like `dspy.Image`) by maintaining their structure for better LM processing
+- **Create detailed signatures**: Write comprehensive signature docstrings with step-by-step analysis guidance and clear requirements for the instruction improvement task


Remove this line.

LakshyAAAgrawal · 2025-09-06T17:53:25Z

docs/docs/api/optimizers/GEPA.md

+- **Mind data serialization**: Serializing everything to strings might not be ideal - handle complex input types (like `dspy.Image`) by maintaining their structure for better LM processing
+- **Create detailed signatures**: Write comprehensive signature docstrings with step-by-step analysis guidance and clear requirements for the instruction improvement task
+- **Test thoroughly**: Test your custom proposer with representative failure cases
+- **Control context size**: Mind the number of examples analyzed to prevent context overflow


This too is not in control of instruction proposer.

andressrg · 2025-09-08T07:17:48Z

Feedback ready, and GEPA docstring implemented

Will also update with "component selection" when #8765 is merged

andressrg · 2025-09-08T09:06:47Z

I also have this other branch https://github.com/andressrg/dspy/tree/feat/verbosity-aware-instruction-proposer

Haven't created the PR yet because it is supposed to be stacked on top of this one.

I'm excited about the verbosity aware proposer! Hope we can merge it soon 🚀

LakshyAAAgrawal · 2025-09-09T00:40:52Z

docs/docs/api/optimizers/GEPA.md

+
+By default, GEPA uses the built-in instruction proposer from the [GEPA library](https://github.com/gepa-ai/gepa), which implements the [`ProposalFn`](https://github.com/gepa-ai/gepa/blob/main/src/gepa/core/adapter.py). This default proposer:
+
+- Uses a comprehensive prompt template that analyzes task context, inputs, outputs, and feedback


What are the tradeoffs with having this list vs actually reproducing the default prompt directly? I feel like showing the default prompt could enable the end-user to judge whether they want to use it or not. Writing it this way makes it seem like there is no need to use custom instruction proposal!

good question!

I think it does work better showing the default prompt. It might be a bit more verbose than what we had before, but it's a lot more clear / explicit.

LakshyAAAgrawal · 2025-09-10T23:07:54Z

Hi @andressrg , feel free to continue working on this branch, or create a PR with the other branch whichever is updated!

andressrg · 2025-09-11T01:21:55Z

Hi @andressrg , feel free to continue working on this branch, or create a PR with the other branch whichever is updated!

sure!
will send you updates soon

LakshyAAAgrawal · 2025-09-11T07:52:29Z

docs/docs/api/optimizers/GEPA.md

+
+**Built-in Options:**
+- **Default Proposer**: The standard GEPA instruction proposer (used when `instruction_proposer=None`). The default instruction proposer IS an instruction proposer as well! It is the most general one, that was used for the diverse experiments reported in the GEPA paper and tutorials.
+- **MultiModalInstructionProposer**: Handles `dspy.Image` inputs and structured multimodal content. This proposer has been specifically optimized for tasks that include one or more `dspy.Image` inputs.


Do you have any run with the MultiModalInstructionProposer? I would avoid writing This proposer has been specifically optimized for tasks that include one or more dspy.Image inputs. till we have at least one benchmark/result.

makes sense.

Me and the team I work with have been using it with multi image usecases, but don't have any public benchmark/results yet.

Will update

LakshyAAAgrawal · 2025-09-11T07:53:52Z

docs/docs/api/optimizers/GEPA.md

+```python
+from dspy.teleprompt.gepa.gepa_utils import ReflectiveExample
+
+def __call__(


I think this should be within a class? Or if it is a function, then it needs to be named differently. I don;'t think a top-level fn can be named call

good point.

Just made it clearer by explaining the class and function form

docs/docs/api/optimizers/GEPA.md

LakshyAAAgrawal · 2025-09-11T08:01:35Z

docs/docs/api/optimizers/GEPA.md

@@ -68,6 +68,432 @@ highest_score_achieved_per_task = new_prog.detailed_results.highest_score_achiev
 best_outputs = new_prog.detailed_results.best_outputs_valset
 ```

+## Custom Instruction Proposers


Reading through the documentation, first of all, this is superb! Very detailed, and filled with great examples.

I feel like since this is a very advanced topic, it would be better to create a separate file/location for this, so that a first time user visiting the GEPA documentation does not immediately see this. It should come up as a thing that a new GEPA user should discover after they have become comfortable with the other GEPA concepts. Can you bring out both the instruction proposer and component selection outside?

Another thing, all of the great examples of instruction proposer that you provide, should probably also live in the library so that an end user can directly import and use it!

I feel like since this is a very advanced topic, it would be better to create a separate file/location for this, so that a first time user visiting the GEPA documentation does not immediately see this. It should come up as a thing that a new GEPA user should discover after they have become comfortable with the other GEPA concepts. Can you bring out both the instruction proposer and component selection outside?

sure man!

Can you point me to a sample file / location that has "advanced" docs for one of the dspy components?

it's not clear to me where should I create it or how to call it 🤔

How about replace GEPA.md with a directory GEPA/ which contains index.md (all the current GEPA content) and "GEPA_Advanced.md"?

directory done 🚀

andressrg · 2025-09-13T01:37:14Z

@LakshyAAAgrawal new folder structure ready

Another thing, all of the great examples of instruction proposer that you provide, should probably also live in the library so that an end user can directly import and use it!

good idea!

but should we do this on a different PR?

would love to merge this one with the docs on how to use instruction_proposer and component_selector

andressrg · 2025-09-13T01:38:21Z

ah wait.

fixing the failing test

andressrg · 2025-09-13T01:43:56Z

docs/mkdocs.yml

@@ -123,7 +123,8 @@ nav:
            - BootstrapRS: api/optimizers/BootstrapRS.md
            - COPRO: api/optimizers/COPRO.md
            - Ensemble: api/optimizers/Ensemble.md
-            - GEPA: api/optimizers/GEPA.md
+            - GEPA: api/optimizers/GEPA/index.md
+            - GEPA - Advanced Features: api/optimizers/GEPA/GEPA_Advanced.md


Probably not the most elegant way to render the sidebar for the advanced features docs 🤔

But I'm not sure how another nested level will behave here

Let me know your thoughts please

If we don't like it, I can figure out how the UI looks with the extra nesting

Can you do mkdocs and check how it is rendered? https://github.com/stanfordnlp/dspy/tree/main/docs#building-docs-locally

LakshyAAAgrawal · 2025-09-13T02:43:20Z

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

+
+### Default Implementation
+
+By default, GEPA uses the built-in instruction proposer from the [GEPA library](https://github.com/gepa-ai/gepa), which implements the [`ProposalFn`](https://github.com/gepa-ai/gepa/blob/main/src/gepa/core/adapter.py). The default proposer uses this prompt template:


Link to the default proposer as well https://github.com/gepa-ai/gepa/blob/main/src/gepa/proposer/reflective_mutation/reflective_mutation.py#L53-L75

LakshyAAAgrawal · 2025-09-13T02:44:34Z

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

+- **Nuanced control on limits and length constraints**: Have more fine-grained control over instruction length, format, and structural requirements
+- **Domain-specific information**: Inject specialized knowledge, terminology, or context that the default proposer lacks and cannot be provided via feedback_func. This is an advanced feature, and most users should not need to use this.
+- **Provider-specific prompting guides**: Optimize instructions for specific LLM providers (OpenAI, Anthropic, etc.) with their unique formatting preferences
+- **Coupled component updates**: Handle situations where 2 or more components need to be updated together in a coordinated manner, rather than optimizing each component independently (refer to component_selector parameter, in Custom Component Selection section, for related functionality)


Link to the component selection section?

LakshyAAAgrawal

The PR is amazing overall. Left some minor comments. If you can build the docs and see if the sidebar renders properly, looks good to merge!

LakshyAAAgrawal · 2025-09-13T21:57:41Z

@andressrg let me know once ready to merge

andressrg · 2025-09-13T23:14:44Z

@LakshyAAAgrawal ready to merge 🚀

I think the extra indentation in the nav bar is good! Tested it and it does work.

How it looks when you are in another optimizer:

When you select overview:

Advanced features:

I've also made it so the base link /api/optimizers/GEPA/ auto redirects to /api/optimizers/GEPA/overview/ to keep backwards compatibility with links on the web.

andressrg · 2025-09-13T23:19:28Z

Ended up replacing the GEPA/index.md implementation to GEPA/overview.md because It the navbar would have looked like this:

Which IMO hides a bit the most important part of the GEPA docs (the system makes it so the GEPA title would render the index.md)

andressrg marked this pull request as draft September 6, 2025 08:00

LakshyAAAgrawal reviewed Sep 6, 2025

View reviewed changes

andressrg requested a review from LakshyAAAgrawal September 8, 2025 07:18

LakshyAAAgrawal reviewed Sep 9, 2025

View reviewed changes

andressrg force-pushed the docs-gepa-instruction-proposer branch from fb68a16 to 278e83e Compare September 11, 2025 07:32

andressrg marked this pull request as ready for review September 11, 2025 07:32

andressrg requested a review from LakshyAAAgrawal September 11, 2025 07:37

LakshyAAAgrawal reviewed Sep 11, 2025

View reviewed changes

docs/docs/api/optimizers/GEPA.md Outdated Show resolved Hide resolved

LakshyAAAgrawal reviewed Sep 11, 2025

View reviewed changes

andressrg added 3 commits September 12, 2025 18:31

Update GEPA.md docs

f81a0c1

Action feedback and introduce ReflectiveExample TypedDict

be9600a

Update GEPA docstring

50e830a

andressrg added 7 commits September 12, 2025 18:31

Replace bullets with default prompt for more clarity

877562a

Addd component_selector docs + simple test for alternating_half_selector

8e1d9e8

Small fix

7a76a2c

Refinements

cfd51b5

Remove claim until we have public results / benchmarks

c1b2ec1

Clarify __call__

d25b9da

Replace issues_found with feedback_summary

cad959b

andressrg force-pushed the docs-gepa-instruction-proposer branch from a13f673 to 3415127 Compare September 13, 2025 01:33

Move new topics into Advanced file

0a4df4f

andressrg force-pushed the docs-gepa-instruction-proposer branch from 3415127 to 0a4df4f Compare September 13, 2025 01:34

andressrg requested a review from LakshyAAAgrawal September 13, 2025 01:35

Update sidebar

0f231a1

andressrg commented Sep 13, 2025

View reviewed changes

LakshyAAAgrawal reviewed Sep 13, 2025

View reviewed changes

LakshyAAAgrawal approved these changes Sep 13, 2025

View reviewed changes

Update docs on Custom Selector protocol

0948c1c

andressrg added 2 commits September 13, 2025 16:07

Update nav bar structure

b5418d8

Small fix

79210f9

andressrg requested a review from LakshyAAAgrawal September 13, 2025 23:14


		### Default Implementation

		By default, GEPA uses the built-in instruction proposer from the [GEPA library](https://github.com/gepa-ai/gepa), which implements the `InstructionProposalSignature`. This default proposer:


		improved_instruction = dspy.OutputField(desc="A new instruction that fixes the issues while staying under the word limit")

		class WordLimitedInstructionImprover(dspy.Module):


		By default, GEPA uses the built-in instruction proposer from the [GEPA library](https://github.com/gepa-ai/gepa), which implements the [`ProposalFn`](https://github.com/gepa-ai/gepa/blob/main/src/gepa/core/adapter.py). This default proposer:

		- Uses a comprehensive prompt template that analyzes task context, inputs, outputs, and feedback

docs: Add comprehensive instruction_proposer documentation and examples for GEPA #8775

Are you sure you want to change the base?

docs: Add comprehensive instruction_proposer documentation and examples for GEPA #8775

Conversation

andressrg commented Sep 6, 2025

Summary

What's Changed

Documentation Enhancements

Working Examples Added

Key Features

Testing

Files Modified

Uh oh!

andressrg commented Sep 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andressrg Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andressrg commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andressrg commented Sep 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LakshyAAAgrawal commented Sep 10, 2025

Uh oh!

andressrg Sep 8, 2025 •

edited

Loading

andressrg commented Sep 8, 2025 •

edited

Loading

andressrg commented Sep 13, 2025 •

edited

Loading

andressrg Sep 13, 2025 •

edited

Loading