System Prompt Optimization for Curator

[RajWorking]

The GEPA paper introduced a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error.

Motivation for this integration:
right now users have to specify and iterate on system prompts when creating synthetic data.
Can we instead use GEPA to auto generate the system prompt first and then generate the data?

Credits: @madiator

[eskayML]

looks like we can leverage the GEPA paper's prompt optimizer to auto-generate system prompts. i think we can integrate the GEPA optimizer as a preprocessing step before generating synthetic data, and then use the optimized prompt to generate the data. should i open a PR for this?

[madiator]

Yes sounds good! thank you @eskayML