All pull requests

Pull requests

• feat(redteam): take agent_factory at run time in RedTeamExperiment Part 4

  area-redteamenhancement
  #426·nhungbi opened Oct 9, 2026Contributor·8 of 9 tasks·

  ·
  3

  

• feat(providers): add HoneyHiveProvider

  area-tracingenhancement
  #419·sanjeed5 opened Oct 9, 2026·9 tasks done·

  ·
  1

• feat(redteam): public as_target_session and default run_attack max_turns Part 3

  area-devxarea-redteamenhancement
  #417·nhungbi opened Oct 8, 2026Contributor·9 tasks done·

  ·
  6

  

• feat(redteam): give each strategy variant its own session_id and attach its strategy as RedTeamCase.strategy Part 2

  area-redteamenhancement
  #416·nhungbi opened Oct 8, 2026Contributor·8 of 9 tasks·

  ·
  10

  

• fix(redteam): carry run stats to the report via environment_state part of RedTeam Graduation Part 1

  area-redteam
  #415·nhungbi opened Oct 8, 2026Contributor·8 of 9 tasks·

  ·
  18

  

• test: add integration tests for output-level judge evaluators

  area-evaluatorschore
  #407·AmitKulkarni23 opened Sep 29, 2026·9 tasks done·

  ·

• feat: report judge context overflow as could-not-evaluate, not score-0

  area-corearea-evaluatorsenhancement
  #399·pdebjyot opened Sep 14, 2026Contributor·

  ·

• feat(output_evaluator): add cache_config support for prompt caching

  area-evaluatorsenhancement
  #397·shssrd opened Sep 11, 2026·

  ·
  8

• feat(tools): add KnowledgeIndex for progressive disclosure over reference knowledge

  area-evaluatorsenhancement
  #396·pdebjyot opened Sep 9, 2026Contributor·

  ·
  1

  1

• fix: expose and serialize user tools on TrajectoryEvaluator

  area-evaluatorsbug
  #394·AmirF194 opened Sep 7, 2026·8 tasks done·

  ·
  1

  1

• fix: make EvaluationReport.to_file() reject non-strict JSON

  area-corearea-devxbug
  #393·serenearyal opened Sep 4, 2026·9 tasks done·

  ·
  1

  2

• feat(providers): add session discovery + evaluate_sessions batch helper (#143)

  area-tracingenhancement
  #391·lifelmy opened Sep 1, 2026·4 tasks done·

  ·
  1

  3

• feat(store): add per-run keying and config-drift guard to EvaluationDataStore (#347)

  area-coreenhancement
  #390·lifelmy opened Sep 1, 2026·6 of 7 tasks·

  ·
  4

• feat(experiment): add ExperimentTrendAnalyzer for cross-run regression detection (#186)

  area-coreenhancement
  #389·lifelmy opened Sep 1, 2026·6 of 7 tasks·

  ·
  1

  3

• feat(classification): add classify_risk utility for multi-run results (#349)

  area-coreenhancement
  #388·lifelmy opened Aug 31, 2026·

  ·
  5

• feat(evaluators): add FuzzyEquals deterministic evaluator

  area-evaluatorsenhancement
  #387·lifelmy opened Aug 31, 2026·

  ·
  3

• ci: update langfuse requirement from <4,>=2.0.0 to >=2.0.0,<5

  area-communitychoredependenciespython
  #367·dependabot[bot] opened Aug 13, 2026Bot·

  ·
  2

• feat: add ToolEfficiencyEvaluator for session-level tool usage analysis

  area-evaluatorsenhancement
  #362·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  8

• feat: add evaluator metadata method and types

  area-devxarea-evaluatorsenhancement
  #361·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  11

• feat: add failure cohort analysis for evaluation reports

  area-corearea-evaluatorsenhancement
  #360·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  7

• feat: add status field to EvaluationOutput

  area-corearea-evaluatorsenhancement
  #359·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  9

• feat: add inclusive language evaluator

  area-evaluatorsenhancement
  #358·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  15

• feat(tools): add TraceIndex for progressive trace disclosure in judge-based evaluators

  area-evaluatorsarea-tracingenhancement
  #343·pdebjyot opened Aug 3, 2026Contributor·5 tasks done·

  ·
  24

• feat(experiment): add opt-in model routing for evaluators

  area-corearea-evaluatorsenhancement
  #325·pdebjyot opened Jul 23, 2026Contributor·4 tasks done·

  ·
  1

  10

• fix: convert Langfuse AGENT observations regardless of parent

  area-tracingbug
  #314·poshinchen opened Jul 15, 2026Contributor·7 of 9 tasks·

  ·
  1

  7