Skip to content

Commit 3fd3939

Browse files
authored
Stdlib llm_as_judge changes to accept parameters (#1370)
Signed-off-by: Mandana Vaziri <[email protected]>
1 parent 5efe54d commit 3fd3939

File tree

1 file changed

+14
-5
lines changed

1 file changed

+14
-5
lines changed

src/pdl/pdl_stdlib.pdl

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,17 +23,26 @@ defs:
2323
function:
2424
model: string
2525
prompt: string
26+
parameters: {optional: object}
2627
return:
28+
defs:
29+
parameters: ${ parameters | default({}) }
30+
params:
31+
data:
32+
logprobs: true
33+
top_logprobs: 5
34+
response_format: {'type': 'json_schema', 'json_schema': {'name': 'schema', 'schema': {'enum': [True, False]}, 'strict': True}}
2735
lastOf:
36+
- lang: python
37+
def: final_parameters
38+
code: |
39+
result = parameters | params
2840
- model: ${ model }
41+
def: evaluation
2942
input: |
3043
${ prompt }
3144
modelResponse: out
32-
parameters:
33-
temperature: 0
34-
logprobs: true
35-
top_logprobs: 5
36-
response_format: {'type': 'json_schema', 'json_schema': {'name': 'schema', 'schema': {'enum': [True, False]}, 'strict': True}}
45+
parameters: ${ final_parameters }
3746
- def: score
3847
data: ${ reward(response=out) }
3948
retry: 3

0 commit comments

Comments
 (0)