PDL obj as BlockType

[eloycoto]

I'm trying to do an abstraction of my LLM calls using function, for that I do this:

description: TEST
defs:
  model_call_with_tools:
    function:
      model: str
      temperature: int
      messages: obj
    return:
      lastOf:
      - model: openai/llama3.2:3b
        input: ${ messages }
        parameters:
          temperature: ${ temperature }

So, I can use like this:

  - call: model_call_with_tools
    args:
      model: openai/llama3.2:3b
      temperature: 0
      messages:
        array:
        - role: user
          content: Translate the word 'Hello' to French

But, I have the following issues when using an obj for messages, it doesn't parse as BlockType in the ast.

$ ->  pdl test.pdl
{"array": [{"role": "user", "content": "Translate the word 'Hello' to French"}]}
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.


Provider List: https://docs.litellm.ai/docs/providers


Provider List: https://docs.litellm.ai/docs/providers


Provider List: https://docs.litellm.ai/docs/providers

test.pdl:16 - Error during model call: litellm.APIConnectionError: APIConnectionError: OpenAIException - string indices must be integers, not 'str'

Full example:

description: TEST
defs:
  test:
    function:
      messages : obj
    return:
      "${ messages }"

  model_call_with_tools:
    function:
      model: str
      temperature: int
      messages: obj
    return:
      lastOf:
      - model: openai/llama3.2:3b
        input: ${ messages }
        parameters:
          temperature: ${ temperature }

text:
  - call: test
    args:
      messages:
        array:
        - role: user
          content: Translate the word 'Hello' to French

  - call: model_call_with_tools
    args:
      model: openai/llama3.2:3b
      temperature: 0
      messages:
        array:
        - role: user
          content: Translate the word 'Hello' to French

[eloycoto]

This works, got confused by the "array" thing in the input, fixed, and with this documented :-)

description: TEST
defs:
  test:
    function:
      messages : obj
    return:
      "${ messages }"

  model_call_with_tools:
    function:
      model: str
      temperature: int
      messages: list
    return:
      lastOf:
      - model: openai/llama3.2:3b
        input: ${ messages }
        parameters:
          temperature: ${ temperature }

text:

  - call: model_call_with_tools
    args:
      model: openai/llama3.2:3b
      temperature: 0
      messages:
      - role: system
        content: "You're a helpful assintant"
      - role: user
        content: Translate the word 'Hello' to French

is there any way to define the array insde the args?

[vazirim]

Hi @eloycoto, yes the above is the correct way. In the first example, the array definition was being interpreted as an object, so that defined an object with field array.

In the above example, in the model_call_with_tools you may want to use ${ model } as well when making the call. Also you could get rid of lastOf since there is only one element in the list:

  model_call_with_tools:
    function:
      model: str
      temperature: int
      messages: list
    return:
      model: ${ model }
      input: ${ messages }
      parameters:
        temperature: ${ temperature }

[eloycoto]

More or less I ended in something like this:

  model_call_with_tools:
    function:
      model: str
      temperature: int
      messages: list
      tools: list
    return:
      lastOf:
        - def: finished
          text: "no"
          contribute: []

        - for:
            itemX: ${messages}
          repeat:
            text:
              - def: "parse_messages"
                contribute:
                  - context:
                      value:
                      - role: "${itemX.role}"
                        content: "${ itemX.content }"
        - repeat:
            lastOf:
            - model: ${ model }
              modelResponse: out
              contribute: []
              parameters:
                temperature: ${ temperature }
                tools: ${tools}

            - def: finished
              contribute: []
              if: ${ out.choices[0].finish_reason == 'stop'}
              then:
                text: "yes"

            - for:
                item: ${out.choices[0].message.tool_calls}
              repeat:
                text:
                  - def: tool_arguments
                    text: ${ item.function.arguments }
                    parser: json
                    contribute: []

                  - def: functionRes
                    call: ${item.function.name}
                    args: "${tool_arguments}"
                    contribute:
                    - context:
                        value:
                          - role: "tool"
                            tool_call_id: "${ item.id }"
                            name: "${ item.function.name }"
                            content: "${ functionRes }"

          until: ${ finished == 'yes'}
          contribute: []

        - "The result is ${finished}"

But for some reason I'm getting this, I tried with join: as: lastOf and not sure why this it fails, but I'll figure it out tomorrow!

eloy@cadelo:~/dev/upstream/eloycoto/eloy-gym$ pdl --stream=none test.pdl
test.pdl:25 - Values inside the For block must be lists.

[eloycoto]

Ok, at the end, the PDL which implement the function to call the models with tools that can be reused looks like this:

description: TEST
defs:
  test:
    function:
      messages : obj
    return:
      "${ messages }"

  retrieve_stock_for_item:
    function:
      product_name: str
    return:
      lang: python
      code: |
        import random

        san_francisco_stock = random.randint(1, 100)
        palo_alto_stock = random.randint(1, 30)
        mountain_view_stock = random.randint(1, 60)

        inventory_report = (
          f"The current stock in the stores for {product_name} is the following: \n"
          f"- {san_francisco_stock} devices in San Francisco\n"
          f"- {palo_alto_stock} devices in Palo Alto\n"
          f"- {mountain_view_stock} devices in Mountain view store\n"
        )
        result = inventory_report;

  model_call_with_tools:
    function:
      model: str
      temperature: int
      messages: list
      tools: list
    return:
      lastOf:
        # This is to send the messages to the model context, so the loop after
        # always pick the initial messages
        - for:
            msg: ${messages}
          repeat:
            text:
              - def: "parse_messages"
                contribute:
                  - context:
                      value:
                      - role: "${msg.role}"
                        content: "${ msg.content }"

        - def: finished
          contribute: []
          text: "false"

        - repeat:
            text:
            - model: ${ model }
              modelResponse: out
              contribute: []
              parameters:
                temperature: ${ temperature }
                tools: ${tools}

            - if: ${ out.choices[0].finish_reason == 'tool_calls'}
              then:
                text:
                  - for:
                      item: ${ out.choices[0].message.tool_calls }
                    repeat:
                      text:
                        - def: tool_arguments
                          text: ${ item.function.arguments }
                          parser: json
                          contribute: []

                        - def: functionRes
                          call: ${ item.function.name }
                          args: "${ tool_arguments }"
                          contribute:
                          - context:
                              value:
                                - role: "tool"
                                  tool_call_id: "${ item.id }"
                                  name: "${ item.function.name }"
                                  content: "${ functionRes }"

            - if: ${ out.choices[0].finish_reason == 'stop'}
              then:
                text:
                  - def: finished
                    contribute: []
                    text: "true"
                  # We need to contribute to the result the lastest message
                  # just to be used by the function results
                  - "${out.choices[0].message.content}"
          until: ${ finished == "true" }
text:
  - call: model_call_with_tools
    args:
      model: openai/llama3.2:3b
      temperature: 0
      messages:
      - role: system
        content: "You're a helpful assintant"
      - role: user
        content: "Do you have any stock for Iphone 8 or Google Pixel 9?"
      tools:
      - type: function
        function:
          name: retrieve_stock_for_item
          description: Get the stock information for the given product
          parameters:
            type: object
            properties:
              product_name:
                type: string
                description: The product name to check the stock.
            required:
              - product_name