Configuring OpenAI models

[IhsanAhmedHaque]

Hi, is there a way to pass a model e.g. gpt-4o via user interface

[IhsanAhmedHaque]

@RamiAwar please help

[RamiAwar]

This isn't currently supported or exposed, reason being that no other model has passed our evaluation pipeline like gpt-3.5-turbo has, which means it's not worth working on right now.

What is the current limitation you're facing with the default configured models?

[IhsanAhmedHaque]

Thank you @RamiAwar, I couldn't get the GPT-4o to work with dataline, even after changing the config.py on the backend it is still trying to use GPT-3.5-turbo as per the error as it says that GPT-3.5-turbo does not have access to project id SP xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx. This is because we have tested GPT-4o on postman and works well. I understand on using GPT-3.5-turbo which is recommended and will work towards to get gpt-3.5-turbo. Also for me to try out could you please advice on where I should change in code for the model to actually pickup the got-4o model please.

[RamiAwar]

Hey Ihsan, there's a few things you need to change to support other models in the code in a hacky way, but it's hard to describe you just have to go through it.

I don't have time to support such side-efforts sadly but I can tell you again that I've tried most models, and so far they're not as good. You'll see a lot of hallucination, errors, etc. I strongly recommend you stick to gpt-3.5-turbo, it's also super cheap.

We'll get there, but we're not there yet. LLMs will still improve a lot. See if you can already use DataLine for your needs as it is, and it will inevitably become 10x better in a year ;)

[yasirali0]

@IhsanAhmedHaque You can try one the following two ways to use another LLM model e.g., gpt-4o, instead of the Dataline's default gpt-3.5-turbo.

1. This line in the conversation service provides the name of the LLM model to the LLM workflow.
   

   dataline/backend/dataline/services/conversation.py

   Line 167 in 51282de

   llm_model=user_with_model_details.preferred_openai_model,

   You can change this to: llm_model="gpt-4o",

2. You can change the following:
   

   dataline/backend/dataline/config.py

   Line 34 in 51282de

   default_model: str = "gpt-3.5-turbo"

   to: default_model: str = "gpt-4o"
   And then change the following:
   

   dataline/backend/dataline/services/settings.py

   Line 140 in 51282de

   user_info.preferred_openai_model = user_info.preferred_openai_model or config.default_model

   to: user_info.preferred_openai_model = config.default_model

Note: I haven't tested any of these two methods, so I'm not sure if they will work.
P.S. The chart generation tool of the LLM workflow and conversation title generation use mirascope calls, so they are not affected by the above changes.
Even after above changes, the chart generation tool will still use gpt-3.5-turbo, and the conversation title generation will keep using gpt-4o-mini.

If you also want to change the LLM model for them, you can do so as follows:
First, add gpt-4o as the available model:

dataline/backend/dataline/services/llm_flow/llm_calls/mirascope_utils.py

Line 14 in 51282de

AvailableModels = Literal["gpt-3.5-turbo"] | Literal["gpt-4o-mini"]

AvailableModels = Literal["gpt-3.5-turbo"] | Literal["gpt-4o-mini"] | Literal["gpt-4o"]

For chart generation tool, change the LLM model name in the following line:

dataline/backend/dataline/services/llm_flow/toolkit.py

Line 517 in 51282de

"gpt-3.5-turbo",

For conversation title generation, you can change the LLM model name here (which might not be necessary, as gpt-4o-mini is quite capable for such a small task):

dataline/backend/dataline/services/conversation.py

Line 88 in 51282de

"gpt-4o-mini",

I hope it helps.