feat: add validator adapter

[tobySolutions]

Description of the pull request

Changes made

Fixes #93

• Added validator adapter for other models from our API
• Wrote tests to handle them
• ....

Related issues

Testings done

Screenshots (if any)

Checklist

• I have written tests
• My code does not produce new errors
• I gave myself a code review before asking others.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
akeru-akeru-frontend	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 2, 2024 11:18pm

accidental approval

[GuiBibeau]

@tobySolutions Great work on this! I'm loving it.

@multipletwigs can you check that the code in run thread is aligned with your architecture of that code?

[tobySolutions]

@tobySolutions Great work on this! I'm loving it.

@multipletwigs can you check that the code in run thread is aligned with your architecture of that code?

Thanks Gui, really appreciate it!!!

[multipletwigs]

@tobySolutions i'm actually a bit confused on what we're trying to achieve with this, since Validators and Assistants are inherently different domains.

  if (assistantData.model === "llama-2-7b-chat-int8") {
    const validatorAdapterRes: any = await validatorAdapter(
      everyRoleAndContent,
      "llama-2-7b-chat-int8",
      assistantData.instruction
    );

    const assistantResponse: string =
      validatorAdapterRes.choices[0].message.content;

    // add assistant response to the thread
    await createMessage(assistant_id, thread_id, assistantResponse);

    const threadRunResponse: ThreadRun = {
      id: uuidv4(),
      assistant_id: assistant_id,
      thread_id: thread_id,
      created_at: new Date(),
    };
    return threadRunResponse;
  }
}

This can get messy very quickly when we have a lot of models to validate.

@GuiBibeau In a way yes, the architecture is kinda how a thread run should be. I ran through the validator code, and it seems like the validator will take in the original Thread content, find the fastest miner, and make a request to it.

Just to confirm my understanding, Akeru can make a request to a Validator, and ask "hey i have a thread with these messages, can anyone give me a response", and Validators will see which Miner to request from, and give the response back to Akeru. Which is different from our existing Assistants right? So it makes sense to have two different domains anyways for it.

[multipletwigs]

If we were to still make Bittensor validators as part of our assistants, then the best course is to just differentiate the models with bt-<insert model name> with a bt prefix, and call the validator adapter instead. Include model data as part of the api call request, and have the Validator decide what to do with it. We shouldn't check when to call Validator models with a bunch of if statements.

[GuiBibeau]

@tobySolutions i'm actually a bit confused on what we're trying to achieve with this, since Validators and Assistants are inherently different domains.

  if (assistantData.model === "llama-2-7b-chat-int8") {
    const validatorAdapterRes: any = await validatorAdapter(
      everyRoleAndContent,
      "llama-2-7b-chat-int8",
      assistantData.instruction
    );

    const assistantResponse: string =
      validatorAdapterRes.choices[0].message.content;

    // add assistant response to the thread
    await createMessage(assistant_id, thread_id, assistantResponse);

    const threadRunResponse: ThreadRun = {
      id: uuidv4(),
      assistant_id: assistant_id,
      thread_id: thread_id,
      created_at: new Date(),
    };
    return threadRunResponse;
  }
}

This can get messy very quickly when we have a lot of models to validate.

@GuiBibeau In a way yes, the architecture is kinda how a thread run should be. I ran through the validator code, and it seems like the validator will take in the original Thread content, find the fastest miner, and make a request to it.

Just to confirm my understanding, Akeru can make a request to a Validator, and ask "hey i have a thread with these messages, can anyone give me a response", and Validators will see which Miner to request from, and give the response back to Akeru. Which is different from our existing Assistants right? So it makes sense to have two different domains anyways for it.

@tobySolutions @multipletwigs just to confirm I love the idea of prefixing the models coming from the subnet with something like bt.

On the main issues at hand: the idea would be that the validator/miner combo can act fully on the same level as the gpt-3.5/4.0 adapter. So if somebody creates an assistant with ex: bt-mistral-7b. All prompts and thread runs would be fulfilled by bt-mistral-7b in the same way that gpt-3.5 backed assistants are fulfilled by open-ai. The subnet becomes an external provider of inference on the same footing as existing adapters.

Does that make sense?

[multipletwigs]

@GuiBibeau in that case, yes this makes sense now! So the Models available should be bt-gpt-4, bt-gpt-3.5, gpt-4, gpt-3.5 respectively and if it's a bt model, run it by the miner/validator adapter, else run it through the conventional adapters. This is more manageable i think.

[tobySolutions]

Made updates with all your feedback @multipletwigs!! Thanks for everything.

[multipletwigs]

LGTM

[tobySolutions]

LGTM

Awesome, thanks!