deepinfra.md

DeepInfra

DeepInfra is a platform that allows users to deploy machine learning models. They offer a variety of models, including text-generation, text-to-image, and automatic speech recognition. Users can pay per use for the models they deploy. DeepInfra offers both custom models and pre-trained models. Pre-trained models include openchat/openchat-3.6-8b, nvidia/Nemotron-4-340B-Instruct, and microsoft/WizardLM-2-7B.

Interface Name

• deepinfra

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'deepinfra': process.env.DEEPINFRA_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('deepinfra', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

• default: openchat/openchat-3.6-8b
• large: nvidia/Nemotron-4-340B-Instruct
• small: microsoft/WizardLM-2-7B
• agent: Qwen/Qwen2-7B-Instruct

Embeddings Model Aliases

• default: BAAI/bge-base-en-v1.5
• large: BAAI/bge-large-en-v1.5
• small: BAAI/bge-base-en-v1.5

Options

The following parameters can be passed through options.

• echo: Details not available, please refer to the LLM provider documentation.
• frequency_penalty: Details not available, please refer to the LLM provider documentation.
• max_tokens: Details not available, please refer to the LLM provider documentation.
• n: Details not available, please refer to the LLM provider documentation.
• presence_penalty: Details not available, please refer to the LLM provider documentation.
• response_format: Details not available, please refer to the LLM provider documentation.
• stop: Details not available, please refer to the LLM provider documentation.
• stream: Details not available, please refer to the LLM provider documentation.
• temperature: Details not available, please refer to the LLM provider documentation.
• tool_choice: Details not available, please refer to the LLM provider documentation.
• tools: Details not available, please refer to the LLM provider documentation.
• top_p: Details not available, please refer to the LLM provider documentation.

Features

• Native JSON Mode
• Streaming
• Tools
• Embeddings

Getting an API Key

Commercial with Free Trial: The DeepInfra API is a commercial product, but new accounts start with a $1.80 credit.

To get an API key, first create a DeepInfra account, then visit the link below.

• https://deepinfra.com/dash/api_keys

DeepInfra Documentation

DeepInfra documentation is available here.

@DeepInfra

Anthropic