Skip to content

Latest commit

 

History

History
74 lines (48 loc) · 2.69 KB

groq.md

File metadata and controls

74 lines (48 loc) · 2.69 KB

Groq

Groq is a company that develops hardware and software for accelerating artificial intelligence and machine learning workloads. They specialize in creating Tensor Streaming Processor (TSP) architecture, which is designed to optimize the performance and efficiency of AI computations. Groq's technology aims to deliver high performance and low latency for various applications, such as natural language processing, computer vision, and recommendation systems. The company's focus on hardware acceleration distinguishes them in the field of AI infrastructure providers.

Interface Name

  • groq

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'groq': process.env.GROQ_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('groq', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: llama3-8b-8192
  • large: llama3-70b-8192
  • small: gemma-7b-it
  • agent: llama3-8b-8192

Options

The following parameters can be passed through options.

  • frequency_penalty: Details not available, please refer to the LLM provider documentation.
  • function_call: Details not available, please refer to the LLM provider documentation.
  • functions: Details not available, please refer to the LLM provider documentation.
  • logit_bias: Details not available, please refer to the LLM provider documentation.
  • max_tokens: Details not available, please refer to the LLM provider documentation.
  • n: Details not available, please refer to the LLM provider documentation.
  • presence_penalty: Details not available, please refer to the LLM provider documentation.
  • stop: Details not available, please refer to the LLM provider documentation.
  • stream: Details not available, please refer to the LLM provider documentation.
  • temperature: Details not available, please refer to the LLM provider documentation.
  • top_p: Details not available, please refer to the LLM provider documentation.
  • user: Details not available, please refer to the LLM provider documentation.

Features

  • Streaming
  • Functions

Getting an API Key

Free: The Groq API is currently free to use.

To get an API key, first create a Groq account, then visit the link below.

Groq Documentation

Groq documentation is available here.