Skip to content

Latest commit

 

History

History
73 lines (44 loc) · 2.37 KB

anyscale.md

File metadata and controls

73 lines (44 loc) · 2.37 KB

Anyscale

Anyscale

Anyscale is a leading AI platform that enables developers and AI teams to build, deploy, and scale AI applications with unmatched efficiency. Built on the Ray open-source framework, Anyscale offers a fully managed platform with capabilities like orchestration, experiment management, and hyperparameter tuning. Anyscale is used by thousands of organizations to accelerate their AI development, providing a seamless experience from laptop to production across diverse AI workloads.

Interface Name

  • anyscale

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'anyscale': process.env.ANYSCALE_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('anyscale', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: mistralai/Mixtral-8x22B-Instruct-v0.1
  • large: meta-llama/Llama-3-70b-chat-hf
  • small: mistralai/Mistral-7B-Instruct-v0.1
  • agent: mistralai/Mixtral-8x22B-Instruct-v0.1

Embeddings Model Aliases

  • default: thenlper/gte-large
  • large: thenlper/gte-large
  • small: BAAI/bge-large-en-v1.5

Options

The following parameters can be passed through options.

  • frequency_penalty: Details not available, please refer to the LLM provider documentation.
  • max_tokens: Details not available, please refer to the LLM provider documentation.
  • temperature: Details not available, please refer to the LLM provider documentation.
  • top_p: Details not available, please refer to the LLM provider documentation.

Features

  • Embeddings

Getting an API Key

Commercial with Free Trial: The Anyscale API does not require a credit card and comes with $10 credit to get started.

To get an API key, first create an Anyscale account, then visit the link below.

Anyscale Documentation

Anyscale documentation is available here.