Merge pull request #5 from samestrin/2.0.7

2.0.7

Author: samestrin

README.md

@@ -2,27 +2,33 @@
 
 [![Star on GitHub](https://img.shields.io/github/stars/samestrin/llm-interface?style=social)](https://github.com/samestrin/llm-interface/stargazers) [![Fork on GitHub](https://img.shields.io/github/forks/samestrin/llm-interface?style=social)](https://github.com/samestrin/llm-interface/network/members) [![Watch on GitHub](https://img.shields.io/github/watchers/samestrin/llm-interface?style=social)](https://github.com/samestrin/llm-interface/watchers)
 
-![Version 2.0.6](https://img.shields.io/badge/Version-2.0.6-blue) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Built with Node.js](https://img.shields.io/badge/Built%20with-Node.js-green)](https://nodejs.org/)
+![Version 2.0.7](https://img.shields.io/badge/Version-2.0.7-blue) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Built with Node.js](https://img.shields.io/badge/Built%20with-Node.js-green)](https://nodejs.org/)
 
 ## Introduction
 
-`llm-interface` is a wrapper designed to interact with multiple Large Language Model (LLM) APIs. `llm-interface` simplifies integrating various LLM providers, including **OpenAI, AI21 Studio, Anthropic, Cloudflare AI, Cohere, Fireworks AI, Google Gemini, Goose AI, Groq, Hugging Face, Mistral AI, Perplexity, Reka AI, watsonx.ai, and LLaMA.cpp**, into your applications. It is available as an [NPM package](https://www.npmjs.com/package/llm-interface).
+`llm-interface` is a wrapper designed to interact with multiple Large Language Model (LLM) APIs. `llm-interface` simplifies integrating various LLM providers, including **OpenAI, AI21 Studio, Anthropic, Cloudflare AI, Cohere, DeepInfra, Fireworks AI, Friendli AI, Google Gemini, Goose AI, Groq, Hugging Face, Mistral AI, Monster API, Octo AI, Perplexity, Reka AI, watsonx.ai, and LLaMA.cpp (ollama compatible)**, into your applications. It is available as an [NPM package](https://www.npmjs.com/package/llm-interface).
 
 This goal of `llm-interface` is to provide a single, simple, unified interface for sending messages and receiving responses from different LLM services. This will make it easier for developers to work with multiple LLMs without worrying about the specific intricacies of each API.
 
 ## Features
 
-- **Unified Interface**: `LLMInterfaceSendMessage` is a single, consistent interface to interact with **fifteen** different LLM APIs.
+- **Unified Interface**: `LLMInterfaceSendMessage` is a single, consistent interface to interact with **19 different LLM APIs**.
 - **Dynamic Module Loading**: Automatically loads and manages LLM interfaces only when they are invoked, minimizing resource usage.
 - **Error Handling**: Robust error handling mechanisms to ensure reliable API interactions.
 - **Extensible**: Easily extendable to support additional LLM providers as needed.
 - **Response Caching**: Efficiently caches LLM responses to reduce costs and enhance performance.
 - **Graceful Retries**: Automatically retry failed prompts with increasing delays to ensure successful responses.
 - **JSON Output**: Simple to use native JSON output for OpenAI, Fireworks AI, and Gemini responses.
-- **JSON Repair**: Detect and repair invalid JSON responses.
+- **JSON Repair**: Detect and repair invalid JSON responses. 
 
 ## Updates
 
+**v2.0.7**
+
+- **New LLM Providers**: Added support for DeepInfra, FriendliAI, Monster API, Octo AI, Together AI, and NVIDIA.
+- **Improved Test Coverage**: New DeepInfra, FriendliAI, Monster API, NVIDIA, Octo AI, Together AI, and watsonx.ai test cases.
+- **Refactor**: Improved support for OpenAI compatible APIs using new BaseInterface class.
+
 **v2.0.6**
 
 - **New LLM Provider**: Added support for watsonx.ai.
@@ -31,16 +37,6 @@ This goal of `llm-interface` is to provide a single, simple, unified interface f
 
 - **New LLM Providers Functions**: `LLMInterface.getAllModelNames()` and `LLMInterface.getModelConfigValue(provider, configValueKey)`.
 
-**v2.0.2**
-
-- **New LLM Providers**: Added support for Cloudflare AI, and Fireworks AI.
-- **JSON Consistency**: A breaking change has been introduced: all responses now return as valid JSON objects.
-- **JSON Repair**: Use `interfaceOptions.attemptJsonRepair` to repair invalid JSON responses when they occur.
-- **Improved Hugging Face Interface**: Refactored interface to support the undocumented chat completion endpoint.
-- **Interface Name Changes**:`reka` becomes `rekaai`, `goose` becomes `gooseai`, `mistral` becomes `mistralai`.
-- **Deprecated**: `handlers` has been removed.
-- **Updated LLM Model Definitions**: Revised `small` models for various providers.
-
 ## Dependencies
 
 The project relies on several npm packages and APIs. Here are the primary dependencies:
@@ -120,12 +116,14 @@ npm test
 #### Test Results (v2.0.0)
 
 ```bash
-Test Suites: 46 passed, 46 total
-Tests:       185 passed, 185 total
+Test Suites: 52 passed, 52 total
+Tests:       2 skipped, 215 passed, 217 total
 Snapshots:   0 total
-Time:        61.064 s, estimated 64 s
+Time:        76.236 s
 ```
 
+_Note: Currently skipping NVIDIA test cases due to API key limits._
+
 ## Contribute
 
 Contributions to this project are welcome. Please fork the repository and submit a pull request with your changes or improvements.

docs/APIKEYS.md

@@ -32,12 +32,22 @@ The Cohere API offers trial keys. Trial keys are rate-limited, and cannot be use
 
 - https://dashboard.cohere.com/api-keys
 
+## Deepinfra
+
+The Deepinfra API is commercial but new accounts will start with a $1.80 credit.
+
 ## Fireworks AI
 
 The Fireworks AI API offers a free developer tier and commercial accounts. A Credit is not required for the free developer tier.
 
 - https://fireworks.ai/api-keys
 
+## Friendli AI
+
+The Friendli AI API is commercial but it comes with a $5.00 credit.
+
+- https://suite.friendli.ai/user-settings/tokens
+
 ## Gemini
 
 The Gemini API is currently free.
@@ -68,6 +78,26 @@ The MistralAI API is a commercial product, but it currently does not require a c
 
 - https://console.mistralai.ai/api-keys/
 
+## Monster API
+
+The Monster API is commercial but it comes with a free tier. You do not need to provide a credit card to get started.
+
+- https://monsterapi.ai/user/dashboard
+
+## NVIDIA
+
+The NVIDIA API comes with 1000 credits, however they run out fast. To get an API key, first navigate to a model like:
+
+- https://build.nvidia.com/meta/llama3-70b
+
+Then click "Get API Key" on the right side of the page.
+
+## Octo AI
+
+The Octo AI API is commercial, but it comes with a $5.00 credit, and does not require a credit card.
+
+- https://octoai.cloud/settings
+
 ## Perplexity
 
 The Perplexity API requires a credit cards.
@@ -76,10 +106,16 @@ The Perplexity API requires a credit cards.
 
 ## Reka AI
 
-The Reka AI API requires a credit card, but currently comes with a $5 credit.
+The Reka AI API requires a credit card, but currently comes with a $5.00 credit.
 
 - https://platform.reka.ai/apikeys
 
+## Together AI
+
+The Together API is commercial, but it did not require a credit card, and it came with a $5.00 credit.
+
+- https://api.together.xyz/settings/api-keys
+
 ## watsonx.ai
 
 The watsonx.ai API is a commercial service, but it offers a free tier of service without requiring a credit card.

env

@@ -11,4 +11,14 @@ AI21_API_KEY=
 FIREWORKSAI_API_KEY=
 CLOUDFLARE_API_KEY=
 CLOUDFLARE_ACCOUNT_ID=
-LLAMACPP_URL=http://localhost:8080/completions
+LLAMACPP_URL=http://localhost:8080/completions
+CLOUDFLARE_API_KEY=
+CLOUDFLARE_ACCOUNT_ID=
+WATSONXSAI_API_KEY=
+WATSONXSAI_SPACE_ID=
+FRIENDLIAI_API_KEY=
+NVIDIA_API_KEY=
+DEEPINFRA_API_KEY=
+TOGETHERAI_API_KEY=
+MONSTERAPI_API_KEY=
+OCTOAI_API_KEY=

src/config/config.js

@@ -24,4 +24,9 @@ module.exports = {
   watsonxaiApiKey: process.env.WATSONXSAI_API_KEY,
   watsonxaiSpaceId: process.env.WATSONXSAI_SPACE_ID,
   friendliaiApiKey: process.env.FRIENDLIAI_API_KEY,
+  nvidiaApiKey: process.env.NVIDIA_API_KEY,
+  deepinfraApiKey: process.env.DEEPINFRA_API_KEY,
+  togetheraiApiKey: process.env.TOGETHERAI_API_KEY,
+  monsterapiApiKey: process.env.MONSTERAPI_API_KEY,
+  octoaiApiKey: process.env.OCTOAI_API_KEY,
 };

src/config/llmProviders.json

@@ -136,16 +136,16 @@
       },
       "small": {
         "name": "accounts/fireworks/models/phi-3-mini-128k-instruct",
-        "tokens": 128000
+        "tokens": 4096
       }
     }
   },
   "friendliai": {
     "url": "https://inference.friendli.ai/v1/chat/completions",
     "model": {
-      "default": { "name": "mixtral-8x7b-instruct-v0-1", "tokens": 32768 },
+      "default": { "name": "mixtral-8x7b-instruct-v0-1", "tokens": 4096 },
       "large": { "name": "meta-llama-3-70b-instruct", "tokens": 8192 },
-      "small": { "name": "mistral-7b-instruct-v0-2", "tokens": 4096 }
+      "small": { "name": "meta-llama-3-8b-instruct", "tokens": 4096 }
     }
   },
   "watsonxai": {
@@ -155,5 +155,63 @@
       "large": { "name": "meta-llama/llama-3-70b-instruct", "tokens": 8192 },
       "small": { "name": "google/flan-t5-xxl", "tokens": 512 }
     }
+  },
+  "nvidia": {
+    "url": "https://integrate.api.nvidia.com/v1/chat/completions",
+    "model": {
+      "default": { "name": "nvidia/llama3-chatqa-1.5-8b", "tokens": 4096 },
+      "large": { "name": "nvidia/nemotron-4-340b-instruct", "tokens": 4096 },
+      "small": { "name": "microsoft/phi-3-mini-128k-instruct", "tokens": 4096 }
+    }
+  },
+  "deepinfra": {
+    "url": "https://api.deepinfra.com/v1/openai/chat/completions",
+    "model": {
+      "default": { "name": "openchat/openchat-3.6-8b", "tokens": 8192 },
+      "large": { "name": "nvidia/nemotron-4-340b-instruct", "tokens": 4096 },
+      "small": { "name": "microsoft/WizardLM-2-7B", "tokens": 4096 }
+    }
+  },
+  "togetherai": {
+    "url": "https://api.together.xyz/v1/chat/completions",
+    "model": {
+      "default": {
+        "name": "deepseek-ai/deepseek-llm-67b-chat",
+        "tokens": 4096
+      },
+      "large": {
+        "name": "NousResearch/Nous-Hermes-2-Mixtral-8x22B-Instruct",
+        "tokens": 65536
+      },
+      "small": { "name": "Qwen/Qwen1.5-0.5B-Chat", "tokens": 32768 }
+    }
+  },
+  "monsterapi": {
+    "url": "https://llm.monsterapi.ai/v1/chat/completions",
+    "model": {
+      "default": {
+        "name": "microsoft/Phi-3-mini-4k-instruct",
+        "tokens": 4096
+      },
+      "large": {
+        "name": "meta-llama/Meta-Llama-3-8B-Instruct",
+        "tokens": 4096
+      },
+      "small": { "name": "TinyLlama/TinyLlama-1.1B-Chat-v1.0", "tokens": 2048 }
+    }
+  },
+  "octoai": {
+    "url": "https://text.octoai.run/v1/chat/completions",
+    "model": {
+      "default": {
+        "name": "mistral-7b-instruct",
+        "tokens": 32768
+      },
+      "large": {
+        "name": "mixtral-8x22b-instruct",
+        "tokens": 65536
+      },
+      "small": { "name": "mistral-7b-instruct", "tokens": 32768 }
+    }
   }
 }