Skip to content

Latest commit

 

History

History
63 lines (48 loc) · 3.87 KB

llm-provider-guide.md

File metadata and controls

63 lines (48 loc) · 3.87 KB

AI - LLM Provider Definition Guide

Documentation for managing LLM Provider which capatures the details of the AI service and configurations.

Overview

This guide explains how to configure an LLM Provider by specifying key details such as the model, API key, temperature, top-p, seed, and max tokens.

Demo Video

AI-ManageLLM-480p.mov

Configuration Fields

Field Description
Label A unique user-friendly name for the LLM provider.
Server URL The endpoint for the chat completion API
Model The model to use. Ensure the model is supported by the platform.
API Key The authentication key for accessing the API. If your service is running locally or does not support authentication set a dummy value
Temperature Controls randomness in responses (1 = balanced, 0 = deterministic, >1 = more creative).
Top-P Controls nucleus sampling (0.8 means tokens with 80% cumulative probability are considered).
Seed Ensures reproducible responses when set (e.g., 32). If left blank, responses may vary.
Max Tokens The maximum number of tokens the response can generate. Higher values allow longer responses but consume more API usage.

Buttons & Actions

Button Function
Help Opens the documentation or guide for configuring LLM Providers.
Test Service Runs a test call to check if the API key, model, and server URL are correct.
Cancel Discards changes and closes the dialog.
Save Saves the configuration.

Temperature - Top-P settings and workings

Temperature (T) Top-P (P) Effect on Output Use Case
Low (T ≈ 0.2) Low (P ≈ 0.5) Very deterministic and strictly follows high-probability words. Almost no variation. Good for structured outputs like math, facts, legal docs, code generation.
Low (T ≈ 0.2) High (P ≈ 0.95) Mostly deterministic but allows some diversity within high-confidence tokens. Good for customer support, FAQs, chatbots with strict correctness.
Medium (T ≈ 0.7) Low (P ≈ 0.5) Balanced response with limited diversity. Works for formal writing, product descriptions, summarization.
Medium (T ≈ 0.7) High (P ≈ 0.95) Best balance between coherence & creativity. Maintains fluency while allowing variety. ✅ Recommended default for chat, general conversations, AI assistants.
High (T ≈ 1.2) Low (P ≈ 0.5) Somewhat unpredictable, but still respects top choices. Good for storytelling, opinion pieces, creative writing.
High (T ≈ 1.2) High (P ≈ 0.95) Most creative & diverse. Generates unique responses but may hallucinate or go off-topic. Best for poetry, fiction, brainstorming ideas.
Very High (T ≈ 2.0) Low (P ≈ 0.5) Chaotic & unstructured but still picks from top probabilities. Avoid for most use cases, unless extreme creativity is required.
Very High (T ≈ 2.0) High (P ≈ 1.0) Fully random, often nonsensical output. Not recommended for practical tasks.

Top-P & Temperature - Recommended Defaults:

Use Case Temperature Top-P
General Chatbot / AI Assistant 0.7 0.95
Technical Writing / FAQs 0.3 0.95
Storytelling / Creative Writing 1.2 0.9
Code Generation / Logical Responses 0.2 0.9
Poetry / Idea Brainstorming 1.5 0.95

Seed

A seed is a fixed starting point for a random number generator. Since language models use random sampling during token selection, setting a seed ensures that you get the same output every time.

How Does It Work?

  1. Without a fixed seed: The model will generate different responses each time you run it (even with the same prompt).
  2. With a fixed seed (seed = 42, for example): The model will follow the same random path and give the same response every time.