Merge pull request #15 from tpaulshippy/red-candle-community

Author: tpaulshippy

CONTRIBUTING.md

@@ -55,6 +55,39 @@ rake vcr:record[all]               # Everything
 
 Always check cassettes for leaked API keys before committing.
 
+## Optional Dependencies
+
+### Red Candle Provider
+
+The Red Candle provider enables local LLM execution using quantized GGUF models. It requires a Rust toolchain, so it's optional for contributors.
+
+**To work WITHOUT Red Candle (default):**
+```bash
+bundle install
+bundle exec rspec  # Red Candle tests will be skipped
+```
+
+**To work WITH Red Candle:**
+```bash
+# Enable the Red Candle gem group
+bundle config set --local with red_candle
+bundle install
+
+# Run tests with stubbed Red Candle (fast, default)
+bundle exec rspec
+
+# Run tests with real inference (slow, downloads models)
+RED_CANDLE_REAL_INFERENCE=true bundle exec rspec
+```
+
+**To switch back to working without Red Candle:**
+```bash
+bundle config unset with
+bundle install
+```
+
+The `bundle config` settings are stored in `.bundle/config` (gitignored), so each developer can choose their own setup without affecting others.
+
 ## Important Notes
 
 * **Never edit `models.json`, `aliases.json`, or `available-models.md`** - they're auto-generated by `rake models`

Gemfile

@@ -41,3 +41,9 @@ group :development do # rubocop:disable Metrics/BlockLength
   # Optional dependency for Vertex AI
   gem 'googleauth'
 end
+
+# Optional group for Red Candle provider (requires Rust toolchain)
+# To include: bundle config set --local with red-candle
+group :red_candle, optional: true do
+  gem 'red-candle', '~> 1.3'
+end

README.md

@@ -127,7 +127,7 @@ response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "pr
 * **Rails:** ActiveRecord integration with `acts_as_chat`
 * **Async:** Fiber-based concurrency
 * **Model registry:** 500+ models with capability detection and pricing
-* **Providers:** OpenAI, Anthropic, Gemini, VertexAI, Bedrock, DeepSeek, Mistral, Ollama, OpenRouter, Perplexity, GPUStack, and any OpenAI-compatible API
+* **Providers:** OpenAI, Anthropic, Gemini, VertexAI, Bedrock, DeepSeek, Mistral, Ollama, OpenRouter, Perplexity, GPUStack, [RedCandle](https://github.com/scientist-labs/red-candle), and any OpenAI-compatible API
 
 ## Installation
 

docs/_advanced/models.md

@@ -95,6 +95,33 @@ RubyLLM.models.refresh!(remote_only: true)
 
 This is useful when you want to refresh only cloud-based models without querying local model servers.
 
+### Dynamic Model Registration (Red Candle)
+
+Some providers register their models dynamically at runtime rather than through the models.json file. Red Candle is one such provider - it registers its GGUF models when the gem is loaded.
+
+**How Red Candle Models Work:**
+
+1. **Not in models.json**: Red Candle models don't appear in the static models.json file since they're only available when the gem is installed.
+
+2. **Dynamic Registration**: When ruby_llm.rb loads and Red Candle is available, it adds models to the in-memory registry:
+   ```ruby
+   # This happens automatically in lib/ruby_llm.rb
+   RubyLLM::Providers::RedCandle.models.each do |model|
+     RubyLLM.models.instance_variable_get(:@models) << model
+   end
+   ```
+
+3. **Excluded from refresh!**: The `refresh!(remote_only: true)` flag excludes Red Candle and other local providers.
+
+4. **Currently Supported Models**:
+   - `google/gemma-3-4b-it-qat-q4_0-gguf`
+   - `TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF`
+   - `TheBloke/Mistral-7B-Instruct-v0.2-GGUF`
+   - `Qwen/Qwen2.5-1.5B-Instruct-GGUF`
+   - `microsoft/Phi-3-mini-4k-instruct`
+
+Red Candle models are only available when the gem is installed with the red_candle group enabled. See the [Configuration Guide]({% link _getting_started/configuration.md %}) for installation instructions.
+
 **For Gem Development:**
 
 The `rake models:update` task is designed for gem maintainers and updates the `models.json` file shipped with the gem:

docs/_getting_started/configuration.md

@@ -65,6 +65,7 @@ RubyLLM.configure do |config|
   config.ollama_api_base = 'http://localhost:11434/v1'
   config.gpustack_api_base = ENV['GPUSTACK_API_BASE']
   config.gpustack_api_key = ENV['GPUSTACK_API_KEY']
+  # Red Candle (optional - see below)
 
   # AWS Bedrock (uses standard AWS credential chain if not set)
   config.bedrock_api_key = ENV['AWS_ACCESS_KEY_ID']
@@ -91,6 +92,37 @@ end
 
 These headers are optional and only needed for organization-specific billing or project tracking.
 
+### Red Candle (Local GGUF Models)
+
+Red Candle is an optional provider that enables local execution of quantized GGUF models. To use it, add the red-candle gem to your Gemfile:
+
+```ruby
+# Gemfile
+gem 'ruby_llm'
+gem 'red-candle'  # Optional: for local GGUF model execution
+```
+
+Then install:
+
+```bash
+bundle install
+```
+
+Red Candle requires no API keys since it runs models locally. Some models may require HuggingFace authentication:
+
+```bash
+huggingface-cli login  # Required for some gated models
+```
+
+See [Red Candle's HuggingFace guide](https://github.com/scientist-labs/red-candle/blob/main/docs/HUGGINGFACE.md) for details on authentication.
+
+Once configured, you can use it like any other provider:
+
+```ruby
+chat = RubyLLM.chat(model: 'TheBloke/Mistral-7B-Instruct-v0.2-GGUF', provider: :red_candle)
+response = chat.ask("Hello!")
+```
+
 ## Custom Endpoints
 
 ### OpenAI-Compatible APIs

docs/_reference/available-models.md

@@ -27,6 +27,7 @@ redirect_from:
 - **OpenRouter**: Direct API
 - **Others**: Local capabilities files
 
+
 ## Last Updated
 {: .d-inline-block }
 
@@ -2491,3 +2492,20 @@ Models that generate embeddings:
 | text-embedding-3-small | openai | - | - | In: $0.02, Out: $0.02 |
 | text-embedding-ada-002 | openai | - | - | In: $0.10, Out: $0.10 |
 
+
+## Local Providers
+
+### Red Candle (5)
+
+Red Candle enables local execution of quantized GGUF models. These models run on your machine with no API costs.
+
+| Model | Provider | Context | Max Output | Standard Pricing (per 1M tokens) |
+| :-- | :-- | --: | --: | :-- |
+| google/gemma-3-4b-it-qat-q4_0-gguf | red_candle | 8192 | 512 | Free (local execution) |
+| TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF | red_candle | 2048 | 512 | Free (local execution) |
+| TheBloke/Mistral-7B-Instruct-v0.2-GGUF | red_candle | 32768 | 512 | Free (local execution) |
+| Qwen/Qwen2.5-1.5B-Instruct-GGUF | red_candle | 32768 | 512 | Free (local execution) |
+| microsoft/Phi-3-mini-4k-instruct | red_candle | 4096 | 512 | Free (local execution) |
+
+> **Note:** Local providers (Ollama, GPUStack, Red Candle) register their models dynamically at runtime based on what's installed locally. Ollama and GPUStack models depend on what you've pulled or configured on your system. Red Candle requires the `red-candle` gem. See the [Configuration Guide]({% link _getting_started/configuration.md %}) for setup instructions.
+{: .note }

docs/index.md

@@ -67,6 +67,10 @@ permalink: /
     <img src="https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/vertexai-color.svg" alt="VertexAI" class="logo-medium">
     <img src="https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/vertexai-text.svg" alt="VertexAI" class="logo-small">
   </div>
+  <div class="provider-logo">
+    <img src="https://raw.githubusercontent.com/scientist-labs/red-candle/refs/heads/main/docs/assets/red-candle-icon.svg" alt="red-candle" class="logo-medium">
+    <img src="https://raw.githubusercontent.com/scientist-labs/red-candle/refs/heads/main/docs/assets/red-candle-text.svg" alt="red-candle" class="logo-small">
+  </div>
 </div>
 
 <div class="badge-container">
@@ -205,4 +209,3 @@ end
 chat = Chat.create! model_id: "claude-sonnet-4"
 chat.ask "What's in this file?", with: "report.pdf"
 ```
-

gemfiles/rails_7.1.gemfile

@@ -35,4 +35,8 @@ group :development do
   gem "googleauth"
 end
 
+group :red_candle, optional: true do
+  gem "red-candle", "~> 1.2"
+end
+
 gemspec path: "../"

gemfiles/rails_7.1.gemfile.lock

@@ -290,9 +290,14 @@ GEM
       zeitwerk (~> 2.6)
     rainbow (3.1.1)
     rake (13.3.0)
+    rake-compiler-dock (1.9.1)
+    rb_sys (0.9.117)
+      rake-compiler-dock (= 1.9.1)
     rdoc (6.14.2)
       erb
       psych (>= 4.0.0)
+    red-candle (1.2.3)
+      rb_sys
     regexp_parser (2.11.2)
     reline (0.6.2)
       io-console (~> 0.5)
@@ -384,7 +389,7 @@ GEM
     zeitwerk (2.7.3)
 
 PLATFORMS
-  arm64-darwin-22
+  arm64-darwin-24
   x86_64-linux
 
 DEPENDENCIES
@@ -406,6 +411,7 @@ DEPENDENCIES
   pry (>= 0.14)
   rails (~> 7.1.0)
   rake (>= 13.0)
+  red-candle (~> 1.2)
   reline
   rspec (~> 3.12)
   rubocop (>= 1.0)

gemfiles/rails_7.2.gemfile

@@ -35,4 +35,8 @@ group :development do
   gem "googleauth"
 end
 
+group :red_candle, optional: true do
+  gem "red-candle", "~> 1.2"
+end
+
 gemspec path: "../"