feat: integrated litellm for tracing

examples/tracing/litellm/litellm_tracing.ipynb

@@ -0,0 +1,169 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openlayer-ai/openlayer-python/blob/main/examples/tracing/litellm/litellm_tracing.ipynb)\n",
+        "\n",
+        "\n",
+        "# <a id=\"top\">LiteLLM monitoring quickstart</a>\n",
+        "\n",
+        "This notebook illustrates how to get started monitoring LiteLLM completions with Openlayer.\n",
+        "\n",
+        "LiteLLM provides a unified interface to call 100+ LLM APIs using the same input/output format. This integration allows you to trace and monitor completions across all supported providers through a single interface.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "!pip install openlayer litellm\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## 1. Set the environment variables\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "\n",
+        "import litellm\n",
+        "\n",
+        "# Set your API keys for the providers you want to use\n",
+        "os.environ[\"OPENAI_API_KEY\"] = \"YOUR_OPENAI_API_KEY_HERE\"\n",
+        "os.environ[\"ANTHROPIC_API_KEY\"] = \"YOUR_ANTHROPIC_API_KEY_HERE\"  # Optional\n",
+        "os.environ[\"GROQ_API_KEY\"] = \"YOUR_GROQ_API_KEY_HERE\"  # Optional\n",
+        "\n",
+        "# Openlayer env variables\n",
+        "os.environ[\"OPENLAYER_API_KEY\"] = \"YOUR_OPENLAYER_API_KEY_HERE\"\n",
+        "os.environ[\"OPENLAYER_INFERENCE_PIPELINE_ID\"] = \"YOUR_OPENLAYER_INFERENCE_PIPELINE_ID_HERE\"\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## 2. Enable LiteLLM tracing\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from openlayer.lib import trace_litellm\n",
+        "\n",
+        "# Enable tracing for all LiteLLM completions\n",
+        "trace_litellm()\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## 3. Use LiteLLM normally - tracing happens automatically!\n",
+        "\n",
+        "### Basic completion with OpenAI\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Basic completion with OpenAI GPT-4\n",
+        "response = litellm.completion(\n",
+        "    model=\"gpt-4\",\n",
+        "    messages=[\n",
+        "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
+        "        {\"role\": \"user\", \"content\": \"What is the capital of France?\"}\n",
+        "    ],\n",
+        "    temperature=0.7,\n",
+        "    max_tokens=100,\n",
+        "    inference_id=\"litellm-openai-example-1\"  # Optional: custom inference ID\n",
+        ")\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Multi-provider comparison\n",
+        "\n",
+        "One of LiteLLM's key features is the ability to easily switch between providers. Let's trace completions from different providers:\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Test the same prompt with different models/providers\n",
+        "prompt = \"Explain quantum computing in simple terms.\"\n",
+        "messages = [{\"role\": \"user\", \"content\": prompt}]\n",
+        "\n",
+        "models_to_test = [\n",
+        "    \"gpt-3.5-turbo\",  # OpenAI\n",
+        "    \"claude-3-haiku-20240307\",  # Anthropic (if API key is set)\n",
+        "    \"groq/llama-3.1-8b-instant\",  # Groq (if API key is set)\n",
+        "]\n",
+        "\n",
+        "for model in models_to_test:\n",
+        "    response = litellm.completion(\n",
+        "        model=model,\n",
+        "        messages=messages,\n",
+        "        temperature=0.5,\n",
+        "        max_tokens=150,\n",
+        "        inference_id=f\"multi-provider-{model.replace('/', '-')}\"\n",
+        "    )\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## 4. View your traces\n",
+        "\n",
+        "Once you've run the examples above, you can:\n",
+        "\n",
+        "1. **Visit your OpenLayer dashboard** to see all the traced completions\n",
+        "2. **Analyze performance** across different models and providers\n",
+        "3. **Monitor costs** and token usage\n",
+        "4. **Debug issues** with detailed request/response logs\n",
+        "5. **Compare models** side-by-side\n",
+        "\n",
+        "The traces will include:\n",
+        "- **Request details**: Model, parameters, messages\n",
+        "- **Response data**: Generated content, token counts, latency\n",
+        "- **Provider information**: Which underlying service was used\n",
+        "- **Custom metadata**: Any additional context you provide\n",
+        "\n",
+        "For more information, check out:\n",
+        "- [OpenLayer Documentation](https://docs.openlayer.com/)\n",
+        "- [LiteLLM Documentation](https://docs.litellm.ai/)\n",
+        "- [LiteLLM Supported Models](https://docs.litellm.ai/docs/providers)\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 2
+}

src/openlayer/lib/__init__.py

@@ -13,6 +13,7 @@
     "trace_bedrock",
     "trace_oci_genai",
     "trace_oci",  # Alias for backward compatibility
+    "trace_litellm",
     "update_current_trace",
     "update_current_step"
 ]
@@ -143,3 +144,37 @@ def trace_oci_genai(client, estimate_tokens: bool = True):
 # --------------------------------- OCI GenAI -------------------------------- #
 # Alias for backward compatibility
 trace_oci = trace_oci_genai
+
+
+# --------------------------------- LiteLLM ---------------------------------- #
+def trace_litellm():
+    """Enable tracing for LiteLLM completions.
+
+    This function patches litellm.completion to automatically trace all completions
+    made through the LiteLLM library, which provides a unified interface to 100+ LLM APIs.
+
+    Example:
+        >>> import litellm
+        >>> from openlayer.lib import trace_litellm
+        >>> 
+        >>> # Enable tracing
+        >>> trace_litellm()
+        >>> 
+        >>> # Use LiteLLM normally - tracing happens automatically
+        >>> response = litellm.completion(
+        ...     model="gpt-3.5-turbo",
+        ...     messages=[{"role": "user", "content": "Hello!"}],
+        ...     inference_id="custom-id-123"  # Optional Openlayer parameter
+        ... )
+    """
+    # pylint: disable=import-outside-toplevel
+    try:
+        import litellm
+    except ImportError:
+        raise ImportError(
+            "litellm is required for LiteLLM tracing. Install with: pip install litellm"
+        )
+
+    from .integrations import litellm_tracer
+
+    return litellm_tracer.trace_litellm()