Make the Grafana metrics link runtime configurable

.gitignore

@@ -1,5 +1,6 @@
 # Ignore environment files
 .env
+infra/compose/prometheus/openrag_token
 
 # generated files and folders
 .files/

docs/assets/env_example.env

@@ -105,8 +105,14 @@ CHAINLIT_AUTH_SECRET=openrag-dev-secret
 # API/auth, so it is the OIDC front door. Zero-config otherwise (same-origin, no
 # CORS); VITE_* build-time options are documented in the env vars reference.
 # ADMIN_UI_PORT=8081
-
-
+# GRAFANA_ADMIN_USER=admin
+# GRAFANA_ADMIN_PASSWORD=replace-with-a-strong-password
+# Direct Grafana access:
+# GRAFANA_URL=http://localhost:3000/d/openrag-http/openrag-http-metrics
+# Or through the Admin UI proxy:
+# GRAFANA_URL=http://localhost:8081/grafana/d/openrag-http/openrag-http-metrics
+# GF_SERVER_ROOT_URL=http://localhost:8081/grafana/
+# GF_SERVER_SERVE_FROM_SUB_PATH=true
 
 # ── Ray (kept as-is by the compose stack; see the docs for what each does) ───
 RAY_DEDUP_LOGS=0
@@ -117,4 +123,4 @@ RAY_ENABLE_UV_RUN_RUNTIME_ENV=0
 # RAY_memory_monitor_refresh_ms=0
 
 # ── Logging (DEBUG on dev, INFO on prod) ──
-LOG_LEVEL=DEBUG
+LOG_LEVEL=DEBUG

docs/assets/env_linux_gpu.env

@@ -52,7 +52,16 @@ RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 # critical with the newest version of UV
 ## replace X.X.X.X with localhost (local) or your server IP, and APP_PORT with your FastAPI port
 
 # ADMIN_UI_PORT=8081                            # Host port for the admin UI (nginx). Default is 8081.
+# GRAFANA_ADMIN_USER=admin                      # Grafana login used on first startup.
+# GRAFANA_ADMIN_PASSWORD=replace-with-a-strong-password
+# Direct Grafana access:
+# GRAFANA_URL=http://localhost:3000/d/openrag-http/openrag-http-metrics
+#                                               # Runtime dashboard link. Restart the API after changing it.
+# Or through the Admin UI proxy:
+# GRAFANA_URL=http://localhost:8081/grafana/d/openrag-http/openrag-http-metrics
+# GF_SERVER_ROOT_URL=http://localhost:8081/grafana/
+# GF_SERVER_SERVE_FROM_SUB_PATH=true             # Serve Grafana through the admin UI's /grafana/ path.
 # VITE_API_BASE_URL=                            # API base baked into the SPA. EMPTY (default) = same-origin via
 #                                               # nginx (no CORS). Only set for a browser-direct build.
-# VITE_GRAFANA_URL=                             # Optional Grafana dashboard link on the admin "System" page.
-# VITE_APP_NAME=OpenRAG                         # App display name used in the UI branding.
+# VITE_GRAFANA_URL=                             # Optional build-time fallback for older API deployments.
+# VITE_APP_NAME=OpenRAG                         # App display name used in the UI branding.

docs/content/docs/documentation/env_vars.md

@@ -641,9 +641,10 @@ flowchart TD
 | Variable | Type | Default | Description |
 |----------|------|---------|-------------|
 | `ADMIN_UI_PORT` | `number` | `8081` | Host port the admin UI (nginx) is published on. Serves `/app/` and reverse-proxies `/auth`, `/v1`, `/chainlit`, … to the backend, so it is the OIDC front door (`OIDC_REDIRECT_URI` targets this port). Deploy-time (not a `VITE_*` build arg). |
+| `GRAFANA_URL` | `string` | `""` | Runtime, browser-reachable URL for the Grafana dashboard opened from **System → Metrics**. Restart the API after changing it. When this is empty or invalid, the action explains how to configure the dashboard instead of opening it. |
 | `VITE_API_BASE_URL` | `string` | `""` (same-origin) | API base baked into the SPA. **Empty (default) = same-origin**: nginx reverse-proxies the API over the Docker network, so the UI works on any host/IP with no CORS. Set to an absolute URL only for a browser-direct build — then list the UI's origin in `CORS_EXTRA_ORIGINS`. |
 | `VITE_BASE_PATH` | `string` | `/app/` | Sub-path the SPA is served under; must match the nginx `location`. |
-| `VITE_GRAFANA_URL` | `string` | `""` | Optional Grafana dashboard link shown on the admin **System** page. |
+| `VITE_GRAFANA_URL` | `string` | `""` | Build-time fallback for deployments whose API does not expose `GRAFANA_URL`. New deployments should use the runtime setting instead. |
 | `VITE_APP_NAME` | `string` | `OpenRAG` | Application display name used in the UI branding. |
 | `VITE_MOCK_API` | `boolean` | `false` | Development only — serves in-browser MSW API mocks when `true`. Ignored in production builds. |
 
@@ -712,3 +713,5 @@ Read only by the opt-in monitoring compose file (`infra/compose/monitoring.docke
 |----------|------|---------|-------------|
 | `GRAFANA_ADMIN_USER` | `str` | `admin` | Grafana admin username. |
 | `GRAFANA_ADMIN_PASSWORD` | `str` | _(required)_ | Grafana admin password — compose refuses to start the monitoring profile if unset. |
+| `GF_SERVER_ROOT_URL` | `str` | `http://localhost:3000` | Browser-facing Grafana root URL. Set this to the admin UI's `/grafana/` URL when using its proxy. |
+| `GF_SERVER_SERVE_FROM_SUB_PATH` | `bool` | `false` | Set to `true` when `GF_SERVER_ROOT_URL` includes the `/grafana/` subpath. |

docs/content/docs/installation/docker.mdx

@@ -13,3 +13,52 @@ The OpenRAG docker image is available on [DockerHub](https://hub.docker.com/r/rc
 
 OpenRAG requires several services to run, which can be orchestrated using Docker Compose.
 
+## Grafana monitoring
+
+The optional monitoring stack provides ready-to-use dashboards for OpenRAG HTTP
+traffic, host resources, and NVIDIA GPUs. It is separate from the default stack
+so deployments that do not need monitoring keep the same footprint.
+
+Before starting it, set a strong `GRAFANA_ADMIN_PASSWORD` in `.env`. The
+`GRAFANA_ADMIN_USER` defaults to `admin`. Create the Prometheus token file from
+the same `AUTH_TOKEN` used by OpenRAG:
+
+```bash
+cd infra/compose
+set -a
+source .env
+set +a
+printf '%s' "$AUTH_TOKEN" | sudo tee prometheus/openrag_token >/dev/null
+sudo chown 65534:65534 prometheus/openrag_token
+sudo chmod 0400 prometheus/openrag_token
+```
+
+Start OpenRAG with the monitoring overlay:
+
+```bash
+export SHARED_ENV="$PWD/.env"
+docker compose --env-file .env \
+  -f docker-compose.yaml \
+  -f monitoring.docker-compose.yaml \
+  up -d
+```
+
+`GRAFANA_URL` controls where **System > Metrics** opens. It must be reachable
+from the user's browser. For direct local access, use:
+
+```dotenv
+GRAFANA_URL=http://localhost:3000/d/openrag-http/openrag-http-metrics
+```
+
+For remote deployments, expose Grafana through the Admin UI proxy and configure
+its subpath:
+
+```dotenv
+GRAFANA_URL=https://openrag.example.com/grafana/d/openrag-http/openrag-http-metrics
+GF_SERVER_ROOT_URL=https://openrag.example.com/grafana/
+GF_SERVER_SERVE_FROM_SUB_PATH=true
+```
+
+Grafana has its own login. Its admin password is initialized when the persistent
+Grafana volume is created; changing the environment variable later does not
+reset an existing password.

infra/charts/openrag-stack/values.yaml

@@ -334,6 +334,7 @@ env:
     ENABLE_RAY_SERVE: "true"
     RAY_SERVE_NUM_REPLICAS: "4"
     RAY_SERVE_PORT: "80"
+    GRAFANA_URL: ""
 
     WITH_CHAINLIT_UI: "false"
     SAVE_UPLOADED_FILES: "false"

infra/compose/.env.example

@@ -105,8 +105,14 @@ CHAINLIT_AUTH_SECRET=openrag-dev-secret
 # API/auth, so it is the OIDC front door. Zero-config otherwise (same-origin, no
 # CORS); VITE_* build-time options are documented in the env vars reference.
 # ADMIN_UI_PORT=8081
-
-
+# GRAFANA_ADMIN_USER=admin
+# GRAFANA_ADMIN_PASSWORD=replace-with-a-strong-password
+# Direct Grafana access:
+# GRAFANA_URL=http://localhost:3000/d/openrag-http/openrag-http-metrics
+# Or through the Admin UI proxy:
+# GRAFANA_URL=http://localhost:8081/grafana/d/openrag-http/openrag-http-metrics
+# GF_SERVER_ROOT_URL=http://localhost:8081/grafana/
+# GF_SERVER_SERVE_FROM_SUB_PATH=true
 
 # ── Ray (kept as-is by the compose stack; see the docs for what each does) ───
 RAY_DEDUP_LOGS=0
@@ -117,4 +123,4 @@ RAY_ENABLE_UV_RUN_RUNTIME_ENV=0
 # RAY_memory_monitor_refresh_ms=0
 
 # ── Logging (DEBUG on dev, INFO on prod) ──
-LOG_LEVEL=DEBUG
+LOG_LEVEL=DEBUG

infra/compose/docker-compose.yaml

@@ -125,7 +125,7 @@ services:
         # to CORS_EXTRA_ORIGINS on the API.
         VITE_API_BASE_URL: ${VITE_API_BASE_URL:-}
         VITE_BASE_PATH: "/app/"
-        # Optional Grafana dashboard link on the admin "System" page; branding name.
+        # Build-time Grafana fallback; prefer runtime GRAFANA_URL in .env.
         VITE_GRAFANA_URL: ${VITE_GRAFANA_URL:-}
         VITE_APP_NAME: ${VITE_APP_NAME:-OpenRAG}
     restart: unless-stopped

infra/compose/grafana/dashboards/gpu-metrics.json

@@ -13,15 +13,15 @@
       {
         "current": {},
         "datasource": { "type": "prometheus", "uid": "prometheus" },
-        "definition": "label_values(nvidia_gpu_duty_cycle, uuid)",
+        "definition": "label_values(nvidia_smi_utilization_gpu_ratio, uuid)",
         "hide": 0,
         "includeAll": true,
         "label": "GPU",
         "multi": true,
         "name": "gpu",
         "options": [],
         "query": {
-          "query": "label_values(nvidia_gpu_duty_cycle, uuid)",
+          "query": "label_values(nvidia_smi_utilization_gpu_ratio, uuid)",
           "refId": "StandardVariableQuery"
         },
         "refresh": 2,
@@ -74,8 +74,8 @@
       },
       "targets": [
         {
-          "expr": "nvidia_gpu_duty_cycle{uuid=~\"$gpu\"}",
-          "legendFormat": "GPU {{ name }} ({{ uuid }})",
+          "expr": "nvidia_smi_utilization_gpu_ratio{uuid=~\"$gpu\"} * 100",
+          "legendFormat": "GPU {{ uuid }}",
           "refId": "A"
         }
       ],
@@ -122,13 +122,13 @@
       },
       "targets": [
         {
-          "expr": "nvidia_gpu_memory_total_bytes{uuid=~\"$gpu\"}",
-          "legendFormat": "Total — {{ name }}",
+          "expr": "nvidia_smi_memory_total_bytes{uuid=~\"$gpu\"}",
+          "legendFormat": "Total — {{ uuid }}",
           "refId": "A"
         },
         {
-          "expr": "nvidia_gpu_memory_used_bytes{uuid=~\"$gpu\"}",
-          "legendFormat": "Used — {{ name }}",
+          "expr": "nvidia_smi_memory_used_bytes{uuid=~\"$gpu\"}",
+          "legendFormat": "Used — {{ uuid }}",
           "refId": "B"
         }
       ],
@@ -164,8 +164,8 @@
       },
       "targets": [
         {
-          "expr": "nvidia_gpu_duty_cycle{uuid=~\"$gpu\"}",
-          "legendFormat": "{{ name }}",
+          "expr": "nvidia_smi_utilization_gpu_ratio{uuid=~\"$gpu\"} * 100",
+          "legendFormat": "{{ uuid }}",
           "refId": "A"
         }
       ],
@@ -202,8 +202,8 @@
       },
       "targets": [
         {
-          "expr": "(nvidia_gpu_memory_used_bytes{uuid=~\"$gpu\"} / nvidia_gpu_memory_total_bytes{uuid=~\"$gpu\"}) * 100",
-          "legendFormat": "{{ name }}",
+          "expr": "(nvidia_smi_memory_used_bytes{uuid=~\"$gpu\"} / nvidia_smi_memory_total_bytes{uuid=~\"$gpu\"}) * 100",
+          "legendFormat": "{{ uuid }}",
           "refId": "A"
         }
       ],
@@ -251,8 +251,8 @@
       },
       "targets": [
         {
-          "expr": "nvidia_gpu_temperature_celsius{uuid=~\"$gpu\"}",
-          "legendFormat": "{{ name }} ({{ uuid }})",
+          "expr": "nvidia_smi_temperature_gpu{uuid=~\"$gpu\"}",
+          "legendFormat": "{{ uuid }}",
           "refId": "A"
         }
       ],
@@ -298,8 +298,8 @@
       },
       "targets": [
         {
-          "expr": "nvidia_gpu_power_draw_watts{uuid=~\"$gpu\"}",
-          "legendFormat": "{{ name }} ({{ uuid }})",
+          "expr": "nvidia_smi_power_draw_watts{uuid=~\"$gpu\"}",
+          "legendFormat": "{{ uuid }}",
           "refId": "A"
         }
       ],
@@ -310,7 +310,7 @@
     },
     {
       "id": 7,
-      "title": "GPU Fan Speed",
+      "title": "GPU Memory Controller Utilization (%)",
       "type": "timeseries",
       "gridPos": { "h": 8, "w": 12, "x": 12, "y": 16 },
       "datasource": { "type": "prometheus", "uid": "prometheus" },
@@ -347,8 +347,8 @@
       },
       "targets": [
         {
-          "expr": "nvidia_gpu_fan_speed_percent{uuid=~\"$gpu\"}",
-          "legendFormat": "{{ name }} ({{ uuid }})",
+          "expr": "nvidia_smi_utilization_memory_ratio{uuid=~\"$gpu\"} * 100",
+          "legendFormat": "{{ uuid }}",
           "refId": "A"
         }
       ],

infra/compose/monitoring.docker-compose.yaml

@@ -22,11 +22,13 @@ services:
     image: grafana/grafana:11.2.2
     container_name: openrag-grafana
     ports:
-      - "3000:3000"
+      - "127.0.0.1:3000:3000"
     environment:
       - GF_SECURITY_ADMIN_USER=${GRAFANA_ADMIN_USER:-admin}
       - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD:?Set GRAFANA_ADMIN_PASSWORD in .env}
       - GF_USERS_ALLOW_SIGN_UP=false
+      - GF_SERVER_ROOT_URL=${GF_SERVER_ROOT_URL:-http://localhost:3000}
+      - GF_SERVER_SERVE_FROM_SUB_PATH=${GF_SERVER_SERVE_FROM_SUB_PATH:-false}
     volumes:
       - ./grafana/provisioning:/etc/grafana/provisioning:ro
       - ./grafana/dashboards:/var/lib/grafana/dashboards:ro
@@ -55,15 +57,13 @@ services:
     restart: unless-stopped
 
   nvidia-gpu-exporter:
-    image: utkuozdemir/nvidia_gpu_exporter:1.2.0
+    image: utkuozdemir/nvidia_gpu_exporter:1.13.1
     container_name: openrag-nvidia-gpu-exporter
     ports:
       # Localhost only: scraped over the compose network by name.
       - "127.0.0.1:9835:9835"
-    volumes:
-      - /usr/lib/x86_64-linux-gnu/libnvidia-ml.so:/usr/lib/x86_64-linux-gnu/libnvidia-ml.so:ro
-      - /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1:/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1:ro
-      - /usr/bin/nvidia-smi:/usr/bin/nvidia-smi:ro
+    environment:
+      - NVIDIA_DRIVER_CAPABILITIES=utility
     deploy:
       resources:
         reservations:

infra/compose/nginx/openrag-admin.conf

@@ -58,6 +58,31 @@ server {
         try_files $uri $uri/ /app/index.html;
     }
 
+    # Keep Grafana on the same browser-facing origin as the admin UI.
+    location = /grafana {
+        return 301 /grafana/;
+    }
+
+    location /grafana/ {
+        resolver 127.0.0.11 valid=30s ipv6=off;
+        set $grafana_upstream grafana;
+
+        proxy_pass http://$grafana_upstream:3000;
+
+        proxy_set_header Host              $http_host;
+        proxy_set_header X-Forwarded-Host  $http_host;
+        proxy_set_header X-Forwarded-Proto $forwarded_scheme;
+        proxy_set_header X-Forwarded-Prefix /grafana;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade    $http_upgrade;
+        proxy_set_header Connection $connection_upgrade;
+
+        proxy_read_timeout 300s;
+    }
+
     # Everything else is the OpenRag API (and /chainlit, /auth, /v1, …).
     location / {
         # Resolve the backend at request time via Docker's embedded DNS so nginx

infra/compose/prometheus/prometheus.yml

@@ -21,8 +21,4 @@ scrape_configs:
       type: Bearer
       credentials_file: /etc/prometheus/openrag_token
     static_configs:
-      - targets: ["host.docker.internal:8000"]
-
-  - job_name: "ray"
-    static_configs:
-      - targets: ["host.docker.internal:8080"]
+      - targets: ["openrag:8080"]

infra/docker/api.Dockerfile

@@ -94,7 +94,7 @@ RUN useradd --uid ${APP_UID} --gid 0 --no-log-init --no-create-home \
         /app/.venv /app/openrag.egg-info /opt/uv/cache \
         /app/openrag/.files /app/openrag/.chainlit \
     && chgrp -R 0 /app /opt/uv \
-    && chmod -R g-w /app /opt/uv \
+    && chmod -R g+rX,g-w /app /opt/uv \
     && chmod -R g=u /app/home /app/data /app/db /app/logs /app/model_weights \
         /app/.venv /app/openrag.egg-info /opt/uv/cache \
         /app/openrag/.files /app/openrag/.chainlit

openrag/api/main.py

@@ -57,6 +57,7 @@
 from api.routers.user.health import router as health_router
 from api.routers.user.search import router as search_router
 from api.runtime_flags import WITH_CHAINLIT_UI, WITH_OPENAI_API
+from api.runtime_ui import get_grafana_url
 from core.config import load_config
 from core.utils.banner import print_startup_banner
 from core.utils.logging import get_logger
@@ -340,6 +341,7 @@ def get_config():
         **redact_secrets(jsonable_encoder(settings)),
         "super_admin_mode": SUPER_ADMIN_MODE,
         "chainlit_enabled": WITH_CHAINLIT_UI,
+        "grafana_url": get_grafana_url(),
     }