Remove unused provider API keys from the backend environment #2466

Summary

The backend process carries provider API keys (OPENAI_API_KEY, GEMINI_API_KEY, GOOGLE_API_KEY, AZURE_OPENAI_API_KEY) that most deployments do not need. Any code path that reaches a provider without an explicit key silently picks these up, which means a tenant's model can end up running on our credentials.

Each deployment should ship exactly the credential its own default model requires, and nothing else.

Background

Every provider falls back to the process environment when handed a falsy key. In LiteLLM (litellm/main.py:2128-2133):

api_key = (
    api_key
    or litellm.api_key
    or litellm.openai_key
    or get_secret("OPENAI_API_KEY")
)

Our own SDK providers do the same (sdk/src/rhesis/sdk/models/providers/openai.py:20, gemini.py:42).

So a tenant Model row with no API key and no endpoint resolves, runs on our account, and looks to the tenant like a working configuration. We pay the provider, and nothing is billed internally because the row belongs to the tenant and tenant rows are correctly never charged.

#2461 closes the known path into this: model creation rejects a row with neither a key nor an endpoint, and the resolution layer refuses to build one. This issue is the layer underneath, which does not depend on us having enumerated every path. The fallback can only borrow what is present in the environment. Remove the keys and the whole class of leak goes away, including paths nobody has thought of yet.

What needs doing

• For each deployment (dev, stg, prd), read the configured DEFAULT_GENERATION_MODEL, DEFAULT_EVALUATION_MODEL, DEFAULT_EXECUTION_MODEL and DEFAULT_EMBEDDING_MODEL and note which provider credential each actually requires. The in-code defaults are rhesis/rhesis-default (apps/backend/src/rhesis/backend/app/config/settings.py:284-298), which needs none of these keys.
• Drop every provider key from that deployment's secrets that no default model needs. Sources to update: infrastructure/k8s/manifests/secrets/rhesis-secrets.yaml.example and the per-cluster external secrets under kubernetes/clusters/*/external-secrets/.
• Confirm nothing else in the deployment consumes them. In application source these variables appear only as provider-constructor fallbacks, which is the behaviour we are removing.
• Keep them in local dev and CI where they are legitimately used for running against real providers.

Notes

This matters for multi-tenant (SaaS) deployments specifically. In a self-hosted deployment a tenant row falling back to the environment is falling back to the operator's own key, which is not a leak.

Worth checking the logs for the warning added by #2461 before and after, to see whether any real row has been hitting this.

Related

• Stop tenant model rows running on our provider credentials #2461 adds the write-time rejection and the runtime refusal
• Centralize LLM token accrual instead of wiring it per call site #2382 centralized token accrual, where this was found