ChatGroq._combine_llm_outputs mutates per-generation token_usage in place and mishandles None values (silent clobber / TypeError) #38659

Submission checklist

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Related Issues / PRs

• TypeError: unsupported operand type(s) for += on dict in _combine_llm_outputs when making multiple LLM calls #38482 — langchain-mistralai has the naive += version of this merge (no nested-dict handling at all)
• ChatFireworks._combine_llm_outputs raises TypeError on nested token_usage dicts (prompt_tokens_details) in batched generate() #38648 / fix(fireworks): merge nested token_usage dicts in _combine_llm_outputs #38646 — the langchain-fireworks sibling (same naive +=) and its fix
• langchain-groq is the interesting case: it already attempts nested-dict handling and None-guarding, but both are implemented incorrectly (details below)

Reproduction Steps / Example Code (Python)

import os

os.environ.setdefault("GROQ_API_KEY", "fake-key")  # offline repro, no network needed

from langchain_groq import ChatGroq

llm = ChatGroq(model="llama-3.3-70b-versatile")

# --- 1. Nested-dict merge mutates the caller's llm_outputs in place ---
o1 = {
    "token_usage": {"prompt_tokens": 10, "input_tokens_details": {"cached_tokens": 5}},
    "model_name": "m",
}
o2 = {
    "token_usage": {"prompt_tokens": 20, "input_tokens_details": {"cached_tokens": 3}},
    "model_name": "m",
}
combined = llm._combine_llm_outputs([o1, o2])
print(o1["token_usage"]["input_tokens_details"])
# {'cached_tokens': 8}  <- the first response's own usage was 5, now corrupted
print(combined["token_usage"]["input_tokens_details"] is o1["token_usage"]["input_tokens_details"])
# True  <- combined result and first response share the same dict object

# --- 2. A later None silently clobbers an accumulated value ---
o3 = {"token_usage": {"queue_time": 0.5, "total_tokens": 30}, "model_name": "m"}
o4 = {"token_usage": {"queue_time": None, "total_tokens": 40}, "model_name": "m"}
print(llm._combine_llm_outputs([o3, o4])["token_usage"])
# {'queue_time': None, 'total_tokens': 70}  <- 0.5 silently lost

# --- 3. None first, then a number -> TypeError ---
o5 = {"token_usage": {"queue_time": None}, "model_name": "m"}
o6 = {"token_usage": {"queue_time": 0.5}, "model_name": "m"}
llm._combine_llm_outputs([o5, o6])
# TypeError: unsupported operand type(s) for +=: 'NoneType' and 'float'

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "repro_groq.py", line 38, in <module>
    llm._combine_llm_outputs([o5, o6])
  File ".../langchain_groq/chat_models.py", line 853, in _combine_llm_outputs
    overall_token_usage[k] += v
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'float'

Description

The merge loop in ChatGroq._combine_llm_outputs (libs/partners/groq/langchain_groq/chat_models.py:829) has three defects, all shown in the repro above:

1. In-place mutation of caller data. The first occurrence of a nested usage dict is stored by reference (overall_token_usage[k] = v), and merging the next output then does overall_token_usage[k][nested_k] += nested_v — mutating the first response's own token_usage dict. This is user-visible: in BaseChatModel.generate (langchain_core/language_models/chat_models.py:1673-1685) _combine_llm_outputs runs before the per-generation llm_output dicts are handed to on_llm_end, so callbacks and traces for the first generation report inflated nested counts (e.g. cached_tokens of generation 0 silently includes generation 1's).

2. Inverted None-guard (silent data loss). The loop reads if k in overall_token_usage and v is not None: ... else: overall_token_usage[k] = v, so a later None value falls into the else and overwrites an already-accumulated number. Groq really does return None here — ChatGroq's own class docstring shows a live response with 'queue_time': None in token_usage.

3. None first, then a number → crash. The same guard only checks the new value for None, so None stored first followed by a number hits overall_token_usage[k] += v → TypeError: unsupported operand type(s) for +=: 'NoneType' and 'float' on a batched generate().

(There's also a dead branch: if k not in overall_token_usage inside the path that requires k in overall_token_usage.)

Related: langchain-mistralai (#38482) and langchain-fireworks (#38648/#38646) have the naive += version of this bug; groq attempted the nested handling but got the aliasing and None semantics wrong. langchain-openai has the reference implementation (_update_token_usage, base.py:486), which rebuilds nested dicts instead of mutating and skips None values.

I have a fix ready on a branch (abcgco/langchain:fix-groq-combine-llm-outputs): a recursive _update_token_usage following the langchain-openai pattern but extended to accept floats (Groq reports float timings such as queue_time / prompt_time, which the openai int-only version would reject), plus a None skip. Includes parametrized unit tests for the None/float cases and an input-immutability regression test; make format lint test green in libs/partners/groq (62 passed). Per the workflow I'll open the PR once assigned — could a maintainer assign this to me?

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.6.0 (macOS 15.6.1, arm64)
Python Version: 3.12.10

Package Information

langchain_core: 1.4.8
langchain_groq: 1.1.3 (reproduced on latest master as well)
langsmith: 0.9.7