Skip to content

Commit 8f09b27

Browse files
authored
Auto-enable tf2onnx large_model for models >1.5GB (keras-team#22825)
* Fix TPU test workflow job name rendering raw template expression The job name used `format()` with a boolean expression inside, which caused GitHub Actions to render the raw template in the checks UI. Replace the dynamic expression with fully static `matrix.include` names so the job name evaluates reliably to: - 'Run tests on TPU' for single-device - 'Run tests on multi-TPU' for multi-device Fixes keras-team#22776 * Auto-enable tf2onnx large_model for models >1.5GB TensorFlow GraphDef has a 2GB protobuf limit. When exporting large models (e.g. Llama 1B, Gemma3 1B) to ONNX via tf2onnx, freezing all variables into the graph causes import_graph_def to fail with: RuntimeError: size too big: ... string length exceeds max size This change estimates total weight size and passes large_model=True to tf2onnx when the model exceeds 1.5GB, triggering external weight storage and avoiding the protobuf limit. Weight size is computed from model.weights metadata (shape + dtype) without materializing tensor values, so load_weights=False is safe. Fixes keras-team/keras-hub#2340
1 parent 766c0ee commit 8f09b27

1 file changed

Lines changed: 28 additions & 0 deletions

File tree

keras/src/export/onnx.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
1+
import math
12
import warnings
23

4+
import numpy as np
5+
36
from keras.src import backend
47
from keras.src import tree
58
from keras.src.export.export_utils import convert_spec_to_tensor
@@ -99,13 +102,38 @@ def export_onnx(
99102
)
100103
decorated_fn = get_concrete_fn(model, input_signature, **kwargs)
101104

105+
# Estimate total weight size to decide if large_model format is needed.
106+
# TensorFlow GraphDef has a 2GB protobuf limit; tf2onnx's large_model
107+
# path stores weights externally to avoid this.
108+
def _dtype_size(dtype):
109+
try:
110+
return np.dtype(dtype).itemsize
111+
except TypeError:
112+
if dtype == "bfloat16":
113+
return 2
114+
if dtype in ("float8_e4m3fn", "float8_e5m2"):
115+
return 1
116+
if dtype == "string":
117+
return 0
118+
raise ValueError(f"Unsupported dtype: {dtype}")
119+
120+
total_bytes = 0
121+
for w in model.weights:
122+
shape = w.shape
123+
if shape is None or None in shape:
124+
continue
125+
total_bytes += math.prod(shape) * _dtype_size(w.dtype)
126+
127+
large_model = total_bytes > (3 * 1024**3 // 2)
128+
102129
# Use `tf2onnx` to convert the `decorated_fn` to the ONNX format.
103130
patch_tf2onnx() # TODO: Remove this once `tf2onnx` supports numpy 2.
104131
tf2onnx.convert.from_function(
105132
decorated_fn,
106133
input_signature,
107134
opset=opset_version,
108135
output_path=filepath,
136+
large_model=large_model,
109137
)
110138

111139
elif backend.backend() == "jax":

0 commit comments

Comments
 (0)