Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG Report]: Keras model.save() throws exception when saving to network path on Windows #1287

Open
abrguyt opened this issue Feb 10, 2025 · 0 comments

Comments

@abrguyt
Copy link

abrguyt commented Feb 10, 2025

Description

Using the custom packaging of the latest Tensorflow.NET version from the Github Repo (to overcome the bug in the official NuGet package - see https://github.com/SciSharp/TensorFlow.NET/issues/1246#issuecomment-2630024514 I got Tensorflow.NET / Keras model fitting, saving and loading working well.

Just when saving a Keras model to a Windows network path like '\FS-01\COM-Data$\ML Models' an message-less exception is thrown in

at Tensorflow.Eager.EagerRunner.SetOpAttrList(Context ctx, SafeEagerOpHandle op, String key, Object values, TF_AttrType type, Dictionary`2 attr_list_sizes, Status status) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs:line 326

Looking at the source it seems line 323 in the file 'TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs' is causing this with

throw new NotImplementedException("");

Saving the model to a local folder on the file system works perfectly well - I suspect somehow the unusual characters in the network file path cause this exception; but I can't dive deep in the Tensorflow.NET source to figure out why this is happening - any suggestions?

Full stack trace:

Source: Tensorflow.Binding
 Strack Trace:    at Tensorflow.Eager.EagerRunner.SetOpAttrList(Context ctx, SafeEagerOpHandle op, String key, Object values, TF_AttrType type, Dictionary`2 attr_list_sizes, Status status) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs:line 326
   at Tensorflow.Eager.EagerRunner.SetOpAttrs(SafeEagerOpHandle op, Object[] attrs) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs:line 241
   at Tensorflow.Eager.EagerRunner.TFE_ExecuteCancelable(Context ctx, String device_name, String op_name, Tensor[] inputs, Object[] attrs, Int32 num_outputs) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_Execute.cs:line 63
   at Tensorflow.Eager.EagerRunner.TFE_Execute(Context ctx, String device_name, String op_name, Tensor[] inputs, Object[] attrs, Int32 num_outputs) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_Execute.cs:line 36
   at Tensorflow.Eager._execute.quick_execute(String op_name, Int32 num_outputs, Tensor[] inputs, Object[] attrs, Context ctx, String name) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\execute.cs:line 31
   at Tensorflow.io_ops.save_v2_eager_fallback(Tensor prefix, String[] tensor_names, String[] shape_and_slices, Tensor[] tensors, String name, Context ctx) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Operations\io_ops.cs:line 58
   at Tensorflow.io_ops.save_v2(Tensor prefix, String[] tensor_names, String[] shape_and_slices, Tensor[] tensors, String name) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Operations\io_ops.cs:line 40
   at Tensorflow.tensorflow.IoApi.save_v2(Tensor prefix, String[] tensor_names, String[] shape_and_slices, Tensor[] tensors, String name) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\APIs\tf.io.cs:line 45
   at Tensorflow.Checkpoint.SingleDeviceSaver.save(Tensor file_prefix, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 79
   at Tensorflow.Checkpoint.MultiDeviceSaver.<>c__DisplayClass5_2.<save>b__6(ITensorFlowObject _) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 285
   at Tensorflow.Binding.tf_with[T](T py, Action`1 action) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Binding.Util.cs:line 199
   at Tensorflow.Checkpoint.MultiDeviceSaver.<>c__DisplayClass5_0.<save>g__save_fn|1() in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 283
   at Tensorflow.Checkpoint.MultiDeviceSaver.save(Tensor file_prefix, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 305
   at Tensorflow.Checkpoint.MultiDeviceSaver.save(String file_prefix, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 309
   at Tensorflow.Checkpoint.TrackableSaver.<>c__DisplayClass14_0.<save_cached_when_graph_building>b__0() in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\checkpoint.cs:line 119
   at Tensorflow.Checkpoint.TrackableSaver.save_cached_when_graph_building(String file_prefix, Tensor object_graph_tensor, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\checkpoint.cs:line 136
   at Tensorflow.Checkpoint.TrackableSaver.save(String file_prefix, Nullable`1 checkpoint_number, Session session, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\checkpoint.cs:line 180
   at Tensorflow.SavedModelUtils.save_and_return_nodes(Trackable obj, String export_dir, ConcreteFunction signatures, SaveOptions options, Boolean experimental_skip_checkpoint) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Training\Saving\SavedModel\save.cs:line 46
   at Tensorflow.Keras.Saving.SavedModel.KerasSavedModelUtils.save_model(Model model, String filepath, Boolean overwrite, Boolean include_optimizer, ConcreteFunction signatures, SaveOptions options, Boolean save_traces)
   at Tensorflow.Keras.Engine.Model.save(String filepath, Boolean overwrite, Boolean include_optimizer, String save_format, SaveOptions options, ConcreteFunction signatures, Boolean save_traces)

Reproduction Steps

No response

Known Workarounds

No response

Configuration and Other Information

Windows Server 2022, Windows 10
Tensorflow.NET v 0.150
Tensorflow.Keras v 0.15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant