tensorflow on OSX Mac M1 pro/max silicon 32 cores and windows 11 13900k with dual RTX-A4500/A4000 workstation and dual GTX-4090 consumer

[obriensystems]

See move/updates on M4 Max, RTX-3500, RTX-6000
#71

and
ObrienlabsDev/machine-learning#30
ObrienlabsDev/machine-learning#31
ObrienlabsDev/machine-learning#32
ObrienlabsDev/machine-learning#33
ObrienlabsDev/machine-learning#34
ObrienlabsDev/machine-learning#35

GTX-4090 Ada generation consumer cards

RTX-A4500 Ampere generation professional workstation cards

Stats

https://github.com/ObrienlabsDev/blog/wiki/Machine-Learning-on-local-or-Cloud-based-NVidia-or-Apple-GPUs

Note: CPU is 340% CPU only and 100% GPU therefore 100% is CPU overhead

Mac Mini 2020 M1

Macbook Pro 14 M1 Pro 16G 4p/4e 8 core GPU = 516ms/step CPU at 50%, 79ms/step GPU = 6.5x faster GPU
Macbook Pro 16 M1 Pro 32G 8p/1e 32 core GPU = 437ms/step CPU at 50%, 54ms/step GPU = 10.5x faster GPU and 1.2x/1.5x faster than M1 Pro (49ms using 32 batch down from 64 - matching GPU size - 2.4/32G vram)

Lenovo P17 Gen1 128g RTX-5000 TU104 using batch of 5120 = 190us and 15.6/16G vram

follow
https://developer.apple.com/metal/tensorflow-plugin/
better
https://www.mrdbourke.com/setup-apple-m1-pro-and-m1-max-for-machine-learning-and-data-science/

base system M1 Pro 4p/4e 8 core GPU

  181  ./Miniforge3-MacOSX-arm64.sh
  183  source ~/miniforge3/bin/activate
  185  cd wse_github
  186  cd tensor
  187  mkdir tensorflow-test
  188  cd tensorflow-test
  189  conda create --prefix ./env python=3.8 
  190  conda activate ./env
  191  conda install -c apple tensorflow-deps
  192  python -m pip install tensorflow-macos
  193  python -m pip install tensorflow-metal
  194  python -m pip install tensorflow-datasets
  195  conda install jupyter pandas numpy matplotlib scikit-learn
  196  jupyter notebook

other tab
vi tftest.py
 python tftest.py
 pip install numpy
 pip install pandas
 pip install sklearn
 pip install -U scikit-learn scipy matplotlib
 pip install -U tensorflow
  
(base) ..ien@mbp6 tensorflow-test % python tftest.py         
TensorFlow has access to the following devices:
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]
TensorFlow version: 2.14.0


missing gpu
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]


try
import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))


(base) michaelobrien@mbp6 tensorflow-test % python gpu.py   
[]

[obriensystems]

running from
https://developer.apple.com/metal/tensorflow-plugin/

only using cpu on m1 pro - checking m1-max

import tensorflow as tf

cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, batch_size=64)


(base) michaelobrien@mbp6 tensorflow-test % python tflow.py 
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 6s 0us/step
Epoch 1/5
205/782 [======>.......................] - ETA: 4:56 - loss: 5.1091 - accuracy: 0.0474


782/782 [==============================] - 412s 525ms/step - loss: 4.7090 - accuracy: 0.0739
Epoch 2/5
372/782 [=============>................] - ETA: 3:37 - loss: 4.2690 - accuracy: 0.1032


(base) michaelobrien@mbp6 tensorflow-test % python tflow.py 
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 6s 0us/step
Epoch 1/5
782/782 [==============================] - 412s 525ms/step - loss: 4.7090 - accuracy: 0.0739
Epoch 2/5
782/782 [==============================] - 415s 530ms/step - loss: 4.3030 - accuracy: 0.1040
Epoch 3/5
782/782 [==============================] - 414s 529ms/step - loss: 4.1217 - accuracy: 0.1116
Epoch 4/5
782/782 [==============================] - 404s 516ms/step - loss: 3.8518 - accuracy: 0.1589
Epoch 5/5
782/782 [==============================] - 403s 516ms/step - loss: 3.5352 - accuracy: 0.1952

[obriensystems]

try https://github.com/apple/tensorflow_macos
to get the gpu working on m1 mac

(/Users/michaelobrien/wse_github/tensor/tensorflow-test/env) michaelobrien@mbp6 tensorflow-test % /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/apple/tensorflow_macos/master/scripts/download_and_install.sh)" 

ERROR: TensorFlow with ML Compute acceleration is only available on macOS 11.0 and later.

i have 13.5

actually that repo is 2y old and links back to https://developer.apple.com/metal/tensorflow-plugin/

(/Users/michaelobrien/wse_github/tensor/tensorflow-test/env) michaelobrien@mbp6 tensorflow-test % python -m pip install tensorflow-metal
Requirement already satisfied: tensorflow-metal in ./env/lib/python3.8/site-packages (1.0.1)
Requirement already satisfied: wheel~=0.35 in ./env/lib/python3.8/site-packages (from tensorflow-metal) (0.41.2)
Requirement already satisfied: six>=1.15.0 in ./env/lib/python3.8/site-packages (from tensorflow-metal) (1.16.0)

[obriensystems]

try again

(base) michaelobrien@mbp6 tensorflow-test % pip install tensorflow-metal
Collecting tensorflow-metal
  Obtaining dependency information for tensorflow-metal from https://files.pythonhosted.org/packages/f3/3d/0796dda099a84e166aacb493f8a161c8816175e514e79012b940364787d4/tensorflow_metal-1.0.1-cp310-cp310-macosx_12_0_arm64.whl.metadata
  Downloading tensorflow_metal-1.0.1-cp310-cp310-macosx_12_0_arm64.whl.metadata (1.2 kB)
Requirement already satisfied: wheel~=0.35 in /Users/michaelobrien/miniforge3/lib/python3.10/site-packages (from tensorflow-metal) (0.41.2)
Requirement already satisfied: six>=1.15.0 in /Users/michaelobrien/miniforge3/lib/python3.10/site-packages (from tensorflow-metal) (1.16.0)
Downloading tensorflow_metal-1.0.1-cp310-cp310-macosx_12_0_arm64.whl (1.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 13.9 MB/s eta 0:00:00
Installing collected packages: tensorflow-metal
Successfully installed tensorflow-metal-1.0.1
(base) michaelobrien@mbp6 tensorflow-test % python gpu.py               
Traceback (most recent call last):
  File "/Users/michaelobrien/wse_github/tensor/tensorflow-test/gpu.py", line 1, in <module>
    import tensorflow as tf
  File "/Users/michaelobrien/miniforge3/lib/python3.10/site-packages/tensorflow/__init__.py", line 445, in <module>
    _ll.load_library(_plugin_dir)
  File "/Users/michaelobrien/miniforge3/lib/python3.10/site-packages/tensorflow/python/framework/load_library.py", line 151, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/michaelobrien/miniforge3/lib/python3.10/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 0x0006): Symbol not found: __ZN10tensorflow16TensorShapeProtoC1ERKS0_
  Referenced from: <10B7FC95-0B10-3E4E-84D0-79A2D52E4D78> /Users/michaelobrien/miniforge3/lib/python3.10/site-packages/tensorflow-plugins/libmetal_plugin.dylib
  Expected in:     <C104091C-297A-300E-A02F-509BCA2330E3> /Users/michaelobrien/miniforge3/lib/python3.10/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so

retry using
https://developer.apple.com/forums/thread/689300

better - see cpu and gpu

reverted from 2.14 to 2.9
conda install -c apple tensorflow-deps==2.9.0
python -m pip install tensorflow-macos==2.9.0
python -m pip install tensorflow-metal==0.5.0

6.5x faster with GPU on the lowest gpu in a m1 pro
2023-09-29 00:07:57.790054: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2023-09-29 00:07:57.790208: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2023-09-29 00:07:59.290834: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
2023-09-29 00:08:01.285460: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.
782/782 [==============================] - 75s 84ms/step - loss: 4.9904 - accuracy: 0.0426
Epoch 2/5
557/782 [====================>.........] - ETA: 17s - loss: 4.5287 - accuracy: 0.0735 


2023-09-29 00:07:57.790054: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2023-09-29 00:07:57.790208: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2023-09-29 00:07:59.290834: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
2023-09-29 00:08:01.285460: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.
782/782 [==============================] - 75s 84ms/step - loss: 4.9904 - accuracy: 0.0426
Epoch 2/5
782/782 [==============================] - 62s 80ms/step - loss: 4.5035 - accuracy: 0.0736
Epoch 3/5
782/782 [==============================] - 63s 81ms/step - loss: 4.0429 - accuracy: 0.1108
Epoch 4/5
782/782 [==============================] - 62s 80ms/step - loss: 3.8094 - accuracy: 0.1470
Epoch 5/5
782/782 [==============================] - 62s 79ms/step - loss: 3.7574 - accuracy: 0.1545
(base) michaelobrien@mbp6 tensorflow-test % python gpu.py                                
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

(base) michaelobrien@mbp6 tensorflow-test % python tftest.py 
TensorFlow has access to the following devices:
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
TensorFlow version: 2.9.0

[obriensystems]

code

(base) michaelobrien@mbp6 tensorflow-test % cat tflow.py 
import tensorflow as tf

cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, batch_size=64)

(base) michaelobrien@mbp6 tensorflow-test % cat tftest.py 
import numpy as np
import pandas as pd
import sklearn
import tensorflow as tf
import matplotlib.pyplot as plt

# Check for TensorFlow GPU access
print(f"TensorFlow has access to the following devices:\n{tf.config.list_physical_devices()}")

# See TensorFlow version
print(f"TensorFlow version: {tf.__version__}")

(base) michaelobrien@mbp6 tensorflow-test % cat gpu.py 
import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))

[obriensystems]

Restest on Macbook Pro M1 Max 32g 8p/2e 32core GPU

https://developer.apple.com/metal/tensorflow-plugin/

Test only CPU first

michaelobrien@mbp7 tensorflow % python3 -m venv ~/venv-metal
michaelobrien@mbp7 tensorflow % source ~/venv-metal/bin/activate
(venv-metal) michaelobrien@mbp7 tensorflow % python -m pip install -U pip
(venv-metal) michaelobrien@mbp7 tensorflow % python -m pip install tensorflow
(venv-metal) michaelobrien@mbp7 tensorflow % python tflow.py 
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 6s 0us/step
Epoch 1/5
782/782 [==============================] - 349s 443ms/step - loss: 4.5777 - accuracy: 0.0837
Epoch 2/5
782/782 [==============================] - 341s 437ms/step - loss: 4.0482 - accuracy: 0.1339
Epoch 3/5
782/782 [==============================] - 346s 442ms/step - loss: 3.8610 - accuracy: 0.1711
Epoch 4/5
782/782 [==============================] - 343s 439ms/step - loss: 3.5612 - accuracy: 0.1989
Epoch 5/5
782/782 [==============================] - 344s 440ms/step - loss: 4.1587 - accuracy: 0.1126

Adding GPU capability

54ms/step

python -m pip install tensorflow-metal
Successfully installed tensorflow-metal-1.1.0
(venv-metal) michaelobrien@mbp7 tensorflow % python tflow.py                       
2023-09-29 22:10:24.617100: I metal_plugin/src/device/metal_device.cc:1154] Metal device set to: Apple M1 Max
2023-09-29 22:10:24.617127: I metal_plugin/src/device/metal_device.cc:296] systemMemory: 32.00 GB
2023-09-29 22:10:24.617133: I metal_plugin/src/device/metal_device.cc:313] maxCacheSize: 10.67 GB
2023-09-29 22:10:24.617185: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2023-09-29 22:10:24.617357: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
Epoch 1/5
2023-09-29 22:10:29.517374: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:117] Plugin optimizer for device_type GPU is enabled.
782/782 [==============================] - 58s 59ms/step - loss: 4.9364 - accuracy: 0.0567   
Epoch 2/5
782/782 [==============================] - 44s 56ms/step - loss: 4.5137 - accuracy: 0.0699
Epoch 3/5
782/782 [==============================] - 42s 54ms/step - loss: 4.0512 - accuracy: 0.1155
Epoch 4/5
782/782 [==============================] - 44s 56ms/step - loss: 3.9736 - accuracy: 0.1288
Epoch 5/5
782/782 [==============================] - 43s 56ms/step - loss: 3.7527 - accuracy: 0.1531     

[obriensystems]

on slowest windows without GPU - Lenovo X1 Carbon Gen9
66%cpu on i7-1185G7 3ghz (3.5ghz)

python -m venv ~/venv-tensor
source ~/venv-tensor/scripts/activate
python -m pip install -U pip
python -m pip install tensorflow
python tflow.py 

2023-09-29 22:52:34.975337: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

 73/782 [=>............................] - ETA: 15:39 - loss: 5.5396 - accuracy: 0.0289

[obriensystems]

on fastest windows laptop - Lenovo P17 gen 1 - Xeon W-10855M 2.8Ghz with RTX-5000 (TU-104)

https://www.tensorflow.org/install/pip#windows-native

python -m venv ~/venv-tensor
source ~/venv-tensor/scripts/activate
python -m pip install -U pip
python -m pip install tensorflow

python tflow.py

2023-09-29 23:02:40.581369: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Epoch 1/5

219/782 [=======>......................] - ETA: 11:23 - loss: 4.9518 - accuracy: 0.0532

GPU support in Windows WSL
https://www.tensorflow.org/install/pip#windows-wsl2

python3 -m pip install tensorflow[and-cuda]
Collecting nvidia-cublas-cu11==11.11.3.6 (from tensorflow[and-cuda])
  Downloading nvidia_cublas_cu11-11.11.3.6-py3-none-win_amd64.whl (427.2 MB)
     -------------------------------------- 427.2/427.2 MB 7.7 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu11==11.8.87 (from tensorflow[and-cuda])
  Downloading nvidia_cuda_cupti_cu11-11.8.87-py3-none-win_amd64.whl (10.0 MB)
     --------------------------------------- 10.0/10.0 MB 21.9 MB/s eta 0:00:00
Collecting nvidia-cuda-nvcc-cu11==11.8.89 (from tensorflow[and-cuda])
  Downloading nvidia_cuda_nvcc_cu11-11.8.89-py3-none-win_amd64.whl (15.7 MB)
     --------------------------------------- 15.7/15.7 MB 25.1 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu11==11.8.89 (from tensorflow[and-cuda])
  Downloading nvidia_cuda_runtime_cu11-11.8.89-py3-none-win_amd64.whl (1.0 MB)
     ---------------------------------------- 1.0/1.0 MB 16.3 MB/s eta 0:00:00

Successfully installed gast-0.4.0 keras-2.13.1 numpy-1.24.3 tensorboard-2.13.0 tensorflow-2.13.1 tensorflow-estimator-2.13.0 tensorflow-intel-2.13.1 typing-extensions-4.5.0


not yet
$ python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
[]

[obriensystems]

windows tensorflow 1.15 only using directml
https://learn.microsoft.com/en-us/windows/ai/directml/gpu-tensorflow-wsl

   29  conda create --name directml python=3.6
   32  conda activate directml
   33  conda init bash
   34  conda activate directml

pip install tensorflow-directml
  WARNING: The script f2py.exe is installed in 'C:\Users\michael\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

70% 4090 - temp only rises 5c with the GPU at 40% power 180W of 450W

michael@13900b MINGW64 /c/wse_github/tensorflow
$ python tflow.py
WARNING:tensorflow:From C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\ops\resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Train on 50000 samples
2023-09-30 11:05:07.151873: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python/directml.d6f03b303ac3c4f2eeb8ca631688c9757b361310.dll
2023-09-30 11:05:07.152247: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library dxgi.dll
2023-09-30 11:05:07.154477: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library d3d12.dll
2023-09-30 11:05:07.987347: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:250] DirectML device enumeration: found 2 compatible adapters.
2023-09-30 11:05:07.987646: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2023-09-30 11:05:07.989784: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:186] DirectML: creating device on adapter 0 (NVIDIA GeForce RTX 4090)
2023-09-30 11:05:08.057829: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library Kernel32.dll
2023-09-30 11:05:08.058325: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:186] DirectML: creating device on adapter 1 (NVIDIA GeForce RTX 4090)
Epoch 1/5
50000/50000 [==============================] - 13s 261us/sample - loss: 4.6415 - acc: 0.0717
Epoch 2/5
50000/50000 [==============================] - 11s 221us/sample - loss: 4.0632 - acc: 0.1264
Epoch 3/5
50000/50000 [==============================] - 11s 220us/sample - loss: 3.9804 - acc: 0.1411
Epoch 4/5
50000/50000 [==============================] - 11s 220us/sample - loss: 3.5434 - acc: 0.1907
Epoch 5/5
50000/50000 [==============================] - 11s 220us/sample - loss: 3.2532 - acc: 0.2384
(directml)
michael@13900b MINGW64 /c/wse_github/tensorflow

change the batch size to 128 from 64 and to 10 epochs from 5 - power from 185 to 226 (idle 38)

$ cat tflow.py
import tensorflow as tf

cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=10, batch_size=128)


Epoch 1/10
50000/50000 [==============================] - 10s 191us/sample - loss: 4.4179 - acc: 0.0768
Epoch 2/10
50000/50000 [==============================] - 7s 141us/sample - loss: 4.1419 - acc: 0.1027
Epoch 3/10
50000/50000 [==============================] - 7s 141us/sample - loss: 3.8350 - acc: 0.1345
Epoch 4/10
50000/50000 [==============================] - 7s 144us/sample - loss: 3.5075 - acc: 0.1830
Epoch 5/10
50000/50000 [==============================] - 7s 145us/sample - loss: 3.4506 - acc: 0.2024
Epoch 6/10
50000/50000 [==============================] - 7s 145us/sample - loss: 3.1387 - acc: 0.2477
Epoch 7/10
50000/50000 [==============================] - 7s 144us/sample - loss: 3.1950 - acc: 0.2530
Epoch 8/10
50000/50000 [==============================] - 7s 143us/sample - loss: 2.8823 - acc: 0.3014
Epoch 9/10
50000/50000 [==============================] - 7s 143us/sample - loss: 2.6766 - acc: 0.3380
Epoch 10/10
50000/50000 [==============================] - 7s 142us/sample - loss: 2.4959 - acc: 0.3769

adjust the batch size based on the gpu - in this case AD102 at 16384 cores

batch = 1024

Epoch 1/10
50000/50000 [==============================] - 6s 117us/sample - loss: 4.5212 - acc: 0.0587
Epoch 2/10
50000/50000 [==============================] - 3s 59us/sample - loss: 3.6212 - acc: 0.1565
Epoch 3/10
50000/50000 [==============================] - 3s 58us/sample - loss: 3.1855 - acc: 0.2325
Epoch 4/10
50000/50000 [==============================] - 3s 58us/sample - loss: 2.8579 - acc: 0.2948
Epoch 5/10
50000/50000 [==============================] - 3s 58us/sample - loss: 2.4748 - acc: 0.3695
Epoch 6/10
50000/50000 [==============================] - 3s 58us/sample - loss: 2.1747 - acc: 0.4333
Epoch 7/10
50000/50000 [==============================] - 3s 58us/sample - loss: 2.0401 - acc: 0.4645
Epoch 8/10
50000/50000 [==============================] - 3s 58us/sample - loss: 2.3375 - acc: 0.3928
Epoch 9/10
50000/50000 [==============================] - 3s 58us/sample - loss: 1.7525 - acc: 0.5260
Epoch 10/10

350 watts

the max with 400 watt peaks is 4096 batch

import tensorflow as tf

cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=10, batch_size=4096)


Epoch 1/10
50000/50000 [==============================] - 5s 100us/sample - loss: 5.3493 - acc: 0.0235
Epoch 2/10
50000/50000 [==============================] - 3s 52us/sample - loss: 4.2236 - acc: 0.0655
Epoch 3/10
50000/50000 [==============================] - 3s 51us/sample - loss: 3.8262 - acc: 0.1216
Epoch 4/10
50000/50000 [==============================] - 3s 51us/sample - loss: 3.4908 - acc: 0.1787
Epoch 5/10
50000/50000 [==============================] - 3s 51us/sample - loss: 3.1391 - acc: 0.2404
Epoch 6/10
50000/50000 [==============================] - 3s 52us/sample - loss: 2.9317 - acc: 0.2797
Epoch 7/10
50000/50000 [==============================] - 3s 51us/sample - loss: 2.7100 - acc: 0.3233
Epoch 8/10
50000/50000 [==============================] - 3s 51us/sample - loss: 3.1979 - acc: 0.2347
Epoch 9/10
50000/50000 [==============================] - 3s 51us/sample - loss: 2.9408 - acc: 0.2769
Epoch 10/10
50000/50000 [==============================] - 3s 51us/sample - loss: 2.5449 - acc: 0.3592

[obriensystems]

force an OOM on 23/24G by using 8192 batch size on 16384 processor

model.fit(x_train, y_train, epochs=100, batch_size=10240)#7168)

2023-10-01 08:23:08.889749: I tensorflow/core/common_runtime/bfc_allocator.cc:943] Sum Total of in-use chunks: 21.66GiB
2023-10-01 08:23:08.889778: I tensorflow/core/common_runtime/bfc_allocator.cc:945] total_region_allocated_bytes_: 23297890560 memory_limit_: 23297890714 available bytes: 154 curr_region_allocation_bytes_: 4293918720
2023-10-01 08:23:08.889821: I tensorflow/core/common_runtime/bfc_allocator.cc:951] Stats:
Limit:                 23297890714
InUse:                 23255949312
MaxInUse:              23256997888
NumAllocs:                    1532
MaxAllocSize:            849346560

2023-10-01 08:23:08.889962: W tensorflow/core/common_runtime/bfc_allocator.cc:446] ****************************************************************************************************
2023-10-01 08:23:08.890007: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at dml_kernel_context.cc:132 : Resource exhausted: OOM when allocating tensor with shape[10240,1024,2,2] and type float on /job:localhost/replica:0/task:0/device:DML:0 by allocator DmlAllocator
Traceback (most recent call last):
  File "tflow.py", line 65, in <module>
    model.fit(x_train, y_train, epochs=100, batch_size=10240)#7168)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training.py", line 727, in fit
    use_multiprocessing=use_multiprocessing)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training_arrays.py", line 675, in fit
    steps_name='steps_per_epoch')
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training_arrays.py", line 394, in model_iteration
    batch_outs = f(ins_batch)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\backend.py", line 3476, in __call__
    run_metadata=self.run_metadata)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1472, in __call__
    run_metadata_ptr)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[10240,1024,2,2] and type float on /job:localhost/replica:0/task:0/device:DML:0 by allocator DmlAllocator
         [[{{node conv4_block3_3_bn/cond/FusedBatchNormV3}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

         [[loss/mul/_1553]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: OOM when allocating tensor with shape[10240,1024,2,2] and type float on /job:localhost/replica:0/task:0/device:DML:0 by allocator DmlAllocator
         [[{{node conv4_block3_3_bn/cond/FusedBatchNormV3}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.
(directml)

Also issues using multi GPU on DirectML
microsoft/tensorflow-directml#352
https://learn.microsoft.com/en-us/windows/ai/directml/gpu-faq

import tensorflow as tf

# https://www.tensorflow.org/guide/distributed_training
#
# https://www.tensorflow.org/tutorials/distribute/keras
#strategy = tf.distribute.MirroredStrategy()
#print('Number of devices: {}'.format(strategy.num_replicas_in_sync))

#NUM_GPUS = 2
#strategy = tf.contrib.distribute.MirroredStrategy()#num_gpus=NUM_GPUS)
# not working
strategy = tf.distribute.MirroredStrategy(devices=["/gpu:0", "/gpu:1"])
#WARNING:tensorflow:Some requested devices in `tf.distribute.Strategy` are not visible to TensorFlow: /replica:0/task:0/device:GPU:1,/replica:0/task:0/device:GPU:0
#Number of devices: 2


#central_storage_strategy = tf.distribute.experimental.CentralStorageStrategy()
#strategy = tf.distribute.MultiWorkerMirroredStrategy() # not in tf 1.5
#print("mirrored_strategy: ",mirrored_strategy)
#strategy = tf.distribute.OneDeviceStrategy(device="/gpu:1")
#mirrored_strategy = tf.distribute.MirroredStrategy(devices=["/gpu:0","/gpu:1"],cross_device_ops=tf.contrib.distribute.AllReduceCrossDeviceOps(all_reduce_alg="hierarchical_copy"))
#mirrored_strategy = tf.distribute.MirroredStrategy(devices= ["/gpu:0","/gpu:1"],cross_device_ops=tf.distribute.HierarchicalCopyAllReduce())

print('Number of devices: {}'.format(strategy.num_replicas_in_sync))

cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()

#with strategy.scope():
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=100, batch_size=7168)

[obriensystems]

Multi GPU specific to DirectML
https://learn.microsoft.com/en-us/windows/ai/directml/gpu-faq

or

# https://learn.microsoft.com/en-us/windows/ai/directml/gpu-faq
a = tf.constant([1.])
b = tf.constant([2.])
c = tf.add(a, b)

gpu_config = tf.GPUOptions()
gpu_config.visible_device_list = "1"#"0"

session = tf.Session(config=tf.ConfigProto(gpu_options=gpu_config))
print(session.run(c))

tensorflow/tensorflow#19083
tensorflow/tensorflow#18861 (comment)

michael@13900b MINGW64 /c/wse_github/tensorflow
$ python tflow.py
WARNING:tensorflow:From tflow.py:32: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.

WARNING:tensorflow:From tflow.py:36: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

WARNING:tensorflow:From tflow.py:36: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

2023-09-30 22:42:26.532728: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2023-09-30 22:42:26.535407: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python/directml.d6f03b303ac3c4f2eeb8ca631688c9757b361310.dll
2023-09-30 22:42:26.535750: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library dxgi.dll
2023-09-30 22:42:26.537833: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library d3d12.dll
2023-09-30 22:42:27.425768: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:250] DirectML device enumeration: found 2 compatible adapters.
2023-09-30 22:42:27.425872: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:186] DirectML: creating device on adapter 1 (NVIDIA GeForce RTX 4090)
2023-09-30 22:42:27.490844: I tensorflow/stream_executor/platform/default/dso_loader.cc:97] Successfully opened dynamic library Kernel32.dll
[3.]
WARNING:tensorflow:From C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\ops\resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Train on 50000 samples
2023-09-30 22:42:30.497139: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:186] DirectML: creating device on adapter 0 (NVIDIA GeForce RTX 4090)
Traceback (most recent call last):
  File "tflow.py", line 53, in <module>
    model.fit(x_train, y_train, epochs=100, batch_size=7168)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training.py", line 727, in fit
    use_multiprocessing=use_multiprocessing)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training_arrays.py", line 675, in fit
    steps_name='steps_per_epoch')
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training_arrays.py", line 271, in model_iteration
    model.reset_metrics()
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\training.py", line 914, in reset_metrics
    m.reset_states()
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\metrics.py", line 210, in reset_states
    K.batch_set_value([(v, 0) for v in self.variables])
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\backend.py", line 3259, in batch_set_value
    get_session().run(assign_ops, feed_dict=feed_dict)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\backend.py", line 483, in get_session
    session = _get_session(op_input_list)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\backend.py", line 455, in _get_session
    config=get_default_session_config())
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1585, in __init__
    super(Session, self).__init__(target, graph, config=config)
  File "C:\Users\michael\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 699, in __init__
    self._session = tf_session.TF_NewSessionRef(self._graph._c_graph, opts)
tensorflow.python.framework.errors_impl.AlreadyExistsError: TensorFlow device (DML:0) is being mapped to multiple DML devices (0 now, and 1 previously), which is not supported. This may be the result of providing different GPU configurations (ConfigProto.gpu_options, for example different visible_device_list) when creating multiple Sessions in the same process. This is not  currently supported, see https://github.com/tensorflow/tensorflow/issues/19083
(directml)

check
https://keras.io/guides/distributed_training/