Anybody have the source code of neural network simulation of LTP and LTD data of memristor?

[xqli666]

Anybody have the source code of neural network simulation of LTP and LTD data of memristor?

[ptxiao]

To simulate memristive neural network training accelerators, please use CrossSim v2.0: https://github.com/sandialabs/cross-sim/releases/tag/v2.0
You can find an example script to run a training simulation in /training/ML_training.py.

This code has an option to use LTP and LTD data from memristor devices in the form of conductance update lookup tables. The raw data can be found here: https://github.com/sandialabs/cross-sim-data/tree/main/lookup_tables

[xjj19]

To simulate memristive neural network training accelerators, please use CrossSim v2.0: https://github.com/sandialabs/cross-sim/releases/tag/v2.0 You can find an example script to run a training simulation in /training/ML_training.py.

This code has an option to use LTP and LTD data from memristor devices in the form of conductance update lookup tables. The raw data can be found here: https://github.com/sandialabs/cross-sim-data/tree/main/lookup_tables

For ADM graphics cards, the following installation package cannot be installed
‘’CuPy 8.3.0 with CUDA 10.2, CuPy 10.3.1 with CUDA 11.2 (if GPU acceleration is enabled)‘’
What should I do to ensure that the program （/training/ML_training.py.） runs correctly？

[bfeinberg]

CuPy support for AMD GPUs using ROCm is currently still experimental, you can find more information here if you are interested: https://docs.cupy.dev/en/stable/install.html#using-cupy-on-amd-gpu-experimental. We've never tested this code using CuPy on ROCm so I have no idea if things will work or perform well, but if you want GPU acceleration with AMD GPUs this is where to start.

Alternatively, if you don't need GPU support you can simply set useGPU to False in the ML_training script: https://github.com/sandialabs/cross-sim/blob/v2.0/training/MLP_training.py#L43. This will use numpy on your CPU and should work.

[KevinNadaud]

Hi,
I have small bug using the MLP_training.py code:

`python3 MLP_training.py
Training on dataset: small

Global max conductance = 150.2796 uS
Global min conductance = 4.1782 uS
The lookup table range used is 18.7884 uS to 135.6695 uS.
The xbar weight limits will be rescaled accordingly
##########
Numeric (ideal), Run 0
##########
Matrix 1 Weight Limit = 1.299
Matrix 2 Weight Limit = 2.895

iteration // avg loss (train) // max loss (train) // fraction correct (test)
0 // 0 // 0 // 0.0984975
1 // 0.868029 // 8.83408 // 0.839176
2 // 0.378953 // 7.991 // 0.864775
3 // 0.310363 // 8.72888 // 0.874791
4 // 0.275783 // 9.0927 // 0.87813
5 // 0.253807 // 9.28997 // 0.885364
Reduced learning rate from 0.0500 to 0.0375
6 // 0.236123 // 9.37349 // 0.888703
7 // 0.225875 // 9.38157 // 0.892599
8 // 0.217274 // 9.35969 // 0.894268
9 // 0.209822 // 9.31899 // 0.897607
10 // 0.20321 // 9.23404 // 0.901503
Reduced learning rate from 0.0375 to 0.0281
11 // 0.196374 // 9.03091 // 0.903728
12 // 0.191944 // 8.95371 // 0.906511
13 // 0.188132 // 8.8716 // 0.910406
14 // 0.184639 // 8.79118 // 0.911519
15 // 0.181356 // 8.70831 // 0.913189
Traceback (most recent call last):
File "/home/nadaud/Téléchargements/CrossSim2.0/training/MLP_training.py", line 247, in
results[:,0,i_run], deltaW_info[i_run][0] = train_net.train(filename=task+"_numeric.txt",dataset=task,params=params_numeric,n_epochs=Nepochs,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nadaud/Téléchargements/CrossSim2.0/training/training_net.py", line 893, in train
bp.write_weights_npz(savePath=saveModelPath)
File "/home/nadaud/Téléchargements/CrossSim2.0/training/../cross_sim/cross_sim/backprop/backprop.py", line 475, in write_weights_npz
np.savez(savePath, mats=matrix_list)
File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 639, in savez
_savez(file, args, kwds, False)
File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 740, in _savez
val = np.asanyarray(val)
^^^^^^^^^^^^^^^^^^
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
`

Are they any setting I need to change or package I need to update ?

[KickYourPort]

Hi, I have small bug using the MLP_training.py code:

`python3 MLP_training.py Training on dataset: small

Global max conductance = 150.2796 uS Global min conductance = 4.1782 uS The lookup table range used is 18.7884 uS to 135.6695 uS. The xbar weight limits will be rescaled accordingly ########## Numeric (ideal), Run 0 ########## Matrix 1 Weight Limit = 1.299 Matrix 2 Weight Limit = 2.895

iteration // avg loss (train) // max loss (train) // fraction correct (test) 0 // 0 // 0 // 0.0984975 1 // 0.868029 // 8.83408 // 0.839176 2 // 0.378953 // 7.991 // 0.864775 3 // 0.310363 // 8.72888 // 0.874791 4 // 0.275783 // 9.0927 // 0.87813 5 // 0.253807 // 9.28997 // 0.885364 Reduced learning rate from 0.0500 to 0.0375 6 // 0.236123 // 9.37349 // 0.888703 7 // 0.225875 // 9.38157 // 0.892599 8 // 0.217274 // 9.35969 // 0.894268 9 // 0.209822 // 9.31899 // 0.897607 10 // 0.20321 // 9.23404 // 0.901503 Reduced learning rate from 0.0375 to 0.0281 11 // 0.196374 // 9.03091 // 0.903728 12 // 0.191944 // 8.95371 // 0.906511 13 // 0.188132 // 8.8716 // 0.910406 14 // 0.184639 // 8.79118 // 0.911519 15 // 0.181356 // 8.70831 // 0.913189 Traceback (most recent call last): File "/home/nadaud/Téléchargements/CrossSim2.0/training/MLP_training.py", line 247, in results[:,0,i_run], deltaW_info[i_run][0] = train_net.train(filename=task+"_numeric.txt",dataset=task,params=params_numeric,n_epochs=Nepochs, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/nadaud/Téléchargements/CrossSim2.0/training/training_net.py", line 893, in train bp.write_weights_npz(savePath=saveModelPath) File "/home/nadaud/Téléchargements/CrossSim2.0/training/../cross_sim/cross_sim/backprop/backprop.py", line 475, in write_weights_npz np.savez(savePath, mats=matrix_list) File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 639, in savez _savez(file, args, kwds, False) File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 740, in _savez val = np.asanyarray(val) ^^^^^^^^^^^^^^^^^^ ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part. `

Are they any setting I need to change or package I need to update ?

Met with the same problem. Have you fixed this error?

[KevinNadaud]

I find a temporary fix, do not load/save the results. Remove the arguments:
loadModelPath=loadModelPaths[i_run][0],saveModelPath=saveModelPaths[i_run][0], (around line 247)
loadModelPath=loadModelPaths[i_run][1],saveModelPath=saveModelPaths[i_run][1], (around line 256)
loadModelPath=loadModelPaths[i_run][2],saveModelPath=saveModelPaths[i_run][2], (around line 265)

It is not perfect, but its work.

[KickYourPort]

I also tried that but found the trained model was not saved. Still need the weight map of the network.

[KevinNadaud]

#31