nvidia_mixed_precision_training

Provide the latest Nvidia mixed-precision training resources/references/samples.
Any issue pls contact [email protected]

General resources

New mixed-precision model examples:
https://developer.nvidia.com/deep-learning-examples

Mixed-precision blog:
https://devblogs.nvidia.com/mixed-precision-training-deep-neural-networks/

Mixed-precision best practices:
https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html

DGX/NGC deep learning framework containers - Latest versions of our software stack and Tensor core optimized examples:
https://www.nvidia.com/en-us/gpu-cloud/deep-learning-containers/

GTC 2018 Sessions:
Training Neural Networks with Mixed Precision
Theory and Practice and Training Neural Networks with Mixed Precision: Real Examples

Updates 07/30/2018:

1. OpenSeq2Seq toolkit
   Code repo: https://github.com/NVIDIA/OpenSeq2Seq
   Paper: https://arxiv.org/abs/1805.10387
   
2. Tensorflow recently added mixed-precision loss scaling code
   https://www.tensorflow.org/api_docs/python/tf/contrib/mixed_precision
   
3. Nvidia RN50 example uses horovod for multi-node and mixed-precision
   https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Classification/imagenet