* Added initial notes concerning the role of floating point precision

in deep learning applications.
Wirawan Purwanto 7 months ago
parent 1d9a91f698
commit 1c0f2aa577
  1. 28

@ -0,0 +1,28 @@
Notes on Floating Point Precisions in Deep Learning Computations
ECCV 2020 Tutorial on Accelerating Computer Vision with Mixed Precision
Topics of the tutorial:
* Training Neural Networks with Tensor Cores
* PyTorch Performance Tuning Guide
* Mixed Precision Training for Conditional GANs
* Mixed Precision Training for FAZE: Few-shot Adaptive Gaze Estimation
* Mixed Precision Training for Video Synthesis
* Mixed Precision Training for Convolutional Tensor-Train LSTM
* Mixed Precision Training for 3D Medical Image Analysis
Has PDF of the slides and the videos.
**What's the difference between FP32 and TF32 modes?**
FP32 cores perform scalar instructions. TF32 is a Tensor Core mode,
which performs matrix instructions - they are 8-16x faster and more
energy efficient. Both take FP32 as inputs. TF32 mode also rounds
those inputs to TF32.