What is automatic mixed precision? How is it implemented? What does it have to do with mode dispatch keys, fallthrough kernels? What are AMP policies? How is its cast caching implemented? How does torchvision also support AMP? What's up with Intel's CPU autocast implementation?Further reading.<ul><li>Autocast implementation lives at <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/autocast_mode.cpp">https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/autocast_mode.cpp</a></li><li>How to add autocast implementations to custom operators that are out of tree <a href="https://pytorch.org/tutorials/advanced/dispatcher.html#autocast">https://pytorch.org/tutorials/advanced/dispatcher.html#autocast</a></li><li>CPU autocasting PR <a href="https://github.com/pytorch/pytorch/pull/57386">https://github.com/pytorch/pytorch/pull/57386</a></li></ul>

Shared memory

The PyTorch Developer Podcast is a place for the PyTorch dev team to do bite sized (10-20 min) topics about all sorts of internal development topics in PyTorch.