What's the current state of backend extensibility? How did PyTorch evolve from being a CPU and CUDA only framework to also support AMD ROCm and XLA? What are some problems with adding an out-of-tree backend, and what's some work to make it better?Further reading:<ul><li>Script for HIPifying PyTorch's source when enabling ROCm <a href="https://github.com/pytorch/pytorch/blob/master/tools/amd_build/build_amd.py">https://github.com/pytorch/pytorch/blob/master/tools/amd_build/build_amd.py</a></li><li>PyTorch/XLA <a href="https://github.com/pytorch/xla/">https://github.com/pytorch/xla/</a></li><li>Brian Hirsh's spec on what out-of-tree backend codegen looks like <a href="https://github.com/pytorch/xla/issues/2871">https://github.com/pytorch/xla/issues/2871</a></li></ul>

The life and death of Variable

The PyTorch Developer Podcast is a place for the PyTorch dev team to do bite sized (10-20 min) topics about all sorts of internal development topics in PyTorch.