What is vmap? How is it implemented? How does our implementation compare to JAX's? What is a good way of understanding what vmap does? What's up with random numbers? Why are there some issues with the vmap that PyTorch currently ships?Further reading.<ul><li>Tracking issue for vmap support <a href="https://github.com/pytorch/pytorch/issues/42368">https://github.com/pytorch/pytorch/issues/42368</a></li><li>BatchedTensor source code <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/BatchedTensorImpl.h">https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/BatchedTensorImpl.h</a> , logical-physical transformation helper code <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/VmapTransforms.h">https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/VmapTransforms.h</a> (well documented, worth a read)</li><li>functorch, the better, more JAX-y implementation of vmap <a href="https://github.com/facebookresearch/functorch">https://github.com/facebookresearch/functorch</a></li><li>Autodidax <a href="https://jax.readthedocs.io/en/latest/autodidax.html">https://jax.readthedocs.io/en/latest/autodidax.html</a> which contains a super simple vmap implementation that is a good model for the internal implementation that PyTorch has</li></ul>

Random number generators

The PyTorch Developer Podcast is a place for the PyTorch dev team to do bite sized (10-20 min) topics about all sorts of internal development topics in PyTorch.