Today I'm going to talk about a famous issue in PyTorch, DataLoader with num_workers > 0 causes memory leak (<a href="https://github.com/pytorch/pytorch/issues/13246">https://github.com/pytorch/pytorch/issues/13246</a>). This bug is a good opportunity to talk about DataSet/DataLoader design in PyTorch, fork and copy-on-write memory in Linux and Python reference counting; you have to know about all of these things to understand why this bug occurs, but once you do, it also explains why the workarounds help.Further reading.<ul><li>A nice summary of the full issue <a href="https://github.com/pytorch/pytorch/issues/13246#issuecomment-905703662">https://github.com/pytorch/pytorch/issues/13246#issuecomment-905703662</a></li><li>DataLoader architecture RFC <a href="https://github.com/pytorch/pytorch/issues/49440">https://github.com/pytorch/pytorch/issues/49440</a></li><li>Cinder Python <a href="https://github.com/facebookincubator/cinder">https://github.com/facebookincubator/cinder</a></li></ul>

Half precision

The PyTorch Developer Podcast is a place for the PyTorch dev team to do bite sized (10-20 min) topics about all sorts of internal development topics in PyTorch.