19:I[66739,["/_next/static/chunks/0.3dg.jzl0xwf.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA","/_next/static/chunks/0qxr~fpya2kw0.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA","/_next/static/chunks/12wfgjmhh21qu.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA","/_next/static/chunks/028~snn2srcd-.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA","/_next/static/chunks/079qbl7l5rd72.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA","/_next/static/chunks/0hi8o2xq05aig.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA"],"default"] 1b:I[38632,["/_next/static/chunks/0qqtp3r0h7kah.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA","/_next/static/chunks/0iukvycbd42rl.js?dpl=dpl_BhmHa8NmNRnoMAUm8Lqp3TPz2qZA"],"IconMark"] 1a:T4d0,

Ever wanted to learn about CUDA but not sure where to start? In this sixteen minute episode I try to jam in as much CUDA knowledge as could be reasonably expected in a podcast. You won't know how to write a kernel after this episode, but you'll know about what a GPU is, what the general CUDA programming model is, why asynchronous execution makes everything complicated, and some general principles PyTorch abides by when designing CUDA kernels.

Further reading:

PyTorch docs on CUDA semantics https://pytorch.org/docs/stable/notes/cuda.html
The book I was recommended for learning CUDA when I first showed up at PyToch: Programming Massively Parallel Processors https://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/dp/0128119861
The environment variable that makes CUDA synchronous is CUDA_LAUNCH_BLOCKING=1. cuda-memcheck is also useful for debugging CUDA problems https://docs.nvidia.com/cuda/cuda-memcheck/index.html

Just enough CUDA to be dangerous

PyTorch Developer Podcast

Just enough CUDA to be dangerous