- Lecture: slides
- Seminar: folder
- Homework: see homework/README.md
- Liger Kernel - Efficient Triton kernels for LLM training
- Liger Kernel Paper - Technical report with kernel details and benchmarks
- Cut Cross-Entropy - Memory-efficient CE for large vocabularies
- torch.compile API - Official API reference for torch.compile
- TorchDynamo Deep-Dive - How TorchDynamo captures and transforms Python bytecode
- Torch Compiler Troubleshooting - Debugging graph breaks and recompilations
- torch.compile: The Missing Manual - Comprehensive guide with practical tips
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Deep dive into GPU architecture and CUDA matmul optimization
- GPU Mode: Optimizing Optimizers - Video on fusing optimizer operations with foreach and torch.compile
- Essential AI: Blog on efficient communications for Muon - Layer sharding for large‑scale training with Muon
- Efficient Muon implementation in Dion library - Efficient implementations of orthonormal optimizers for distributed ML training