Issue #83 Inference Edwin Dong, Lalit Julapalli
GPU Optimization for Machine Learning, Simply Explained
Free editionPRO edition
This week’s PRO issue is accompanied by a Colab notebook with a deep dive into GPU optimization for ML. The notebook writes a custom GPU kernel for tiling, dramatically reducing the number of expensive global memory accesses.
As always, the code in this issue can be executed on a normal home PC and does not require a large amount of VRAM.
Machine Learning (ML) revolves around one central operation: matrix multiplication. Depending on the size of these matrices, the computation can be decently fast, but that’s a rare phenomenon in many applications—where matrices can have millions of entries.
Graphical Processing Units (GPUs), originally intended for graphical display and gaming, solve the problem of slow computation with the ability of parallel processing.
Instead of computing repetitive tasks with limited parallelism (although not none), which is the strategy of Central Processing Units (CPUs), GPUs split these tasks into different threads and complete them simultaneously.
Companies like NVIDIA, AMD, and Intel compete to create more capable GPUs, which are especially in demand due to the field of machine learning, where processing loads of millions of data points is a routine task.
In this issue, we’ll break down:
The infrastructure of a GPU (where its parallel processing comes from)
Methods of optimizing its performance
This edition is for PRO members.
