Batch Processing & Vectorization

Linear Algebra for Machine Learning · Axiom Academy

Batch Processing: Multiple Inputs How X × W transforms multiple samples simultaneously Training one sample at a time is slow. Modern deep learning processes batches of 32-256 samples simultaneously: Efficiency: GPUs excel at matrix operations; batching maximizes parallelism Gradient stability: Batch gradients are more stable than single-sample gradients Speed: Processing 32 samples costs roughly the same as 1 sample on GPU Weight Matrix (same as before) The same bias is added to each sample. This makes sense: biases shift the decision boundary identically for all inputs. Key Insight: Vectorized Operations Batch processing isn't a different operation—it's the same operation applied to multiple samples in parallel. If y = Wx for a single sample, then Y = XW^T for a batch of samples. This vectorization is why GPUs are so efficient: they multiply entire matrices in parallel. Batch size: Typical values are 32, 64, 128, 256. Larger batches are more efficient but use more memory. Memory: A batch of 256 images (224×224×3) = 256 × 150,528 ≈ 38.5 million values Gradient averaging: During backprop, gradients are typically averaged over the batch Mini-batch SGD: Using batches is "stochastic" gradient descent because you're computing gradients on a sample of data Unit 8: Linear Algebra in Neural Networks | Module 5: Batch Processing

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.