t-SNE Overview

Linear Algebra for Machine Learning · Axiom Academy

The most popular visualization technique for high-dimensional data t-SNE stands for "t-Distributed Stochastic Neighbor Embedding." It's a powerful technique for visualizing high-dimensional data by converting pairwise distances into probabilities and minimizing their divergence. Convert Euclidean distances to similarities using a Gaussian kernel. Points that are close have high similarity. Interpret similarities as conditional probabilities: p(j|i) = probability that i would pick j as a neighbor. Place points randomly in 2D or 3D space Use Student-t distribution instead of Gaussian. This has heavier tails, which helps preserve global structure better. Adjust low-dimensional positions to minimize KL divergence between high-D and low-D probability distributions. In high dimensions, compute pairwise Euclidean distances. Convert to probabilities: σ is a bandwidth parameter (usually chosen per-point). For low dimensions, use Student-t distribution instead of Gaussian: The heavier tails prevent overcrowding of points. Minimize the Kullback-Leibler divergence between P and Q distributions: This cost function is optimized using gradient descent. It's related to σ but chosen automatically via binary search. Typical values: 5-50. Focuses on very local structure

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.