Convolution: Adding Probability Distributions

Grant Sanderson asks what distribution results when two independent normally distributed random variables are added. This lesson builds the convolution machinery needed to answer; it leaves the visual proof of the normal case for the next video.

More generally, convolution combines two distributions to describe the sum of independent random variables. Sanderson develops two visualizations: diagonal slices of a joint distribution and a flipped, sliding product.

Begin with two weighted dice. Their outcomes form a six-by-six grid, and for independent rolls the probability of each pair is the product of the two individual probabilities.

All outcome pairs with the same sum lie along a diagonal of that grid. Add the probabilities along a diagonal to find the probability of that sum.

A second visualization flips one distribution and slides it past the other. At each position, multiply aligned probabilities and sum them; the aligned values are exactly the pairs producing one chosen sum.

For discrete variables, P(X + Y = s) = Σₓ P(X = x)P(Y = s − x). Impossible outcomes contribute zero. The formula depends on independence for the multiplication step.

For continuous variables, a density is not the probability of a single point. Probability over an interval is the area under the density curve over that interval.

The convolution of densities f and g is (f * g)(s) = ∫ f(x)g(s − x) dx over the real line. It combines all possible pairs constrained to add to s.

To visualize the formula, flip g horizontally, shift it by s, multiply it pointwise by f, and integrate the product. Changing s changes the overlap and traces the output density.

If X and Y are each uniform on [−1/2, 1/2], their densities are rectangular. The overlap grows and then shrinks linearly as one rectangle slides, producing a triangular density for the sum on [−1, 1].

To add a third independent uniform variable, convolve the triangular result with another rectangle. This acts as a moving average, producing a smoother curve. Repeating the operation describes larger sums.

After centering and rescaling appropriately, sums of independent identically distributed variables with finite, nonzero variance approach a normal distribution under the central limit theorem. Without rescaling, the distribution spreads rather than retaining a fixed bell shape.

For the second continuous visualization, place f(x)g(y) above the xy-plane. Independence makes this product the joint density. The plane x + y = s cuts the surface along all pairs with the desired sum.

The slice corresponds to the same pointwise product as the flip-and-slide picture. Its geometric area differs from the convolution value by a factor of √2 because distance along the diagonal is √2 times the change in x.

This diagonal picture also makes commutativity intuitive: swapping f and g exchanges the axes but leaves the distribution of X + Y unchanged.

The opening normal-distribution question can be solved by integrating the convolution. Sanderson proposes instead using the diagonal-slice geometry and invites viewers to consider equal standard deviations before the next lesson.