Turning Images Into Sound, One Pixel at a Time

SUMMARY

In 30 seconds

A 3Blue1Brown short introduces one possible way to encode images as sound. The speaker imagines assigning each pixel a unique frequency and playing brighter pixels louder, leaving the listener to interpret the combined signal.

1Start with a small imageThe example uses a 256-by-256-pixel image.
2Give each pixel a frequencyA unique tone can represent each position in the image.
3Brightness becomes volumeBrighter pixels produce louder components in the combined sound.

Takeaways

An image can be treated as a grid of pixels.
Each pixel can be assigned a distinct frequency whose volume reflects brightness.
The mixed sound contains image information but may initially be difficult to interpret.

TopicsMathematics Science Communication

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

Representing the image

A low-resolution image is represented as a grid of pixels.

Suppose we want software that lets people perceive images through sound. To make an initial experiment simpler, we can start with a low-resolution image, perhaps 256 by 256 pixels, and represent it as a square grid.

Mapping pixels to sound

Frequency represents position and volume represents brightness; all tones are played together.

SLIDE 1 / 2A 256-by-256 imageThe image is treated as a grid whose pixels become inputs to the sound encoding.

One approach is to assign a unique sound frequency to each pixel. The position in the image determines which frequency represents it.

SLIDE 2 / 2Many frequencies played togetherThe sound combines components representing different pixel positions.

Play the frequency louder when its pixel is bright and more quietly when it is dark. Playing all the frequencies together produces a mixture whose strongest components correspond to brighter regions. At first it may sound like a cacophony; the idea is that a listener could learn to interpret the information within it.