Testing Meta’s SAM Audio: Isolating Voices and Instruments

SUMMARY

In 30 seconds

Matthew Berman tests Meta’s SAM Audio playground with game footage, a café recording and music. He separates a target sound from the remaining audio, applies effects and describes possible uses. The video was made in partnership with Meta; hearing-aid applications are a future possibility rather than a demonstrated product.

1Meta partnershipBerman discloses that he partnered with Meta for the demonstration.
2Separate sound tracksA prompt selects a woman’s sound in a game clip, and the interface offers isolated and remaining tracks.
3Café soundsHe tries voice, footsteps and utensils in a noisy restaurant video.
4Music and effectsHe isolates guitar, applies effects and describes downloading tracks.

Takeaways

A text prompt can identify a sound to isolate from a video or audio clip.
The playground presents the original, isolated sound and remaining audio as separate tracks.
Berman tests speech, footsteps, utensils and guitar, then applies effects.
The video is a Meta partnership; the examples are demonstrations rather than a comprehensive evaluation.

TopicsMultimodal AI Generative media Product & UX

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

Sound separation in video

Berman introduces the Meta partnership and isolates a sound in game footage.

Meta’s SAM Audio can isolate sounds from audio or video using a prompt. In this video, made in partnership with Meta, I demonstrate its playground and show how the separated tracks can be used.

I upload a Tomb Raider clip and enter “woman.” The interface produces the original audio, the isolated sound and the remaining sound. I play the isolated track and then the remainder; the result sounds unusual to me, but the selected voice is separated in this example.

Voices and background sounds

He separates speech and other sounds from a café clip, then applies a voice effect.

Next I use a café video where a woman speaks on the phone amid background noise. I enter “voice” and listen to the isolated speech and the remainder. I also try “footsteps” and “utensils” to select other sounds from the same scene.

SLIDE 1 / 2Isolated and remaining audioThe café example shows separate tracks and controls for applying effects.

The playground offers effects on the isolated audio or the remainder. I apply a studio-sound effect to the voice and adjust its level; other options include reverb and more stylized effects.

Music, effects and export

He tests instrument isolation and effects, then discusses possible audio-production uses.

For a music clip I isolate guitar, listen to the remaining track and try concert-hall and underwater effects.

SLIDE 2 / 2Effects on a music clipThe interface displays the original, isolated and remaining tracks with an underwater effect selected.

The interface lets me download tracks individually. I see potential uses in cleanup and mixing; a hearing aid using similar technology is an idea for the future, not something demonstrated here.