Turning 2D Images Into 3D Objects With Meta SAM 3D

SUMMARY

In 30 seconds

Matthew Berman shows how he used Meta’s SAM 3D to turn a character image into a 3D-printable model, then demonstrates selecting and refining objects in a sample scene. He discloses that Meta partnered with him on the video.

1From character image to printBerman converts an image of Spike from Brawl Stars into a model and prints it.
2Select and generateHe clicks an object in a sample scene, checks the highlight and generates a 3D model.
3Refine the selectionHe adds the record player’s glass cover after the initial selection captures only the base.

Takeaways

SAM 3D can generate a 3D object from a selected object in a 2D image.
Selections may need manual correction when an object is only partly highlighted.
Generated models can be sent to 3D-printing software or given visual effects.
The video demonstrates scene objects but does not walk through the separate body-creation feature.

TopicsGenerative media Computer Vision Multimodal AI

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

A 3D-printed character

Berman shows his image-to-print workflow and discloses his partnership with Meta.

SLIDE 1 / 3Preparing the generated character for printingThe generated Spike model appears in 3D-printing software.

I took an image of Spike from Brawl Stars, loaded it into Meta’s SAM 3D, converted it to a 3D model and then printed it using my 3D-printing software. The example shows a route from a 2D reference to a physical object, though it is one demonstration rather than a measure of every model’s print readiness.

I describe SAM 3D as an open-source, open-weights model for extracting an object from a 2D image and generating a 3D representation. Meta partnered with me on this video. The interface offers options to create 3D scenes and 3D bodies; here I demonstrate the scene option.

Selecting objects and generating models

He turns a speaker and a record player from a sample image into 3D objects, correcting a partial selection.

Using a provided sample image, I click a speaker. The interface highlights the selected object, and I choose Generate 3D to produce a model from it. Clicking an object is the starting point, but the selection still needs to cover the part I actually want.

SLIDE 2 / 3A partial record-player selectionThe highlight initially captures the base, motivating the selection adjustment.

I then select a record player. The initial highlight includes only the bottom, so I remove the old selection and add the glass cover to include it in the object. After refining the selection, I generate the 3D model again.

Applying effects

He shows several effects that can be applied to a generated object.

After generation, I open the effects menu. The available examples include a shimmer, a gold appearance and an explode effect. These are visual treatments applied to the generated object.