Ways to Try DeepSeek R1: Hosted and Local Distilled Models

RÉSUMÉ

En 30 secondes

Matthew Berman compares ways to use DeepSeek R1 or its distilled variants at the time of the video. He demonstrates hosted chat, Groq and local inference, explains model size and quantization, and discloses investments in Groq and LM Studio.

1Hosted chatThe demonstration selects DeepThink (R1) in DeepSeek’s interface.
2Groq’s distillThe hosted Llama 70B distill is not the full R1 model.
3Choosing a local fileQuantization trades memory use against potential model quality.
4Local inferenceBerman loads an 8B distill and shows speed on his particular hardware.

À retenir

DeepSeek’s hosted chat and a hosted R1 distill are distinct options.
Distilled models are not the full DeepSeek R1 model.
Local model size and quantization determine memory needs and performance.
Hosted services and local setups have different data-handling considerations.
The speeds shown are examples from particular hardware and services.

ThèmesLocal inference Open-weight models Quantization

TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Afficher la transcription

Transcription seule

Hosted options

Berman shows DeepSeek’s chat and a distilled model on Groq. He discusses privacy and discloses an investment.

I demonstrate three ways to try DeepSeek R1: DeepSeek’s hosted chat, a distilled model hosted by Groq, and a local distilled model in LM Studio. Interfaces and model availability shown here reflect the time of the video.

In DeepSeek’s chat interface, I select DeepThink (R1); the interface also offers web search. Hosted services receive the information sent to them, so I encourage viewers to consider privacy before choosing a provider. The video does not establish how any government or third party handles a particular user’s prompts.

On Groq, I select DeepSeek-R1-Distill-Llama-70B. This is a distilled model, not the full R1. In my Tetris coding example the display reaches about 275 tokens per second. I disclose that I am a small investor in Groq.

DIAPOSITIVE 1 / 3Selecting a hosted R1 distillThe Groq interface lists DeepSeek-R1-Distill-Llama-70B.

Running a distill locally

He uses LM Studio to find, download and load a quantized model. He notes hardware needs and an investment in the company.

For local inference I use LM Studio, another company in which I disclose a small investment. After installing it, open the discovery tab and search for DeepSeek. Smaller distilled versions are more practical on a personal machine than the full R1, which requires substantial hardware.

DIAPOSITIVE 2 / 3Quantized model choicesLM Studio displays several quantizations and their download sizes.

The download menu offers different quantizations. Higher-bit variants generally use more memory and may preserve more quality; I commonly choose Q4_K_M. LM Studio indicates whether the model can be offloaded to the GPU, helping you choose a file that fits your hardware.

After downloading, select the model and load it. I demonstrate the 8B Llama distill on a machine with a high-end GPU and ask it to write Snake in Python; the displayed speed is about 77 tokens per second. Your results depend on the model, quantization and hardware.

DIAPOSITIVE 3 / 3Loading a local distillThe model selector shows an 8B distilled variant in LM Studio.

The choice is between a hosted service and local inference, with different performance and data-handling considerations. Local execution can keep prompts on the machine when configured that way, though the application and any connected features still matter. I also mention Ollama as another local option.