Qwen2 0.5B and 72B: A Hands-On Comparison

Alibaba’s Qwen team released Qwen2 in June 2024. I compare the smallest 0.5B instruction-tuned model with the 72B instruct model and look at the tradeoff between speed and quality. My tests are informal examples, not a controlled evaluation of the full model family.

The release includes base and instruction-tuned models at 0.5B, 1.5B, 7B, 57B-A14B and 72B. Qwen reports up to 128K context for the 7B and 72B instruct models using techniques such as YaRN. A retrieval test at a given context length does not establish flawless performance on every long-context task.

Qwen’s published tables compare its models with Llama 3 and other models across coding, math and language benchmarks. The base 72B model leads the listed base-model comparisons, while instruct-model results vary by task. These are the developer’s reported scores under its chosen conditions, not evidence that Qwen2 wins every possible comparison.

Dell and NVIDIA gave me the computer used for local tests; I say this video is not sponsored by them. It has two RTX A6000 GPUs with 48 GB of VRAM each. I disclose in the video description that I am an investor in LM Studio.

I run Qwen2-0.5B-Instruct in FP16 locally through LM Studio. For the main 72B tests I use Qwen2-72B-Instruct on Hugging Face Spaces; for a separate refusal comparison I load a local Q8-quantized Dolphin 2.9.2 Qwen2 72B fine-tune. They are different variants and environments, so their outputs and speeds cannot be attributed to parameter count alone.

Both instruct models produce a Python script to print 1 through 100. The small model generates the response at about 53 tokens per second on my machine. That is one observed run, not a general speed guarantee.

For Snake in Python, the 0.5B model initially does not produce the requested code. After I specify Pygame it generates code, but variables including head and movement coordinates are undefined. I request fixes and try editing it myself, but still cannot get the game working and mark the attempt a failure.

The 72B instruct model first produces a Tkinter version. I add an import for random, as indicated in the response, and the game runs. A second Pygame version also runs, though I notice its controls do not match expectations. I call both passes, with those qualifications.

I ask a harmful question about breaking into a car. Both official instruct models refuse; the separate Dolphin fine-tune responds. I treat the refusal as a failure in the video because I am testing compliance, but refusing that request is a safety behavior, not evidence of poor reasoning. One prompt cannot establish the overall safety of any model.

In the shirt-drying problem, five shirts take four hours in the sun. The small model gives an inconsistent explanation but reaches the same-time conclusion; I mark it wrong. The large model concludes 20 shirts can also dry in four hours if they can be spread out under equal conditions, while noting space constraints. Its initial statement about direct proportionality conflicts with that later explanation.

On `25 - 4 * 2 + 3`, the small model answers incorrectly. The large model first gives 23 and then corrects itself to 20, which is the result using multiplication before addition. A correct final answer does not erase its initial inconsistency.

The hotel question asks for the total for x nights at $99.95 per night, 8% room tax and a one-time untaxed $5 fee. The relevant expression is `1.08(99.95x) + 5`. I say the 0.5B response is wrong and the 72B response gives the correct expression.

Asked to count the words in its own response, the small model does not give a useful count. The large model claims seven, but I count ten in its displayed response. Both fail this informal test.

In a puzzle about three killers and an entrant who kills one, the small model answers zero. The 72B instruct model gives the conventional answer of three living killers, but adds a scenario that conflicts with the stated initial conditions. The Dolphin fine-tune gives a different muddled explanation about the dead person remaining in the room; I mark that response wrong.

For the marble puzzle, a glass is turned upside down on a table, then picked up and put in a microwave. Both models fail to track the marble correctly in my test; under the usual assumptions, the marble remains on the table when the inverted glass is lifted.

In the belief-tracking example, John puts a ball in a box and leaves; Mark moves it to a basket before leaving. On their return, without learning what happened, John expects the box and Mark the basket. The 72B answer captures that distinction, while the 0.5B response does not. The on-screen prompt clarifies the box-to-basket sequence that the spoken caption misstates.

I ask for ten sentences ending in “apple.” The small model fails the instruction, while I count ten compliant sentences from the larger model. This is a single prompt for each model, so results may vary across attempts.

Finally, I ask how long 50 people would take to dig a hole that takes one person five hours. The small model gives 250 hours. The 72B model offers six minutes only under an idealized perfectly parallel assumption and notes the practical constraints. I conclude that the smallest version may suit simple tasks, while harder work may warrant testing larger models.