Testing WizardLM 70B: Setup, Coding and Reasoning Results

WizardLM 70B V1.0 is a 70-billion-parameter model based on Llama 2. I am testing it in August 2023 after finding earlier WizardLM versions promising. Its weights are available under the [Llama 2 license](https://huggingface.co/WizardLMTeam/WizardLM-70B-V1.0), which has conditions; “open weight” does not imply unrestricted open-source terms.

The central idea is to use existing language models to create instruction data for fine-tuning. The [WizardLM paper](https://arxiv.org/abs/2304.12244) describes Evol-Instruct: start with seed instructions and iteratively rewrite them into more complex examples. This reduces reliance on writing every training instruction by hand, though generated data still needs evaluation.

TheBloke published different quantized formats. I discuss GGML for CPU-oriented use and GPTQ for a suitable GPU; the best choice depends on software and hardware, not format alone. The GPTQ file shown is roughly 35 GB. My RunPod instance uses one 48 GB NVIDIA L40, 250 GB of system RAM and 32 virtual CPUs, at a little over $1 per hour at the time. Prices and availability change, and the video description identifies my RunPod link as an affiliate link.

In the text-generation web UI, I download the model from its model-card repository, refresh the list, select it, choose the ExLlama loader and load it. I use the instruction interface and the prompt template provided on the model card. This is the setup used in the video, rather than a guarantee that today’s interfaces or deployment steps are identical.

My first prompt asks for a Python script printing 1 through 100. The generated loop uses the correct range and passes my test. I then ask for Snake; the first attempt uses the curses library and stops at the output limit, so I continue the generation. On my machine it fails to import curses, which is an environment issue and does not by itself prove the generated game logic is wrong.

I try again with Pygame and increase the maximum output to 2,048 tokens. The window nearly opens but the program does not complete successfully. I feed the error back, yet the revised output is incomplete. I score the Snake test as a failure; this result describes the observed run, not every possible environment or retry.

A requested 50-word poem is coherent but shorter than 50 words. I give it a pass under my lenient historical rubric, although it fails the exact word-count constraint. The model also writes a verbose but usable resignation email and correctly identifies Bill Clinton as US president in 1996.

To check refusals, I present an unlawful request; the model declines to provide instructions. That is one observed response, not a comprehensive measure of its safeguards or behavior across requests. The Masterworks segment later in the video is sponsored, and investment performance claims in an advertisement should not be treated as guaranteed returns.

For a drying puzzle, five shirts take four hours in the sun; how long for 20? The model says 16 hours and gives inconsistent reasoning. I accept it on a possible sequential-batch interpretation, but the prompt does not specify a five-shirt capacity. If all 20 dry simultaneously under equivalent conditions, the natural answer is about four hours; 16 hours requires an additional serial-batch assumption.

On an ordering question, the model correctly concludes that Sam is not faster than Jane when Jane is faster than Joe and Joe is faster than Sam. It also answers 4 + 4 and applies order of operations to 25 − 4 × 2 + 3, reaching 20. These examples show individual successes rather than a statistical measure of reliability.

The meal-plan response looks balanced to me, though it is generic and not tailored to an individual’s health needs. Asked to count the words in its own response, the model instead says it cannot count them, so I mark that test as a failure. My scoring captures what happened in this run and should not be read as a rigorous medical or language evaluation.

In the three-killers riddle, a person enters a room and kills one of three killers, with nobody leaving. The model discusses several interpretations but overlooks the entrant and changes who committed the killing. I mark it as a failure. Answers also depend on whether the question counts dead people as killers or only living people.

The model summarizes a short passage on nuclear fusion in bullet points and generates a JSON object containing three named people, genders and ages. A JSON validator accepts the syntax in the displayed example. Validation of syntax is useful but does not by itself prove the accuracy of arbitrary generated data.

For a hypothetical choice between many small horses and one large duck, the model lays out factors instead of choosing. I accept it because its reasoning is relevant, though there is no single factual answer. This highlights the subjective nature of my rubric.

The final puzzle places a ball in a cup, turns the cup upside down on a table, and then moves the cup to a microwave. The model assumes the ball stays inside and answers that it is in the microwave; I give credit for its explanation. That assumption is not stated: if an ordinary open cup is lifted from the table without holding the ball, the ball would typically remain on the table. The phrasing leaves room for different handling, so an explicit assumption matters.

Overall, I am impressed by the model’s performance for this 2023 setup, despite its failures and the ambiguity in several tests. The results come from a small, informal set of prompts with some subjective scoring and cannot establish general superiority over other models.