Grok 4 and Heavy Tested: Coding, Images, Reasoning and Practical Questions

Matthew Berman’s July 2025 hands-on trials produce interactive prototypes and several failures, with important limits on what single examples can establish.

MBMatthew BermanVortrag · Forward Future
Kanal
Matthew Berman
Veröffentlicht
11. Juli 2025
Video
26:09
Lesezeit
6 Min.
Folien
6
Original
Englisch
Auf YouTube öffnen
Beim Scrollen schrumpft das Video in die rechte untere Ecke und läuft weiter. Untertitel folgen Ihrer Lesesprache.
LesespracheIn 2 Sprachen verfügbar
Diesen Artikel gibt es noch nicht auf Deutsch, daher wird das Original (Englisch) angezeigt.
ZUSAMMENFASSUNG

In 30 Sekunden

Matthew Berman tests Grok 4 and Grok 4 Heavy shortly after release, including fluid simulations, long-context retrieval, image tasks, visual puzzles, safety prompts and advice scenarios. His demonstrations show mixed results; some scientific, legal and medical outputs are not independently validated and the trials are not a controlled model comparison.

  1. 1Interactive codingA smoke simulation responds to obstacles and settings, but its scientific accuracy was not validated.
  2. 2Mixed implementation resultsA basic trade diagram worked visually, while requested animation and the Rubik’s Cube simulation did not.
  3. 3Retrieval in contextA test string was retrieved from a long excerpt; a second prompt found a story password instead.
  4. 4Unverified researchThe host did not enable deep-research mode or validate the superconductivity summary.
  5. 5Visual ARC-style puzzleThe model’s output did not match the host’s interpretation; the prompt differs from official benchmark testing.
  6. 6Medical vignetteThe model names an apparent heart attack in a hypothetical case; actual warning signs require urgent care.

Das Wichtigste

  1. The coding trials produced interactive fluid and Game of Life prototypes, while animation and Rubik’s Cube requests failed or needed revisions.
  2. A few successful retrieval and image examples do not measure general reliability or isolate the effects of tools.
  3. The room-temperature superconductivity summary and economic proof were not independently validated.
  4. One visual ARC-style puzzle failed, but it was not administered as the official benchmark.
  5. The medical vignette is hypothetical; real heart-attack warning signs require immediate emergency care.
  6. Observed memory and safety responses reflect the particular July 2025 tests, not guarantees.
VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Nur das Transkript
01

Coding prototypes

Berman tests fluid simulation, Game of Life, trade visualization, hand tracking and a Rubik’s Cube. Results range from usable prototypes to failed revisions.

Within a day of the July 2025 launch, I test Grok 4 and Grok 4 Heavy across coding, long-context retrieval, images, reasoning and practical questions. I choose Heavy for some harder tasks and the regular model for others. These are informal, mostly single-run demonstrations, not a controlled comparison or a reliable ranking of all models.

First I ask Grok 4 Heavy for a two-dimensional stable-fluids simulation that outputs a smoke plume. After about eight minutes it produces code and 500 images, which I play in sequence. The plume curls and reacts to a wall; this is a visually interesting result, not a validated scientific fluid solver.

I then ask for a browser version with sliders and obstacles. The interface lets me change viscosity, diffusion, time step and buoyancy, draw barriers and reset the simulation. I can see the smoke respond to the obstacles, although the video does not test numerical accuracy.

Interactive smoke simulation
FOLIE 1 / 6Interactive smoke simulationThe generated browser prototype includes a plume and controls for fluid parameters.

A second coding prompt asks for Conway's Game of Life in an HTML canvas. The first version runs but offers few controls. Follow-up requests add update speed, grid dimensions, cell size and other settings; I regard it as a good result while acknowledging that I do not understand every control.

Game of Life controls
FOLIE 2 / 6Game of Life controlsThe generated canvas includes controls for speed, density and grid dimensions.

For a D3 chord diagram of trade flows, Grok produces a simple visualization with four countries and references to data sources. It does not establish that the displayed figures are correct. Subsequent requests for animation do not work as intended, even after another attempt.

Trade-flow chord diagram
FOLIE 3 / 6Trade-flow chord diagramThe output visualizes four countries; the data values were not independently checked.

Next I ask for a desktop drawing app controlled by a tracked fingertip. The first version follows my hand and clears the drawing with an open palm. A color-selection follow-up is awkward and unreliable, but a later gesture-based version lets me select colors, though not exactly as requested.

The model fails to produce a visible Rubik's Cube simulation in my attempts, even after revisions. I compare it unfavorably with my previous Gemini 2.5 Pro test; that is an anecdotal comparison, not a general benchmark result.

02

Long context, images and safety responses

He checks retrieval, image generation, sycophancy and responses to illicit requests. The outcomes are specific to these prompts.

For long-context retrieval, I insert a test password deep in a large book excerpt and ask the model to find it. It returns the inserted string. In a second prompt without my insertion, it identifies a password used in the story; this distinguishes the examples in my view, but it is not a systematic test across positions and texts.

Image generation produces several versions of a cartoon astronaut and a raindrop scene that I consider acceptable. A two-panel comic about a cat and quantum mechanics comes out broken and largely unreadable. I do not establish which underlying image model produced the results.

Generated comic example
FOLIE 4 / 6Generated comic exampleThe requested two-panel comic shows malformed layout and unreadable text.

To test sycophancy, I describe an irresponsible plan involving leaving a job and abandoning children. Grok pushes back on the abandonment and discusses practical difficulties instead of simply validating the plan. One favorable response does not establish consistent behavior, and its legal and cost claims should not be treated as verified advice.

I test how it handles requests for illicit conduct. In this run it supplies detailed information to one vehicle-theft prompt but refuses a request involving illegal drug production. The contrast shows inconsistent boundaries in these examples; I do not reproduce the operational details.

03

Visual understanding and research

The model describes photographs and a visual search task. A superconductivity summary is not independently validated.

For image understanding I upload a retired Google TPU display and ask what is visible. The model describes the object and reads markings that I consider correct. In a separate cluttered-desk photograph, it identifies numerous objects and notes, which I check against the image.

I ask where Waldo is in a beach scene and find a figure near the location described by Grok. The response screen also shows web sources, so this is an observed success in one setting with possible tool use, not a clean measurement of visual perception alone.

I request research on room-temperature superconductivity and citations. I later realize I did not enable the deep-research mode. I say the answer reads well but acknowledge I cannot assess its scientific summaries; the citations and material claims are not independently verified by this video and do not establish an ambient-pressure breakthrough.

04

Reasoning, puzzles and memory

Berman examines an economic thought experiment, an ARC-style grid, conversation memory and an investment summary.

A first-principles prompt asks for a medium of exchange in a space colony and a proof of stability. Grok proposes centrally managed digital credits and mathematical reasoning. I find it interesting but cannot verify the proof or whether similar arguments appeared in training; a formal-looking answer is not itself proof.

I give the model a visual ARC-style grid puzzle and ask it to generate a visualization of the answer. It does not match my interpretation of the pattern. This single visual prompt differs from the official ARC benchmark procedure, so its failure cannot be equated with a benchmark score.

ARC-style visual puzzle
FOLIE 5 / 6ARC-style visual puzzleBerman asks the model to infer a pattern from colored grid examples.

In a memory test, a string is correctly recalled after several exchanges in one conversation. When I ask from another thread, the model says it cannot retrieve the previous conversation. That describes the behavior observed in this setup in July 2025, not a permanent guarantee about future memory features.

I ask for a five-slide executive summary to help assess a Tesla investment. It covers financial performance, market position, risks and opportunities, and I consider the structure useful. Its facts and apparent recency are not independently checked here; the output is not investment advice.

05

Spatial, writing, medical and life tasks

The final tests cover rotations, word counting, creative writing, a medical scenario, Tower of Hanoi and career planning.

A spatial prompt asks for a cube's orientation after rotations around three axes. I check it with a physical cube and believe the answer matches. The prompt leaves axis and rotation conventions ambiguous, so the result should be read as my informal check rather than a uniquely specified mathematical test.

For a word puzzle, Grok gives three as the number of r's in “strawberry.” A self-referential word-count prompt takes several minutes in Heavy; a later regular-model response counts as 43 words in my word counter. These isolated results say little about reliability across similar prompts.

A creative-writing request produces a cyberpunk noir opening with the requested closing line. I like the result while noting that many models can produce creative text; I do not verify the requested word count in the video.

A medical vignette describes acute chest pain, sweating, elevated troponin and ECG changes. Grok identifies an anterior ST-elevation myocardial infarction, which I regard as a reasonable response to the scenario. This is a hypothetical test, not a clinical evaluation; for real heart-attack warning signs, the [American Heart Association](https://www.heart.org/health-topics/heart-attack/warning-signs-of-a-heart-attack) advises calling emergency services immediately rather than waiting for an AI response.

For a four-disk Tower of Hanoi, the model outputs a move table and then generates a visualization. I watch the moves reach the goal and consider the demonstration successful. The visualization checks this particular sequence, not every possible reasoning task.

Tower of Hanoi visualization
FOLIE 6 / 6Tower of Hanoi visualizationThe generated terminal display traces a particular sequence of disk moves.

Finally, I ask for a 12-month transition from accounting to a carpentry business with $40,000 saved. The response suggests keeping the current job initially, budgeting conservatively, building skills and setting milestones. I find the plan useful as a starting point, but its costs and suitability depend on the person's circumstances.

I invite suggestions for additional tests. The overall picture is mixed: several prototypes and answers worked in my informal trials, while animations, a cube simulation, a visual puzzle and image text exposed limitations.

Mehr aus diesem Kanal

12 Folien34:21EN
Matthew Berman· 27. Okt. 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyEnterprise-KI+2
0:26EN
Matthew Berman· 25. Okt. 2026Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.Human-AI CollaborationFuture of WorkAI Strategy
2 Folien0:36EN
Matthew Berman· 25. Okt. 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.EvalsCoding-AgentenOpen-weight models
2 Folien0:21EN
Matthew Berman· 24. Okt. 2026An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.BenchmarksEvalsGenerative media
6 Folien25:50EN
Matthew Berman· 24. Okt. 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 Folien32:20EN
Matthew Berman· 23. Okt. 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1