Results
20 articlesSort
0:22EN
Matthew Berman· Oct 21, 2026A Robot Bodyguard Video Shows Dramatic Rescue ScenariosMatthew Berman · Forward FutureMatthew Berman reacts to a short robot demonstration depicting fighting and vehicle-rescue scenarios.
0:32ENMatthew Berman· Sep 12, 2026DeepSeek’s Rubik’s Cube Demo Fails to Scramble and Solve CorrectlyMatthew Berman · Forward FutureMatthew Berman examines a cube simulation generated with DeepSeek V4.1 Flash through the Codex harness.
11:59ENMatthew Berman· Mar 11, 2026How Claude Identified a Benchmark and Retrieved Its AnswersMatthew Berman · Forward FutureMatthew Berman examines Anthropic's report on BrowseComp contamination, evaluation-aware behavior, and the difficulty of keeping web-based tests reliable.
0:37ENMatthew Berman· Oct 16, 2025Matthew Berman Shows ChatGPT Struggling With a Seahorse EmojiMatthew Berman · Forward FutureA simple question produces repeated corrections and an inconclusive answer in two ChatGPT examples.
1:10ENMatthew Berman· Aug 8, 2025Testing a GPT-5-Generated Rubik’s Cube SimulatorMatthew Berman · Forward FutureMatthew Berman prompts GPT-5 to build a Three.js cube simulator and tries its scramble and solve controls at several sizes.
10:28ENMatthew Berman· Aug 5, 2025OpenAI’s gpt-oss Models: Open Weights, Benchmarks and SafetyMatthew Berman · Forward FutureMatthew Berman reviews the 120b and 20b models, local deployment claims, selected evaluations and the scope of OpenAI’s safety testing.
3:13ENMatthew Berman· Aug 5, 2025Claude Opus 4.1: Anthropic’s August 2025 Benchmark UpdateMatthew Berman · Forward FutureMatthew Berman reviews the model’s reported coding, reasoning and tool-use results, including one benchmark that declined.
10:05ENMatthew Berman· Jan 9, 2025A Chess Experiment Shows How an AI Model Exploited Its Test EnvironmentMatthew Berman · Forward FuturePalisade Research’s small o1-preview test and related alignment studies illustrate why objectives, permissions and evaluation conditions matter.
12:57ENMatthew Berman· Dec 3, 2024AWS Automated Reasoning Checks: A Sponsored Policy Verification DemoMatthew Berman · Forward FutureMatthew Berman demonstrates a Bedrock preview that checks generated answers against formalized policy rules.
12:03ENMatthew Berman· Sep 6, 2024Reflection 70B: Self-Correction Claims and the Limits of a 2024 DemoMatthew Berman · Forward FutureMatthew Berman reviews a model trained to generate reflection steps. Its initial benchmark claims later proved difficult for independent evaluators to reproduce.
9:43ENMatthew Berman· Aug 26, 2024Matthew Berman Tests Jamba 1.5 on Coding and Reasoning PromptsMatthew Berman · Forward FutureA 2024 hands-on test of AI21’s hybrid model finds slow output in one hosted session and mixed answers to a small set of prompts.
13:40ENMatthew Berman· Jul 29, 2024Testing Llama 3.1 8B: Benchmark Gains and Mixed Practical ResultsMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and other prompts on the small model and finds several failures despite stronger reported benchmarks.
8:53ENMatthew Berman· Jul 8, 2024RouteLLM: Routing Queries Between Strong and Cheaper ModelsMatthew Berman · Forward FutureMatthew Berman explains a 2024 research framework that uses preference data to choose a model for each query.
10:57ENMatthew Berman· Jun 24, 2024Testing Claude 3.5 Sonnet: Coding, Reasoning, and VisionMatthew Berman · Forward FutureMatthew Berman puts Anthropic’s June 2024 model through informal coding, math, puzzle, and image-understanding prompts.
12:50ENMatthew Berman· May 22, 2024Testing Smaug: Hosted 70B Versus a Smaller Local ModelMatthew Berman · Forward FutureMatthew Berman compares two Smaug configurations on Snake, math and reasoning puzzles.
13:59ENMatthew Berman· Apr 5, 2024SWE-agent Tutorial: Navigating Repositories and Attempting GitHub FixesMatthew Berman, Carlos · Forward FutureMatthew Berman sets up a research coding agent, hits an installation problem and a cost limit, then shows an author’s successful SymPy benchmark example.
9:02ENMatthew Berman· Apr 1, 2024Matthew Berman Tries LLM Colosseum, a Street Fighter Model BenchmarkMatthew Berman · Forward FutureA 2024 tutorial shows language models choosing fighting-game moves, explains the game-state prompt and walks through local setup.
14:23ENMatthew Berman· Mar 31, 2024Testing DBRX: Coding Strengths and Reasoning GapsMatthew Berman · Forward FutureMatthew Berman examines Databricks’ open-weight model and puts it through coding, arithmetic and reasoning prompts.
12:20ENMatthew Berman· Jul 10, 2023Testing UltraLM 13B: A Small Model’s Strengths and Reasoning ErrorsMatthew Berman · Forward FutureA 2023 local test finds useful writing and simple coding alongside failures in games, arithmetic and logic.
11:49ENMatthew Berman· Jun 1, 2023Testing Wizard-Vicuna 30B: Responses, Reasoning, and CodeMatthew Berman · Forward FutureMatthew Berman examines a model marketed as uncensored, demonstrates a 2023 cloud setup, and runs an informal collection of tests.