Results
13 articlesSort
17:15ENThe Hugging Face Incident: AI Evaluation, Sandbox Escape and Reward HackingMatthew Berman · Forward FutureMatthew Berman explains the reported OpenAI evaluation incident and the security and incentive failures it exposed.
11:59ENHow Claude Identified a Benchmark and Retrieved Its AnswersMatthew Berman · Forward FutureMatthew Berman examines Anthropic's report on BrowseComp contamination, evaluation-aware behavior, and the difficulty of keeping web-based tests reliable.
32:22ENQwen Image Edit Plus Compared With Nano Banana, Seedream and GPT Image 1Matthew Berman · Forward FutureMatthew Berman compares four image-editing models across compositing, precise edits, style transfer, object removal and weather effects.
10:28ENOpenAI’s gpt-oss Models: Open Weights, Benchmarks and SafetyMatthew Berman · Forward FutureMatthew Berman reviews the 120b and 20b models, local deployment claims, selected evaluations and the scope of OpenAI’s safety testing.
3:13ENClaude Opus 4.1: Anthropic’s August 2025 Benchmark UpdateMatthew Berman · Forward FutureMatthew Berman reviews the model’s reported coding, reasoning and tool-use results, including one benchmark that declined.
10:05ENA Chess Experiment Shows How an AI Model Exploited Its Test EnvironmentMatthew Berman · Forward FuturePalisade Research’s small o1-preview test and related alignment studies illustrate why objectives, permissions and evaluation conditions matter.
19:06ENBuilding Effective AI Agents: Patterns, Trade-offs and EvaluationMatthew Berman · Forward FutureMatthew Berman discusses Anthropic’s guide to workflows and agents, adding examples from his own projects.
9:43ENMatthew Berman Tests Jamba 1.5 on Coding and Reasoning PromptsMatthew Berman · Forward FutureA 2024 hands-on test of AI21’s hybrid model finds slow output in one hosted session and mixed answers to a small set of prompts.
12:50ENTesting Smaug: Hosted 70B Versus a Smaller Local ModelMatthew Berman · Forward FutureMatthew Berman compares two Smaug configurations on Snake, math and reasoning puzzles.
16:28ENTesting Reka Core: Multimodal Benchmarks, Coding and ReasoningMatthew Berman · Forward FutureMatthew Berman reviews Reka’s reported model comparisons and tests Core on code, logic and image understanding.
15:01ENTesting Llama 3 at Launch: Code, Reasoning and Image GenerationMatthew Berman · Forward FutureMatthew Berman tries Meta AI’s new interface on a snake game, logic puzzles and rapid image generation.
14:23ENTesting DBRX: Coding Strengths and Reasoning GapsMatthew Berman · Forward FutureMatthew Berman examines Databricks’ open-weight model and puts it through coding, arithmetic and reasoning prompts.
23:00ENTesting Google Gemma 7B at Launch: Setup MattersMatthew Berman · Forward FutureMatthew Berman compares local and hosted tests of Google’s early open-weight model.