Results
9 articlesSort
17:15ENThe Hugging Face Incident: AI Evaluation, Sandbox Escape and Reward HackingMatthew Berman · Forward FutureMatthew Berman explains the reported OpenAI evaluation incident and the security and incentive failures it exposed.
17:03ENDeepSWE: Testing Coding Agents on Longer Software TasksMatthew Berman · Forward FutureMatthew Berman reviews Datacurve’s benchmark, its verifier audit and the tradeoffs of comparing models under a shared harness.
27:15ENMeta-Harness: Searching for Better AI Agent InfrastructureMatthew Berman · Forward FutureMatthew Berman reviews a paper on using a coding agent to iteratively improve model harnesses and discusses its experimental limits.
26:05ENInside Claude: What Interpretability Research Reveals—and What It Cannot Yet ExplainMatthew Berman · Forward FutureMatthew Berman examines Anthropic’s case studies on multilingual features, planning, arithmetic, hallucinations and safety.
19:06ENBuilding Effective AI Agents: Patterns, Trade-offs and EvaluationMatthew Berman · Forward FutureMatthew Berman discusses Anthropic’s guide to workflows and agents, adding examples from his own projects.
16:28ENTesting Reka Core: Multimodal Benchmarks, Coding and ReasoningMatthew Berman · Forward FutureMatthew Berman reviews Reka’s reported model comparisons and tests Core on code, logic and image understanding.
15:01ENTesting Llama 3 at Launch: Code, Reasoning and Image GenerationMatthew Berman · Forward FutureMatthew Berman tries Meta AI’s new interface on a snake game, logic puzzles and rapid image generation.
21:16ENArtPrompt: What a 2024 Study Found About Visual Text and Model SafetyMatthew Berman · Forward FutureMatthew Berman reviews an ASCII-art recognition benchmark and reports mixed results from his own informal tests.
23:00ENTesting Google Gemma 7B at Launch: Setup MattersMatthew Berman · Forward FutureMatthew Berman compares local and hosted tests of Google’s early open-weight model.