Kürzlich aktualisierte Kanäle

Wählen, um nur diesen Kanal zu sehenAlle Kanäle →

Ergebnisse

12 Artikel
Sortieren
Folien0:22EN
MBMatthew Berman· 21. Okt. 2026A Robot Bodyguard Video Shows Dramatic Rescue ScenariosMatthew Berman · Forward FutureMatthew Berman reacts to a short robot demonstration depicting fighting and vehicle-rescue scenarios.RoboticsEvals
Folien0:32EN
MBMatthew Berman· 12. Sept. 2026DeepSeek’s Rubik’s Cube Demo Fails to Scramble and Solve CorrectlyMatthew Berman · Forward FutureMatthew Berman examines a cube simulation generated with DeepSeek V4.1 Flash through the Codex harness.Coding-AgentenEvals
3 Folien11:59EN
MBMatthew Berman· 11. März 2026How Claude Identified a Benchmark and Retrieved Its AnswersMatthew Berman · Forward FutureMatthew Berman examines Anthropic's report on BrowseComp contamination, evaluation-aware behavior, and the difficulty of keeping web-based tests reliable.EvalsAgent EngineeringAnthropic+2
3 Folien10:28EN
MBMatthew Berman· 5. Aug. 2025OpenAI’s gpt-oss Models: Open Weights, Benchmarks and SafetyMatthew Berman · Forward FutureMatthew Berman reviews the 120b and 20b models, local deployment claims, selected evaluations and the scope of OpenAI’s safety testing.Modelle & TrainingAI researchEvals+1
Folien3:13EN
MBMatthew Berman· 5. Aug. 2025Claude Opus 4.1: Anthropic’s August 2025 Benchmark UpdateMatthew Berman · Forward FutureMatthew Berman reviews the model’s reported coding, reasoning and tool-use results, including one benchmark that declined.Modelle & TrainingEvalsCoding-Agenten
3 Folien10:05EN
MBMatthew Berman· 9. Jan. 2025A Chess Experiment Shows How an AI Model Exploited Its Test EnvironmentMatthew Berman · Forward FuturePalisade Research’s small o1-preview test and related alignment studies illustrate why objectives, permissions and evaluation conditions matter.Sicherheit & GovernanceEvalsAI research
3 Folien12:57EN
MBMatthew Berman· 3. Dez. 2024AWS Automated Reasoning Checks: A Sponsored Policy Verification DemoMatthew Berman · Forward FutureMatthew Berman demonstrates a Bedrock preview that checks generated answers against formalized policy rules.AI SafetyEnterprise-KIEvals+1
2 Folien9:43EN
MBMatthew Berman· 26. Aug. 2024Matthew Berman Tests Jamba 1.5 on Coding and Reasoning PromptsMatthew Berman · Forward FutureA 2024 hands-on test of AI21’s hybrid model finds slow output in one hosted session and mixed answers to a small set of prompts.Modelle & TrainingEvals
3 Folien13:40EN
MBMatthew Berman· 29. Juli 2024Testing Llama 3.1 8B: Benchmark Gains and Mixed Practical ResultsMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and other prompts on the small model and finds several failures despite stronger reported benchmarks.Open-weight modelsEvalsModelle & Training+1
2 Folien12:50EN
MBMatthew Berman· 22. Mai 2024Testing Smaug: Hosted 70B Versus a Smaller Local ModelMatthew Berman · Forward FutureMatthew Berman compares two Smaug configurations on Snake, math and reasoning puzzles.Open-weight modelsEvalsModelle & Training
3 Folien14:23EN
MBMatthew Berman· 31. März 2024Testing DBRX: Coding Strengths and Reasoning GapsMatthew Berman · Forward FutureMatthew Berman examines Databricks’ open-weight model and puts it through coding, arithmetic and reasoning prompts.Open-weight modelsModelle & TrainingEvals+1
3 Folien12:20EN
MBMatthew Berman· 10. Juli 2023Testing UltraLM 13B: A Small Model’s Strengths and Reasoning ErrorsMatthew Berman · Forward FutureA 2023 local test finds useful writing and simple coding alongside failures in games, arithmetic and logic.Open-weight modelsEvalsModelle & Training+1