絞り込み結果
221 件並び順
21:02EN
Matthew Berman· 2024年12月24日Reactions to OpenAI’s 2024 o3 Preview: Benchmarks, Cost and LimitsMatthew Berman · Forward FutureMatthew Berman surveys the responses to the o3 preview, from mathematical benchmarks to debates about AGI, compute and reliability.
1:46:41EN
Cognitive Revolution "How AI Changes Everything"· 2024年12月21日ポッドキャストFrontier Models Hit the 10th–40th Percentile on Real AI R&Dホスト Nathan LabenzNeev Parikh of METR walks Nathan Labenz through REBench, a seven-task test of whether agents can do machine learning research—and what happened when one of them decided not to train.
26:23ENMatthew Berman· 2024年12月20日OpenAI’s o3 Announcement: Benchmark Gains and the Question of AGIMatthew Berman, Sam Altman, Mark Chen, Greg Kamradt, Hongyu Ren · Forward FutureMatthew Berman reviews the December 2024 o3 and o3-mini announcement, including coding, math and ARC-AGI results. He calls the results evidence of AGI, a conclusion the benchmarks alone cannot establish.
9:00ENMatthew Berman· 2024年12月20日Best-of-N Jailbreaking: What Repeated Prompt Variations Revealed in 2024Matthew Berman · Forward FutureMatthew Berman reviews research on how repeated input variations can expose safety failures in text, vision and audio models.
52:21ENMatthew Berman· 2024年12月7日The SWE-bench Team on Coding Agents, Evaluation and the Future of ProgrammingMatthew Berman, Kilian Lieret, Ofir Press, Carlos E. Jimenez · Forward FutureKilian Lieret, Ofir Press and Carlos Jimenez explain how real GitHub issues became a benchmark and how interface design shaped SWE-agent.
1:22:09ENMatthew Berman· 2024年12月5日Testing ChatGPT o1 Pro Live: Puzzles, Code and the Cost of More ReasoningMatthew Berman · Forward FutureMatthew Berman tries OpenAI’s newly announced $200 monthly plan on audience prompts, encountering slow responses, simple successes and a wrong Sudoku conclusion.
1:49:41ENCognitive Revolution "How AI Changes Everything"· 2024年11月21日ポッドキャストA Team of Fifteen Matched Meta’s Post-Training on Meta’s Own Baseホスト Nathan LabenzNathan Lambert walks through Tulu 3: the three-stage recipe, the 1 percent versus 14 percent lesson, and why open post-training now lives or dies on data and evals.
14:50ENMatthew Berman· 2024年11月19日AI News in November 2024: Mistral, Gemini, Nuclear Safety and Medical AIMatthew Berman · Forward FutureMatthew Berman surveys Le Chat updates, model rankings, nuclear safeguards and a diagnostic reasoning study.
14:05ENMatthew Berman· 2024年11月15日Test-Time Training Improves ARC Puzzle Results in an MIT StudyMatthew Berman · Forward FutureMatthew Berman explains a 2024 paper that adapts a language model to each puzzle at inference time and combines its predictions with program synthesis.
11:56ENMatthew Berman· 2024年11月3日IBM Granite 3.0 and InstructLab: Small Models for Enterprise AIMatthew Berman · Forward FutureMatthew Berman reviews IBM's 2024 model family and its approach to adding domain knowledge. IBM partnered on the video and invited him to its event.
10:39ENMatthew Berman· 2024年10月29日Zamba2-7B: A Hybrid Mamba–Transformer Model Tested on Coding and ReasoningMatthew Berman · Forward FutureMatthew Berman compares Zyphra’s efficiency claims with his own informal tests, including Python games, spatial puzzles and simple word tasks.
12:29ENMatthew Berman· 2024年10月25日Testing the Updated Claude 3.5 Sonnet: Coding, Reasoning and VisionMatthew Berman · Forward FutureMatthew Berman tries the October 2024 model on Python games, logic puzzles and images, with mixed results and some ambiguous grading.
18:28ENMatthew Berman· 2024年10月23日Claude 3.5 Sonnet’s 2024 Computer-Use Beta: Demos and LimitsMatthew Berman · Forward FutureMatthew Berman reviews Anthropic’s updated models and shows how a beta agent navigated software with screenshots and tool actions.
1:23:37ENCognitive Revolution "How AI Changes Everything"· 2024年10月23日ポッドキャストNine of the top ten research ideas were Claude'sホスト Nathan LabenzStanford PhD student Chenglei Si ran a blind study pitting Claude 3.5 Sonnet against more than a hundred NLP researchers on novel research ideas.
2:37:47ENCognitive Revolution "How AI Changes Everything"· 2024年10月19日ポッドキャストThe intern who wrote GELU now wants humanity's last examホスト Nathan LabenzDan Hendrycks on activation functions, five-year benchmarks, jailbreaks that do not yield to scale, and stacking defenses because no single layer is trustworthy.
2:25:53ENCognitive Revolution "How AI Changes Everything"· 2024年10月16日ポッドキャストWhen GPT-4's Base Model Introduced Itself, 75% of the Answers Were About AIホスト Nathan LabenzOwain Evans tells Michael Trazzi why situational awareness and out-of-context reasoning are leading indicators of deceptive alignment, and why a pretrained model with almost no self-knowledge still kept talking about being an AI.
9:38ENMatthew Berman· 2024年10月15日Liquid Foundation Models: Company Benchmarks and a Hands-On TestMatthew Berman · Forward FutureMatthew Berman reviews Liquid AI’s 2024 model and memory claims, then tries the 40B preview on coding, logic, and instruction-following prompts.
26:43ENMatthew Berman· 2024年10月10日Testing OpenAI o1 on Planning, Rules and Spatial ReasoningMatthew Berman · Forward FutureMatthew Berman reviews a 2024 pilot study comparing o1-preview, o1-mini and GPT-4 on six planning domains.
1:09:09ENCognitive Revolution "How AI Changes Everything"· 2024年10月2日ポッドキャスト30 Percent on SWE-bench: Cosine Taught Genie How Engineers Actually Thinkホスト Nathan LabenzAlistair Pullen tells Latent Space how Cosine reconstructed human problem-solving, fine-tuned on billions of tokens, and took the top score on software engineering’s hardest public test.
11:14ENMatthew Berman· 2024年9月18日Pixtral 12B: Testing a 2024 Vision-Language ModelMatthew Berman · Forward FutureMatthew Berman runs informal image, text and interface-generation tests of Mistral’s Pixtral on a sponsor-provided cloud GPU.
18:09ENMatthew Berman· 2024年9月17日AI News in September 2024: Copilot, NotebookLM, o1 and World ModelsMatthew Berman · Forward FutureMatthew Berman reviews a week of AI announcements, from workplace assistants and audio overviews to benchmark results, generative video and spatial AI.
10:57ENMatthew Berman· 2024年9月13日Testing OpenAI o1-preview: Tetris, Physical Reasoning and a North Pole ErrorMatthew Berman · Forward FutureIn September 2024, Matthew Berman runs the new reasoning-model preview through a personal rubric and examines where its answers succeed or fail.
21:21ENMatthew Berman· 2024年9月12日OpenAI o1’s 2024 Launch: Reasoning Benchmarks and Early TestsMatthew Berman · Forward FutureMatthew Berman reviews o1-preview and o1-mini, OpenAI’s reported evaluations, and his own early coding experiments.
19:38ENMatthew Berman· 2024年9月9日Reflection 70B: Conflicting Results and Unanswered QuestionsMatthew Berman · Forward FutureMatthew Berman reviews his early coverage, independent evaluations and competing claims about the September 2024 release.もっと見る221 件中 24 件を表示