필터 결과
219개정렬
21:02EN
Matthew Berman· 2024년 12월 24일Reactions to OpenAI’s 2024 o3 Preview: Benchmarks, Cost and LimitsMatthew Berman · Forward FutureMatthew Berman surveys the responses to the o3 preview, from mathematical benchmarks to debates about AGI, compute and reliability.
1:46:41EN
Cognitive Revolution "How AI Changes Everything"· 2024년 12월 21일팟캐스트Frontier Models Hit the 10th–40th Percentile on Real AI R&D진행자 Nathan LabenzNeev Parikh of METR walks Nathan Labenz through REBench, a seven-task test of whether agents can do machine learning research—and what happened when one of them decided not to train.
26:23ENMatthew Berman· 2024년 12월 20일OpenAI’s o3 Announcement: Benchmark Gains and the Question of AGIMatthew Berman, Sam Altman, Mark Chen, Greg Kamradt, Hongyu Ren · Forward FutureMatthew Berman reviews the December 2024 o3 and o3-mini announcement, including coding, math and ARC-AGI results. He calls the results evidence of AGI, a conclusion the benchmarks alone cannot establish.
9:00ENMatthew Berman· 2024년 12월 20일Best-of-N Jailbreaking: What Repeated Prompt Variations Revealed in 2024Matthew Berman · Forward FutureMatthew Berman reviews research on how repeated input variations can expose safety failures in text, vision and audio models.
52:21ENMatthew Berman· 2024년 12월 7일The SWE-bench Team on Coding Agents, Evaluation and the Future of ProgrammingMatthew Berman, Kilian Lieret, Ofir Press, Carlos E. Jimenez · Forward FutureKilian Lieret, Ofir Press and Carlos Jimenez explain how real GitHub issues became a benchmark and how interface design shaped SWE-agent.
1:22:09ENMatthew Berman· 2024년 12월 5일Testing ChatGPT o1 Pro Live: Puzzles, Code and the Cost of More ReasoningMatthew Berman · Forward FutureMatthew Berman tries OpenAI’s newly announced $200 monthly plan on audience prompts, encountering slow responses, simple successes and a wrong Sudoku conclusion.
1:49:41ENCognitive Revolution "How AI Changes Everything"· 2024년 11월 21일팟캐스트A Team of Fifteen Matched Meta’s Post-Training on Meta’s Own Base진행자 Nathan LabenzNathan Lambert walks through Tulu 3: the three-stage recipe, the 1 percent versus 14 percent lesson, and why open post-training now lives or dies on data and evals.
14:50ENMatthew Berman· 2024년 11월 19일AI News in November 2024: Mistral, Gemini, Nuclear Safety and Medical AIMatthew Berman · Forward FutureMatthew Berman surveys Le Chat updates, model rankings, nuclear safeguards and a diagnostic reasoning study.
14:05ENMatthew Berman· 2024년 11월 15일Test-Time Training Improves ARC Puzzle Results in an MIT StudyMatthew Berman · Forward FutureMatthew Berman explains a 2024 paper that adapts a language model to each puzzle at inference time and combines its predictions with program synthesis.
11:56ENMatthew Berman· 2024년 11월 3일IBM Granite 3.0 and InstructLab: Small Models for Enterprise AIMatthew Berman · Forward FutureMatthew Berman reviews IBM's 2024 model family and its approach to adding domain knowledge. IBM partnered on the video and invited him to its event.
10:39ENMatthew Berman· 2024년 10월 29일Zamba2-7B: A Hybrid Mamba–Transformer Model Tested on Coding and ReasoningMatthew Berman · Forward FutureMatthew Berman compares Zyphra’s efficiency claims with his own informal tests, including Python games, spatial puzzles and simple word tasks.
12:29ENMatthew Berman· 2024년 10월 25일Testing the Updated Claude 3.5 Sonnet: Coding, Reasoning and VisionMatthew Berman · Forward FutureMatthew Berman tries the October 2024 model on Python games, logic puzzles and images, with mixed results and some ambiguous grading.
18:28ENMatthew Berman· 2024년 10월 23일Claude 3.5 Sonnet’s 2024 Computer-Use Beta: Demos and LimitsMatthew Berman · Forward FutureMatthew Berman reviews Anthropic’s updated models and shows how a beta agent navigated software with screenshots and tool actions.
1:23:37ENCognitive Revolution "How AI Changes Everything"· 2024년 10월 23일팟캐스트Nine of the top ten research ideas were Claude's진행자 Nathan LabenzStanford PhD student Chenglei Si ran a blind study pitting Claude 3.5 Sonnet against more than a hundred NLP researchers on novel research ideas.
2:37:47ENCognitive Revolution "How AI Changes Everything"· 2024년 10월 19일팟캐스트The intern who wrote GELU now wants humanity's last exam진행자 Nathan LabenzDan Hendrycks on activation functions, five-year benchmarks, jailbreaks that do not yield to scale, and stacking defenses because no single layer is trustworthy.
2:25:53ENCognitive Revolution "How AI Changes Everything"· 2024년 10월 16일팟캐스트When GPT-4's Base Model Introduced Itself, 75% of the Answers Were About AI진행자 Nathan LabenzOwain Evans tells Michael Trazzi why situational awareness and out-of-context reasoning are leading indicators of deceptive alignment, and why a pretrained model with almost no self-knowledge still kept talking about being an AI.
9:38ENMatthew Berman· 2024년 10월 15일Liquid Foundation Models: Company Benchmarks and a Hands-On TestMatthew Berman · Forward FutureMatthew Berman reviews Liquid AI’s 2024 model and memory claims, then tries the 40B preview on coding, logic, and instruction-following prompts.
26:43ENMatthew Berman· 2024년 10월 10일Testing OpenAI o1 on Planning, Rules and Spatial ReasoningMatthew Berman · Forward FutureMatthew Berman reviews a 2024 pilot study comparing o1-preview, o1-mini and GPT-4 on six planning domains.
1:09:09ENCognitive Revolution "How AI Changes Everything"· 2024년 10월 2일팟캐스트30 Percent on SWE-bench: Cosine Taught Genie How Engineers Actually Think진행자 Nathan LabenzAlistair Pullen tells Latent Space how Cosine reconstructed human problem-solving, fine-tuned on billions of tokens, and took the top score on software engineering’s hardest public test.
11:14ENMatthew Berman· 2024년 9월 18일Pixtral 12B: Testing a 2024 Vision-Language ModelMatthew Berman · Forward FutureMatthew Berman runs informal image, text and interface-generation tests of Mistral’s Pixtral on a sponsor-provided cloud GPU.
18:09ENMatthew Berman· 2024년 9월 17일AI News in September 2024: Copilot, NotebookLM, o1 and World ModelsMatthew Berman · Forward FutureMatthew Berman reviews a week of AI announcements, from workplace assistants and audio overviews to benchmark results, generative video and spatial AI.
10:57ENMatthew Berman· 2024년 9월 13일Testing OpenAI o1-preview: Tetris, Physical Reasoning and a North Pole ErrorMatthew Berman · Forward FutureIn September 2024, Matthew Berman runs the new reasoning-model preview through a personal rubric and examines where its answers succeed or fail.
21:21ENMatthew Berman· 2024년 9월 12일OpenAI o1’s 2024 Launch: Reasoning Benchmarks and Early TestsMatthew Berman · Forward FutureMatthew Berman reviews o1-preview and o1-mini, OpenAI’s reported evaluations, and his own early coding experiments.
19:38ENMatthew Berman· 2024년 9월 9일Reflection 70B: Conflicting Results and Unanswered QuestionsMatthew Berman · Forward FutureMatthew Berman reviews his early coverage, independent evaluations and competing claims about the September 2024 release.더 보기219개 중 24개 표시