Results
64 articlesSort
1:46:41EN
Cognitive Revolution "How AI Changes Everything"· Dec 21, 2024PodcastFrontier Models Hit the 10th–40th Percentile on Real AI R&DHost Nathan LabenzNeev Parikh of METR walks Nathan Labenz through REBench, a seven-task test of whether agents can do machine learning research—and what happened when one of them decided not to train.
1:49:41ENCognitive Revolution "How AI Changes Everything"· Nov 21, 2024PodcastA Team of Fifteen Matched Meta’s Post-Training on Meta’s Own BaseHost Nathan LabenzNathan Lambert walks through Tulu 3: the three-stage recipe, the 1 percent versus 14 percent lesson, and why open post-training now lives or dies on data and evals.
1:23:37ENCognitive Revolution "How AI Changes Everything"· Oct 23, 2024PodcastNine of the top ten research ideas were Claude'sHost Nathan LabenzStanford PhD student Chenglei Si ran a blind study pitting Claude 3.5 Sonnet against more than a hundred NLP researchers on novel research ideas.
2:37:47ENCognitive Revolution "How AI Changes Everything"· Oct 19, 2024PodcastThe intern who wrote GELU now wants humanity's last examHost Nathan LabenzDan Hendrycks on activation functions, five-year benchmarks, jailbreaks that do not yield to scale, and stacking defenses because no single layer is trustworthy.
2:25:53ENCognitive Revolution "How AI Changes Everything"· Oct 16, 2024PodcastWhen GPT-4's Base Model Introduced Itself, 75% of the Answers Were About AIHost Nathan LabenzOwain Evans tells Michael Trazzi why situational awareness and out-of-context reasoning are leading indicators of deceptive alignment, and why a pretrained model with almost no self-knowledge still kept talking about being an AI.
1:09:09ENCognitive Revolution "How AI Changes Everything"· Oct 2, 2024Podcast30 Percent on SWE-bench: Cosine Taught Genie How Engineers Actually ThinkHost Nathan LabenzAlistair Pullen tells Latent Space how Cosine reconstructed human problem-solving, fine-tuned on billions of tokens, and took the top score on software engineering’s hardest public test.
1:54:17ENCognitive Revolution "How AI Changes Everything"· Aug 28, 2024PodcastMetaculus Flagged Covid in January 2020. Now It Is Putting AI on the Forecasting DeskHost Nathan LabenzNathan Labenz talks with Metaculus CEO Deger Turan about epistemic security, a year-long bot tournament, and why a useful forecast is harder than an accurate one.
1:56:03ENCognitive Revolution "How AI Changes Everything"· Jul 13, 2024PodcastHumans Near a Perfect Score. The Best AI Still Gets 39%Host Nathan LabenzZapier co-founder Mike Knoop and François Chollet put $1 million on ARC-AGI, a 2019 test of sample-efficient intelligence that language models still cannot crack under a one-cent-per-puzzle budget.
55:13EN
Sequoia Capital· Jul 2, 2024PodcastThe Best AI Scores 39% on This Test. Zapier’s Co-Founder Put $1 Million on Beating ItFrançois Chollet designed ARC to measure skill-acquisition efficiency; five years later humans still crush it and frontier models do not.
26:00EN
No Priors: AI, Machine Learning, Tech, & Startups· Jun 11, 2024Podcast · 2 speakersWhy a Zapier founder thinks AGI research has been stalled for four yearsHost Sarah Guo, Elad GilMike Knoop and François Chollet put more than a million dollars on beating ARC-AGI, the eval they say actually measures general intelligence.
17:12ENTesting the Mystery 'gpt2-chatbot' Model in April 2024Matthew Berman · Forward FutureMatthew Berman tries an anonymous model on coding, arithmetic and reasoning tasks while its identity remains uncertain.
16:28ENTesting Reka Core: Multimodal Benchmarks, Coding and ReasoningMatthew Berman · Forward FutureMatthew Berman reviews Reka’s reported model comparisons and tests Core on code, logic and image understanding.
1:20:44ENCognitive Revolution "How AI Changes Everything"· Mar 30, 2024PodcastSeeing Like a Mamba: Vision, Rotting States, and a Cell Model in 90 DaysHost Nathan LabenzNathan Labenz and Jason Meaux survey the vision papers, long-context experiments, and biology work that followed the original Mamba release.
1:25:35ENCognitive Revolution "How AI Changes Everything"· May 21, 2023PodcastTwo and a Half Years From 33.3% to 86.5%: Google Sent Foundation Models to Medical SchoolHost Nathan LabenzVivek Natarajan walks Nathan Labenz through Med-PaLM and Med-PaLM 2: the data-light techniques, the nine-axis clinician evals, and why a high USMLE-style score is the wrong trophy.
58:24ENCognitive Revolution "How AI Changes Everything"· May 11, 2023PodcastAmong 100 Random Doctors, Zak Kohane Would Be Surprised If One Beat GPT-4Host Nathan LabenzHarvard's biomedical informatics chair on a rare newborn case, why drug-style trials will not certify a general medical model, and why the AI should still work for the patient.
53:37ENNo Priors: AI, Machine Learning, Tech, & Startups· Mar 9, 2023Podcast · 2 speakers30 models, 42 scenarios, 7 metrics: Percy Liang's public checkup for language modelsHost Sarah Guo, Elad GilA PubMed-only model once led the U.S. medical licensing exam — until a system roughly 200 times larger overtook it.
2:34:20EN
Lex Fridman· Aug 30, 2020Podcast · 2 speakersIntelligence Is How Efficiently You Acquire Skills You Were Not Prepared ForGuest François Chollet · Host Lex FridmanFrançois Chollet tells Lex Fridman why skill is not intelligence, why GPT-3 is still pattern matching, and why a grid puzzle called ARC still separates children from machines.
1:49:51ENLex Fridman· Jun 27, 2020Podcast · 2 speakersThe Idea Once Called a Devolution Now Powers Almost Every New ChipGuest David Patterson · Host Lex FridmanTuring Award winner David Patterson talks with Lex Fridman about RISC, RAID, Moore's Law, open hardware, and why chips without a software stack die.
16:03ENLex Fridman· Feb 22, 2020Podcast · 2 speakersQuantum Supremacy Is a Well-Defined Speedup, Not a Useful TaskGuest Scott Aaronson · Host Lex FridmanScott Aaronson explains what Google's 53-qubit experiment actually demonstrated, how it was verified, and why complexity theory can offer evidence but not proof against classical spoofing.
57:57ENSebastian Funk on a Fast, Safe and Maintainable Packet ProcessorSebastian Funk · Jane StreetA single-core market-data case study shows how latency targets, allocation control and generated code can coexist with high-level abstractions.
11:11ENLex FridmanPodcast · 2 speakersYann LeCun on Benchmarks, Interactive Environments, and Why Human Intelligence Is Not GeneralGuest Yann Lecun · Host Lex FridmanIn a 2019 clip, LeCun defends benchmarks as a test of AI claims, explains why interactive environments break the usual train-test protocol, and argues that "general intelligence" is a misnomer.
1:22:57ENCognitive Revolution "How AI Changes Everything"PodcastHuman-Level but Not Human-Like: A GPT-4 Red Teamer on What He Saw Behind the MaskHost Nathan LabenzNathan Labenz recounts two months of full-time testing of an early GPT-4, from the AI doctor of his dreams to a model that would plan assassinations on request.
14:41ENLex FridmanPodcast · 2 speakersTalking for 20 Minutes: How the Alexa Prize Turns Real Customers into a Testbed for Conversational AIGuest Rohit Prasad · Host Lex FridmanRohit Prasad, head scientist of Amazon Alexa, explains the Alexa Prize grand challenge, how it is judged and why true conversation still needs reasoning.
16:41ENLex FridmanPodcastAlphaFold 2 and the Protein-Folding Breakthrough: Why Lex Fridman Calls It a Landmark for AI and BiologyHost Lex FridmanA quick explainer on DeepMind's CASP14 result, how protein folding works, how AlphaFold 1 and 2 likely differ, and what the advance could unlock.Load moreShowing 24 of 64