Results
269 articlesSort
7:21EN
Matthew Berman· Jun 26, 2023Testing Orca Mini 13B, an Unofficial Model Inspired by OrcaMatthew Berman · Forward FutureMatthew Berman loads a quantized Orca Mini 13B and tries coding, reasoning, arithmetic, and summarization prompts. His 2023 informal test produces several failures and a few successes.
13:50ENMatthew Berman· Jun 23, 2023MPT-30B: Running MosaicML’s Model Locally and Testing Its Chat VariantMatthew Berman · Forward FutureMatthew Berman reviews MosaicML’s June 2023 release, demonstrates a local setup, and puts the hosted chat model through an informal prompt rubric.
10:23ENMatthew Berman· Jun 15, 2023Testing Hosted Falcon 40B and Document Chat in h2oGPTMatthew Berman · Forward FutureMatthew Berman tries H2O.ai’s 2023 hosted Falcon 40B interface on coding, writing and reasoning prompts, then demonstrates document chat.
11:49ENMatthew Berman· Jun 1, 2023Testing Wizard-Vicuna 30B: Responses, Reasoning, and CodeMatthew Berman · Forward FutureMatthew Berman examines a model marketed as uncensored, demonstrates a 2023 cloud setup, and runs an informal collection of tests.
1:33:51EN
Cognitive Revolution "How AI Changes Everything"· May 23, 2023PodcastGPT-4 still writes SQL that will not run. She put a guardrail on the modelHost Nathan LabenzShreya Rajpal built Guardrails AI so developers can encode what “correct” means, validate the pieces, and send failures back to the model instead of hoping the prompt holds.
1:25:35ENCognitive Revolution "How AI Changes Everything"· May 21, 2023PodcastTwo and a Half Years From 33.3% to 86.5%: Google Sent Foundation Models to Medical SchoolHost Nathan LabenzVivek Natarajan walks Nathan Labenz through Med-PaLM and Med-PaLM 2: the data-light techniques, the nine-axis clinician evals, and why a high USMLE-style score is the wrong trophy.
42:18EN
No Priors: AI, Machine Learning, Tech, & Startups· May 18, 2023Podcast · 2 speakersTen Percent of Search Is Health — and the Doctor Is Googling TooHost Sarah Guo, Elad GilGoogle researcher Karan Singhal on Med-PaLM 2, the real bar for medical language models, and why medicine is a harder alignment lab than chatbots.
1:28:43ENCognitive Revolution "How AI Changes Everything"· May 16, 2023PodcastClinical Guidelines Are 11% Evidence. Curai’s Founder Still Wants GPT-4 in the Exam RoomHost Nathan LabenzNeal Khosla of Curai Health argues that large language models already match median U.S. physicians on today’s tests — and that the real work is scaffolding, supervision, and a doctor’s signature.
58:24ENCognitive Revolution "How AI Changes Everything"· May 11, 2023PodcastAmong 100 Random Doctors, Zak Kohane Would Be Surprised If One Beat GPT-4Host Nathan LabenzHarvard's biomedical informatics chair on a rare newborn case, why drug-style trials will not certify a general medical model, and why the AI should still work for the patient.
6:53ENMatthew Berman· May 4, 2023StableVicuna’s 2023 Release: An Early RLHF Chatbot Put to the TestMatthew Berman · Forward FutureMatthew Berman reviews the 13-billion-parameter model and tests poetry, simple code, reasoning and a resignation email.
1:21:06ENCognitive Revolution "How AI Changes Everything"· Apr 27, 2023PodcastEveryone Talks About RLHF. Almost Nobody Is Doing It.Host Nathan LabenzHumanloop CEO Raza Habib on why an LLM API is maybe five percent of a working product, and what actually fills the rest.
6:24ENMatthew Berman· Apr 6, 2023BloombergGPT: Mixing Financial and General Data for a Language ModelMatthew Berman · Forward FutureMatthew Berman examines the 2023 finance-focused research model, its training corpus, evaluation and decision not to release weights.
1:56:16ENCognitive Revolution "How AI Changes Everything"· Apr 6, 2023PodcastThey Used Humans as Stand-In AIs. Now 250,000 People Run Elicit.Host Nathan LabenzOught's founders tell Nathan Labenz why open-ended reasoning has to be decomposed, supervised, and grounded—not scaled until it merely looks persuasive.
1:23:53ENCognitive Revolution "How AI Changes Everything"· Mar 17, 2023PodcastGPT-4, 24 Hours Later: No Safety Locks, a Captcha Hire, and a U.S.–China DeadlockHost Nathan LabenzA day after OpenAI shipped GPT-4, ChinaTalk sat a red teamer, a pessimist, and a policy researcher down to argue over capability, safety, and a race nobody can brake.
11:53ENMatthew Berman· Mar 14, 2023Testing GPT-4 Against GPT-3.5 at LaunchMatthew Berman · Forward FutureMatthew Berman reviews OpenAI's March 2023 GPT-4 announcement and compares the models on scheduling, reasoning, safety and travel prompts.
1:13:06EN
Lex Fridman· Aug 19, 2019Podcast · 2 speakersLockheed Martin's CTO on Autonomy, the Moon, and the Culture of Not Getting It WrongGuest Keoki Jackson · Host Lex FridmanKeoki Jackson tells Lex Fridman why going back to the Moon is the easy part—and why proving a learning system will still do the right thing is not.
1:45:49ENCognitive Revolution "How AI Changes Everything"PodcastTrust at Scale: Elicit's Founders on Reasoning Languages, World Models and Automating the CompanyHost Nathan LabenzAndreas Stuhlmüller and Jungwon Byun explain how Elicit keeps frontier models checkable for high-stakes science, and where they think AI reasoning is heading.
1:25:13EN
Lenny's PodcastPodcastCodex Is Not Just a Coding Tool: OpenAI's Plan for an Agent That Behaves Like a TeammateHost Lenny RachitskyCodex product lead Alexander Embiricos explains the 20x growth, why every useful agent may need to write code, and why human review speed is the real bottleneck.
1:33:50ENLenny's PodcastPodcastEvals Are the New PRDs: Inside Anthropic's Research Product TeamHost Lenny RachitskyDianne Penn, Anthropic's head of product for research and Labs, explains how the company found its identity, how her team turns user feedback into evals, and why human judgment still matters.
1:31:41ENLenny's PodcastPodcast · 2 speakersThe Worst Model You'll Ever Use: Kevin Weil on Building Products at OpenAI's PaceGuest Kevin Weil · Host Lenny RachitskyOpenAI's chief product officer explains how evals, light planning, fine-tuned model ensembles and a small PM team shape the way the company ships.
1:22:36ENLenny's PodcastPodcast · 2 speakersStop Chasing AI News: Chip Huyen on What Actually Makes AI Products BetterGuest Chip Huyen · Host Lenny RachitskyAuthor of AI Engineering walks through pre-training, post-training, RL, evals and RAG, then explains what separates companies that get value from AI from those that stall.
1:07:08ENLenny's PodcastPodcast · 2 speakersIf the Model Is the Product, the Eval Is the PRD: Inside Mercor's RiseGuest Brendan Foody · Host Lenny RachitskyMercor CEO Brendan Foody explains why AI labs will pay almost anything for expert-written evals, how his company grew from $1 million to a $400 million run rate in 16 months, and what that means for jobs and founders.
51:30EN
Unsupervised Learning: With Jacob EffronPodcastLess Email, Better Thinking: Grammarly’s CEO on AI That Augments CommunicationHost Jacob EffronGrammarly CEO Rahul explains how the company is moving from grammar checks to end-to-end communication assistance—and why evals, user data, and measurable outcomes matter.
14:41ENLex FridmanPodcast · 2 speakersTalking for 20 Minutes: How the Alexa Prize Turns Real Customers into a Testbed for Conversational AIGuest Rohit Prasad · Host Lex FridmanRohit Prasad, head scientist of Amazon Alexa, explains the Alexa Prize grand challenge, how it is judged and why true conversation still needs reasoning.Load moreShowing 24 of 269