絞り込み結果
251 件並び順
19:37EN
Matthew Berman· 2023年12月13日How Rephrased Test Data Inflated a 13B Model’s Benchmark ScoresMatthew Berman · Forward FutureMatthew Berman examines a 2023 experiment showing how reworded benchmark questions can evade simple contamination checks and inflate model scores.
1:22:59EN
Cognitive Revolution "How AI Changes Everything"· 2023年12月2日ポッドキャストIf Your Agent Is Right 80% of the Time, Ten Steps Leave You With 10%ホスト Nathan LabenzImbue CTO Josh Albrecht tells Nathan Labenz how a 27-person, pre-product research company is putting $200 million and 10,000 H100s toward agents that can actually finish a long job.
18:29ENMatthew Berman· 2023年12月1日OpenAI's 2023 Leadership Crisis, Q* Speculation and Synthetic DataMatthew Berman, Andrew Ross Sorkin, Elon Musk · Forward FutureMatthew Berman connects Sam Altman's return, an Elon Musk interview and Nathan Lambert's discussion of synthetic training data, separating reported events from his own theories.
32:48EN
No Priors: AI, Machine Learning, Tech, & Startups· 2023年11月16日ポッドキャスト · 2 人Fourteen People, $200 Million of Compute, and the Reliability Bottleneck for Agentsホスト Sarah Guo, Elad GilImbue co-founders Kanjun Qiu and Josh Albrecht on why agents fail, why language models cannot learn general algorithms, and why they are spending a giant round on GPUs rather than headcount.
1:23:10ENCognitive Revolution "How AI Changes Everything"· 2023年9月19日ポッドキャストLuxury Purses Under $500 Beat a Bank's Fraud Modelホスト Nathan LabenzArthur.ai CEO Adam Wenchel on boundary attacks, LLM firewalls, and why a model that eats raw text cannot be secured from the inside.
10:20ENMatthew Berman· 2023年8月30日Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
55:08ENCognitive Revolution "How AI Changes Everything"· 2023年8月29日ポッドキャストOne Model Inside 38 Products: Paige Bailey on How Google Trains and Ships PaLM 2ホスト Nathan LabenzAt Google I/O in May 2023, PaLM 2 arrived already running in more than two dozen products — and its lead product manager walks through what that actually took.
39:45ENNo Priors: AI, Machine Learning, Tech, & Startups· 2023年7月27日ポッドキャスト · 2 人Palantir's 13th Employee: The Model Is a Stochastic Genie. Ontology Is the Stake That Tethers Itホスト Sarah Guo, Elad GilCTO Shyam Sankar on 550 microservices, air-gapped submarines, and why Palantir's ontology is what large language models were waiting for.
4:59ENMatthew Berman· 2023年7月25日Testing Airoboros L2 70B on Coding, Writing and ReasoningMatthew Berman · Forward FutureMatthew Berman runs a small set of prompts against a Llama 2 fine-tune, finding useful structured output alongside failures on code and logic.
9:46ENMatthew Berman· 2023年7月17日Orca Mini V2 vs. OpenOrca: A 2023 Informal Model ComparisonMatthew Berman · Forward FutureMatthew Berman tests two Orca-inspired models on coding, writing and reasoning, with results shaped by prompt setup and subjective scoring.
8:04ENMatthew Berman· 2023年7月11日WizardLM 13B v1.1 Tested on Coding, Writing and LogicMatthew Berman · Forward FutureMatthew Berman tries an instruction-tuned model on a small set of prompts, with mixed results and several subjective scoring choices.
12:20ENMatthew Berman· 2023年7月10日Testing UltraLM 13B: A Small Model’s Strengths and Reasoning ErrorsMatthew Berman · Forward FutureA 2023 local test finds useful writing and simple coding alongside failures in games, arithmetic and logic.
54:00EN
20VC with Harry Stebbings· 2023年6月30日ポッドキャスト · 2 人Douwe Kiela on Data, Retrieval and Enterprise AIゲスト Douwe Kiela · ホスト Harry StebbingsThe Contextual AI CEO discusses language model trade-offs, evaluation, open models and the hurdles to enterprise use.
18:08ENMatthew Berman· 2023年6月29日Six Language Models Compared on an Informal 2023 Prompt SetMatthew Berman · Forward FutureMatthew Berman tests coding, writing and reasoning with H2O GPT, finding a tie under a small and subjective rubric.
7:21ENMatthew Berman· 2023年6月26日Testing Orca Mini 13B, an Unofficial Model Inspired by OrcaMatthew Berman · Forward FutureMatthew Berman loads a quantized Orca Mini 13B and tries coding, reasoning, arithmetic, and summarization prompts. His 2023 informal test produces several failures and a few successes.
10:23ENMatthew Berman· 2023年6月15日Testing Hosted Falcon 40B and Document Chat in h2oGPTMatthew Berman · Forward FutureMatthew Berman tries H2O.ai’s 2023 hosted Falcon 40B interface on coding, writing and reasoning prompts, then demonstrates document chat.
11:49ENMatthew Berman· 2023年6月1日Testing Wizard-Vicuna 30B: Responses, Reasoning, and CodeMatthew Berman · Forward FutureMatthew Berman examines a model marketed as uncensored, demonstrates a 2023 cloud setup, and runs an informal collection of tests.
1:33:51ENCognitive Revolution "How AI Changes Everything"· 2023年5月23日ポッドキャストGPT-4 still writes SQL that will not run. She put a guardrail on the modelホスト Nathan LabenzShreya Rajpal built Guardrails AI so developers can encode what “correct” means, validate the pieces, and send failures back to the model instead of hoping the prompt holds.
1:25:35ENCognitive Revolution "How AI Changes Everything"· 2023年5月21日ポッドキャストTwo and a Half Years From 33.3% to 86.5%: Google Sent Foundation Models to Medical Schoolホスト Nathan LabenzVivek Natarajan walks Nathan Labenz through Med-PaLM and Med-PaLM 2: the data-light techniques, the nine-axis clinician evals, and why a high USMLE-style score is the wrong trophy.
42:18ENNo Priors: AI, Machine Learning, Tech, & Startups· 2023年5月18日ポッドキャスト · 2 人Ten Percent of Search Is Health — and the Doctor Is Googling Tooホスト Sarah Guo, Elad GilGoogle researcher Karan Singhal on Med-PaLM 2, the real bar for medical language models, and why medicine is a harder alignment lab than chatbots.
1:28:43ENCognitive Revolution "How AI Changes Everything"· 2023年5月16日ポッドキャストClinical Guidelines Are 11% Evidence. Curai’s Founder Still Wants GPT-4 in the Exam Roomホスト Nathan LabenzNeal Khosla of Curai Health argues that large language models already match median U.S. physicians on today’s tests — and that the real work is scaffolding, supervision, and a doctor’s signature.
58:24ENCognitive Revolution "How AI Changes Everything"· 2023年5月11日ポッドキャストAmong 100 Random Doctors, Zak Kohane Would Be Surprised If One Beat GPT-4ホスト Nathan LabenzHarvard's biomedical informatics chair on a rare newborn case, why drug-style trials will not certify a general medical model, and why the AI should still work for the patient.
1:21:06ENCognitive Revolution "How AI Changes Everything"· 2023年4月27日ポッドキャストEveryone Talks About RLHF. Almost Nobody Is Doing It.ホスト Nathan LabenzHumanloop CEO Raza Habib on why an LLM API is maybe five percent of a working product, and what actually fills the rest.
1:56:16ENCognitive Revolution "How AI Changes Everything"· 2023年4月6日ポッドキャストThey Used Humans as Stand-In AIs. Now 250,000 People Run Elicit.ホスト Nathan LabenzOught's founders tell Nathan Labenz why open-ended reasoning has to be decomposed, supervised, and grounded—not scaled until it merely looks persuasive.もっと見る251 件中 24 件を表示