필터 결과
269개정렬
19:37EN
Matthew Berman· 2023년 12월 13일How Rephrased Test Data Inflated a 13B Model’s Benchmark ScoresMatthew Berman · Forward FutureMatthew Berman examines a 2023 experiment showing how reworded benchmark questions can evade simple contamination checks and inflate model scores.
1:22:59EN
Cognitive Revolution "How AI Changes Everything"· 2023년 12월 2일팟캐스트If Your Agent Is Right 80% of the Time, Ten Steps Leave You With 10%진행자 Nathan LabenzImbue CTO Josh Albrecht tells Nathan Labenz how a 27-person, pre-product research company is putting $200 million and 10,000 H100s toward agents that can actually finish a long job.
18:29ENMatthew Berman· 2023년 12월 1일OpenAI's 2023 Leadership Crisis, Q* Speculation and Synthetic DataMatthew Berman, Andrew Ross Sorkin, Elon Musk · Forward FutureMatthew Berman connects Sam Altman's return, an Elon Musk interview and Nathan Lambert's discussion of synthetic training data, separating reported events from his own theories.
32:48EN
No Priors: AI, Machine Learning, Tech, & Startups· 2023년 11월 16일팟캐스트 · 2명Fourteen People, $200 Million of Compute, and the Reliability Bottleneck for Agents진행자 Sarah Guo, Elad GilImbue co-founders Kanjun Qiu and Josh Albrecht on why agents fail, why language models cannot learn general algorithms, and why they are spending a giant round on GPUs rather than headcount.
1:23:10ENCognitive Revolution "How AI Changes Everything"· 2023년 9월 19일팟캐스트Luxury Purses Under $500 Beat a Bank's Fraud Model진행자 Nathan LabenzArthur.ai CEO Adam Wenchel on boundary attacks, LLM firewalls, and why a model that eats raw text cannot be secured from the inside.
10:20ENMatthew Berman· 2023년 8월 30일Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
55:08ENCognitive Revolution "How AI Changes Everything"· 2023년 8월 29일팟캐스트One Model Inside 38 Products: Paige Bailey on How Google Trains and Ships PaLM 2진행자 Nathan LabenzAt Google I/O in May 2023, PaLM 2 arrived already running in more than two dozen products — and its lead product manager walks through what that actually took.
39:45ENNo Priors: AI, Machine Learning, Tech, & Startups· 2023년 7월 27일팟캐스트 · 2명Palantir's 13th Employee: The Model Is a Stochastic Genie. Ontology Is the Stake That Tethers It진행자 Sarah Guo, Elad GilCTO Shyam Sankar on 550 microservices, air-gapped submarines, and why Palantir's ontology is what large language models were waiting for.
5:34ENMatthew Berman· 2023년 7월 26일FreeWilly2: A 2023 Prompt Test with Uncertain FormattingMatthew Berman · Forward FutureMatthew Berman retries an instruction-tuned Llama 2 model after a poor first run, finding mixed results and a likely template problem.
4:59ENMatthew Berman· 2023년 7월 25일Testing Airoboros L2 70B on Coding, Writing and ReasoningMatthew Berman · Forward FutureMatthew Berman runs a small set of prompts against a Llama 2 fine-tune, finding useful structured output alongside failures on code and logic.
11:07ENMatthew Berman· 2023년 7월 24일Installing Llama 2 13B Chat and Testing Its ResponsesMatthew Berman · Forward FutureMatthew Berman demonstrates a 2023 local setup and tests a 13-billion-parameter chat model on coding, writing, arithmetic and reasoning. His small set of prompts illustrates both useful outputs and mistakes.
9:46ENMatthew Berman· 2023년 7월 17일Orca Mini V2 vs. OpenOrca: A 2023 Informal Model ComparisonMatthew Berman · Forward FutureMatthew Berman tests two Orca-inspired models on coding, writing and reasoning, with results shaped by prompt setup and subjective scoring.
8:04ENMatthew Berman· 2023년 7월 11일WizardLM 13B v1.1 Tested on Coding, Writing and LogicMatthew Berman · Forward FutureMatthew Berman tries an instruction-tuned model on a small set of prompts, with mixed results and several subjective scoring choices.
12:20ENMatthew Berman· 2023년 7월 10일Testing UltraLM 13B: A Small Model’s Strengths and Reasoning ErrorsMatthew Berman · Forward FutureA 2023 local test finds useful writing and simple coding alongside failures in games, arithmetic and logic.
54:00EN
20VC with Harry Stebbings· 2023년 6월 30일팟캐스트 · 2명Douwe Kiela on Data, Retrieval and Enterprise AI게스트 Douwe Kiela · 진행자 Harry StebbingsThe Contextual AI CEO discusses language model trade-offs, evaluation, open models and the hurdles to enterprise use.
18:08ENMatthew Berman· 2023년 6월 29일Six Language Models Compared on an Informal 2023 Prompt SetMatthew Berman · Forward FutureMatthew Berman tests coding, writing and reasoning with H2O GPT, finding a tie under a small and subjective rubric.
7:21ENMatthew Berman· 2023년 6월 26일Testing Orca Mini 13B, an Unofficial Model Inspired by OrcaMatthew Berman · Forward FutureMatthew Berman loads a quantized Orca Mini 13B and tries coding, reasoning, arithmetic, and summarization prompts. His 2023 informal test produces several failures and a few successes.
13:50ENMatthew Berman· 2023년 6월 23일MPT-30B: Running MosaicML’s Model Locally and Testing Its Chat VariantMatthew Berman · Forward FutureMatthew Berman reviews MosaicML’s June 2023 release, demonstrates a local setup, and puts the hosted chat model through an informal prompt rubric.
10:23ENMatthew Berman· 2023년 6월 15일Testing Hosted Falcon 40B and Document Chat in h2oGPTMatthew Berman · Forward FutureMatthew Berman tries H2O.ai’s 2023 hosted Falcon 40B interface on coding, writing and reasoning prompts, then demonstrates document chat.
11:49ENMatthew Berman· 2023년 6월 1일Testing Wizard-Vicuna 30B: Responses, Reasoning, and CodeMatthew Berman · Forward FutureMatthew Berman examines a model marketed as uncensored, demonstrates a 2023 cloud setup, and runs an informal collection of tests.
1:33:51ENCognitive Revolution "How AI Changes Everything"· 2023년 5월 23일팟캐스트GPT-4 still writes SQL that will not run. She put a guardrail on the model진행자 Nathan LabenzShreya Rajpal built Guardrails AI so developers can encode what “correct” means, validate the pieces, and send failures back to the model instead of hoping the prompt holds.
1:25:35ENCognitive Revolution "How AI Changes Everything"· 2023년 5월 21일팟캐스트Two and a Half Years From 33.3% to 86.5%: Google Sent Foundation Models to Medical School진행자 Nathan LabenzVivek Natarajan walks Nathan Labenz through Med-PaLM and Med-PaLM 2: the data-light techniques, the nine-axis clinician evals, and why a high USMLE-style score is the wrong trophy.
42:18ENNo Priors: AI, Machine Learning, Tech, & Startups· 2023년 5월 18일팟캐스트 · 2명Ten Percent of Search Is Health — and the Doctor Is Googling Too진행자 Sarah Guo, Elad GilGoogle researcher Karan Singhal on Med-PaLM 2, the real bar for medical language models, and why medicine is a harder alignment lab than chatbots.
1:28:43ENCognitive Revolution "How AI Changes Everything"· 2023년 5월 16일팟캐스트Clinical Guidelines Are 11% Evidence. Curai’s Founder Still Wants GPT-4 in the Exam Room진행자 Nathan LabenzNeal Khosla of Curai Health argues that large language models already match median U.S. physicians on today’s tests — and that the real work is scaffolding, supervision, and a doctor’s signature.더 보기269개 중 24개 표시