Results
275 articlesSort
1:28:37EN
Cognitive Revolution "How AI Changes Everything"· Mar 14, 2024PodcastTwelve Hours After a GPT-4 Demo, He Turned a Decade-Old Legal Startup AroundHost Nathan LabenzPablo Arredondo, co-founder of CaseText and now VP of CoCounsel at Thomson Reuters, traces legal research from 19th-century citators to a retrieval-anchored GPT-4 product that law firms treat more like a colleague than a search box.
21:16EN
Matthew Berman· Mar 9, 2024ArtPrompt: What a 2024 Study Found About Visual Text and Model SafetyMatthew Berman · Forward FutureMatthew Berman reviews an ASCII-art recognition benchmark and reports mixed results from his own informal tests.
23:00ENMatthew Berman· Feb 22, 2024Testing Google Gemma 7B at Launch: Setup MattersMatthew Berman · Forward FutureMatthew Berman compares local and hosted tests of Google’s early open-weight model.
1:06:39EN
Unsupervised Learning: With Jacob Effron· Jan 9, 2024PodcastOpenAI's First DevRel Writes 90% of His Code with ChatGPT — and Says Web Agents Still Need YearsHost Jacob EffronLogan Kilpatrick on picking GPT-3.5 versus GPT-4, why plugins stalled, and why the public internet is not ready for autonomous agents.
19:37ENMatthew Berman· Dec 13, 2023How Rephrased Test Data Inflated a 13B Model’s Benchmark ScoresMatthew Berman · Forward FutureMatthew Berman examines a 2023 experiment showing how reworded benchmark questions can evade simple contamination checks and inflate model scores.
1:22:59ENCognitive Revolution "How AI Changes Everything"· Dec 2, 2023PodcastIf Your Agent Is Right 80% of the Time, Ten Steps Leave You With 10%Host Nathan LabenzImbue CTO Josh Albrecht tells Nathan Labenz how a 27-person, pre-product research company is putting $200 million and 10,000 H100s toward agents that can actually finish a long job.
18:29ENMatthew Berman· Dec 1, 2023OpenAI's 2023 Leadership Crisis, Q* Speculation and Synthetic DataMatthew Berman, Andrew Ross Sorkin, Elon Musk · Forward FutureMatthew Berman connects Sam Altman's return, an Elon Musk interview and Nathan Lambert's discussion of synthetic training data, separating reported events from his own theories.
32:48EN
No Priors: AI, Machine Learning, Tech, & Startups· Nov 16, 2023Podcast · 2 speakersFourteen People, $200 Million of Compute, and the Reliability Bottleneck for AgentsHost Sarah Guo, Elad GilImbue co-founders Kanjun Qiu and Josh Albrecht on why agents fail, why language models cannot learn general algorithms, and why they are spending a giant round on GPUs rather than headcount.
1:23:10ENCognitive Revolution "How AI Changes Everything"· Sep 19, 2023PodcastLuxury Purses Under $500 Beat a Bank's Fraud ModelHost Nathan LabenzArthur.ai CEO Adam Wenchel on boundary attacks, LLM firewalls, and why a model that eats raw text cannot be secured from the inside.
10:20ENMatthew Berman· Aug 30, 2023Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
55:08ENCognitive Revolution "How AI Changes Everything"· Aug 29, 2023PodcastOne Model Inside 38 Products: Paige Bailey on How Google Trains and Ships PaLM 2Host Nathan LabenzAt Google I/O in May 2023, PaLM 2 arrived already running in more than two dozen products — and its lead product manager walks through what that actually took.
39:45ENNo Priors: AI, Machine Learning, Tech, & Startups· Jul 27, 2023Podcast · 2 speakersPalantir's 13th Employee: The Model Is a Stochastic Genie. Ontology Is the Stake That Tethers ItHost Sarah Guo, Elad GilCTO Shyam Sankar on 550 microservices, air-gapped submarines, and why Palantir's ontology is what large language models were waiting for.
5:34ENMatthew Berman· Jul 26, 2023FreeWilly2: A 2023 Prompt Test with Uncertain FormattingMatthew Berman · Forward FutureMatthew Berman retries an instruction-tuned Llama 2 model after a poor first run, finding mixed results and a likely template problem.
4:59ENMatthew Berman· Jul 25, 2023Testing Airoboros L2 70B on Coding, Writing and ReasoningMatthew Berman · Forward FutureMatthew Berman runs a small set of prompts against a Llama 2 fine-tune, finding useful structured output alongside failures on code and logic.
11:07ENMatthew Berman· Jul 24, 2023Installing Llama 2 13B Chat and Testing Its ResponsesMatthew Berman · Forward FutureMatthew Berman demonstrates a 2023 local setup and tests a 13-billion-parameter chat model on coding, writing, arithmetic and reasoning. His small set of prompts illustrates both useful outputs and mistakes.
9:46ENMatthew Berman· Jul 17, 2023Orca Mini V2 vs. OpenOrca: A 2023 Informal Model ComparisonMatthew Berman · Forward FutureMatthew Berman tests two Orca-inspired models on coding, writing and reasoning, with results shaped by prompt setup and subjective scoring.
8:04ENMatthew Berman· Jul 11, 2023WizardLM 13B v1.1 Tested on Coding, Writing and LogicMatthew Berman · Forward FutureMatthew Berman tries an instruction-tuned model on a small set of prompts, with mixed results and several subjective scoring choices.
12:20ENMatthew Berman· Jul 10, 2023Testing UltraLM 13B: A Small Model’s Strengths and Reasoning ErrorsMatthew Berman · Forward FutureA 2023 local test finds useful writing and simple coding alongside failures in games, arithmetic and logic.
54:00EN
20VC with Harry Stebbings· Jun 30, 2023Podcast · 2 speakersDouwe Kiela on Data, Retrieval and Enterprise AIGuest Douwe Kiela · Host Harry StebbingsThe Contextual AI CEO discusses language model trade-offs, evaluation, open models and the hurdles to enterprise use.
18:08ENMatthew Berman· Jun 29, 2023Six Language Models Compared on an Informal 2023 Prompt SetMatthew Berman · Forward FutureMatthew Berman tests coding, writing and reasoning with H2O GPT, finding a tie under a small and subjective rubric.
7:21ENMatthew Berman· Jun 26, 2023Testing Orca Mini 13B, an Unofficial Model Inspired by OrcaMatthew Berman · Forward FutureMatthew Berman loads a quantized Orca Mini 13B and tries coding, reasoning, arithmetic, and summarization prompts. His 2023 informal test produces several failures and a few successes.
13:50ENMatthew Berman· Jun 23, 2023MPT-30B: Running MosaicML’s Model Locally and Testing Its Chat VariantMatthew Berman · Forward FutureMatthew Berman reviews MosaicML’s June 2023 release, demonstrates a local setup, and puts the hosted chat model through an informal prompt rubric.
10:23ENMatthew Berman· Jun 15, 2023Testing Hosted Falcon 40B and Document Chat in h2oGPTMatthew Berman · Forward FutureMatthew Berman tries H2O.ai’s 2023 hosted Falcon 40B interface on coding, writing and reasoning prompts, then demonstrates document chat.
11:49ENMatthew Berman· Jun 1, 2023Testing Wizard-Vicuna 30B: Responses, Reasoning, and CodeMatthew Berman · Forward FutureMatthew Berman examines a model marketed as uncensored, demonstrates a 2023 cloud setup, and runs an informal collection of tests.Load moreShowing 24 of 275