Results
269 articlesSort
12:51EN
Matthew Berman· Apr 21, 2024Llama 3 70B on Groq: Fast Inference and an Informal Prompt TestMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and repeated prompts on Groq’s April 2024 Llama 3 service, with mixed results and several scoring caveats.
15:01ENMatthew Berman· Apr 19, 2024Testing Llama 3 at Launch: Code, Reasoning and Image GenerationMatthew Berman · Forward FutureMatthew Berman tries Meta AI’s new interface on a snake game, logic puzzles and rapid image generation.
24:32EN
No Priors: AI, Machine Learning, Tech, & Startups· Apr 18, 2024Podcast · 2 speakersImage Generation Is Still a Loot Box. Suhail Doshi Wants to Scale PixelsHost Sarah Guo, Elad GilThe Playground AI founder tells No Priors why he skipped language models, how a noise-sampling trick fixed SDXL's muddy brightness, and what a large vision model would actually do.
12:02ENMatthew Berman· Apr 13, 2024Testing Karasu Mixtral 8x22B on Coding and Reasoning TasksMatthew Berman · Forward FutureMatthew Berman tries an early community fine-tune of Mistral's large mixture-of-experts model on a Snake game, arithmetic and logic puzzles.
13:59ENMatthew Berman· Apr 5, 2024SWE-agent Tutorial: Navigating Repositories and Attempting GitHub FixesMatthew Berman, Carlos · Forward FutureMatthew Berman sets up a research coding agent, hits an installation problem and a cost limit, then shows an author’s successful SymPy benchmark example.
1:23:09EN
Cognitive Revolution "How AI Changes Everything"· Apr 3, 2024PodcastThe Right Size of an AI Task Is Whatever You Can CheckHost Nathan LabenzElicit founders Andreas Stuhlmüller and Jungwon Byun walk through how a twelve-person research assistant cut summary hallucinations from 1.5 to 0.5, ships about every week and a half, and treats task decomposition as the way to spend more compute without a black box.
9:02ENMatthew Berman· Apr 1, 2024Matthew Berman Tries LLM Colosseum, a Street Fighter Model BenchmarkMatthew Berman · Forward FutureA 2024 tutorial shows language models choosing fighting-game moves, explains the game-state prompt and walks through local setup.
14:23ENMatthew Berman· Mar 31, 2024Testing DBRX: Coding Strengths and Reasoning GapsMatthew Berman · Forward FutureMatthew Berman examines Databricks’ open-weight model and puts it through coding, arithmetic and reasoning prompts.
27:32ENNo Priors: AI, Machine Learning, Tech, & Startups· Mar 28, 2024Podcast · 2 speakersFrom a side project to 700 integrations: Harrison Chase on what still blocks agentsHost Sarah Guo, Elad GilThe LangChain CEO walks through a two-year churn of patterns, and why memory, planning, and few-shot data still matter more than swapping models.
1:28:37ENCognitive Revolution "How AI Changes Everything"· Mar 14, 2024PodcastTwelve Hours After a GPT-4 Demo, He Turned a Decade-Old Legal Startup AroundHost Nathan LabenzPablo Arredondo, co-founder of CaseText and now VP of CoCounsel at Thomson Reuters, traces legal research from 19th-century citators to a retrieval-anchored GPT-4 product that law firms treat more like a colleague than a search box.
21:16ENMatthew Berman· Mar 9, 2024ArtPrompt: What a 2024 Study Found About Visual Text and Model SafetyMatthew Berman · Forward FutureMatthew Berman reviews an ASCII-art recognition benchmark and reports mixed results from his own informal tests.
23:00ENMatthew Berman· Feb 22, 2024Testing Google Gemma 7B at Launch: Setup MattersMatthew Berman · Forward FutureMatthew Berman compares local and hosted tests of Google’s early open-weight model.
1:06:39EN
Unsupervised Learning: With Jacob Effron· Jan 9, 2024PodcastOpenAI's First DevRel Writes 90% of His Code with ChatGPT — and Says Web Agents Still Need YearsHost Jacob EffronLogan Kilpatrick on picking GPT-3.5 versus GPT-4, why plugins stalled, and why the public internet is not ready for autonomous agents.
19:37ENMatthew Berman· Dec 13, 2023How Rephrased Test Data Inflated a 13B Model’s Benchmark ScoresMatthew Berman · Forward FutureMatthew Berman examines a 2023 experiment showing how reworded benchmark questions can evade simple contamination checks and inflate model scores.
1:22:59ENCognitive Revolution "How AI Changes Everything"· Dec 2, 2023PodcastIf Your Agent Is Right 80% of the Time, Ten Steps Leave You With 10%Host Nathan LabenzImbue CTO Josh Albrecht tells Nathan Labenz how a 27-person, pre-product research company is putting $200 million and 10,000 H100s toward agents that can actually finish a long job.
18:29ENMatthew Berman· Dec 1, 2023OpenAI's 2023 Leadership Crisis, Q* Speculation and Synthetic DataMatthew Berman, Andrew Ross Sorkin, Elon Musk · Forward FutureMatthew Berman connects Sam Altman's return, an Elon Musk interview and Nathan Lambert's discussion of synthetic training data, separating reported events from his own theories.
32:48ENNo Priors: AI, Machine Learning, Tech, & Startups· Nov 16, 2023Podcast · 2 speakersFourteen People, $200 Million of Compute, and the Reliability Bottleneck for AgentsHost Sarah Guo, Elad GilImbue co-founders Kanjun Qiu and Josh Albrecht on why agents fail, why language models cannot learn general algorithms, and why they are spending a giant round on GPUs rather than headcount.
1:23:10ENCognitive Revolution "How AI Changes Everything"· Sep 19, 2023PodcastLuxury Purses Under $500 Beat a Bank's Fraud ModelHost Nathan LabenzArthur.ai CEO Adam Wenchel on boundary attacks, LLM firewalls, and why a model that eats raw text cannot be secured from the inside.
10:20ENMatthew Berman· Aug 30, 2023Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
55:08ENCognitive Revolution "How AI Changes Everything"· Aug 29, 2023PodcastOne Model Inside 38 Products: Paige Bailey on How Google Trains and Ships PaLM 2Host Nathan LabenzAt Google I/O in May 2023, PaLM 2 arrived already running in more than two dozen products — and its lead product manager walks through what that actually took.
39:45ENNo Priors: AI, Machine Learning, Tech, & Startups· Jul 27, 2023Podcast · 2 speakersPalantir's 13th Employee: The Model Is a Stochastic Genie. Ontology Is the Stake That Tethers ItHost Sarah Guo, Elad GilCTO Shyam Sankar on 550 microservices, air-gapped submarines, and why Palantir's ontology is what large language models were waiting for.
5:34ENMatthew Berman· Jul 26, 2023FreeWilly2: A 2023 Prompt Test with Uncertain FormattingMatthew Berman · Forward FutureMatthew Berman retries an instruction-tuned Llama 2 model after a poor first run, finding mixed results and a likely template problem.
4:59ENMatthew Berman· Jul 25, 2023Testing Airoboros L2 70B on Coding, Writing and ReasoningMatthew Berman · Forward FutureMatthew Berman runs a small set of prompts against a Llama 2 fine-tune, finding useful structured output alongside failures on code and logic.
11:07ENMatthew Berman· Jul 24, 2023Installing Llama 2 13B Chat and Testing Its ResponsesMatthew Berman · Forward FutureMatthew Berman demonstrates a 2023 local setup and tests a 13-billion-parameter chat model on coding, writing, arithmetic and reasoning. His small set of prompts illustrates both useful outputs and mistakes.Load moreShowing 24 of 269