Results
273 articlesSort
2:03:55EN
Cognitive Revolution "How AI Changes Everything"· Sep 18, 2024PodcastThe More You Constrain an Agent, the More You Can Trust ItHost Nathan LabenzAgent.ai engineering lead Andrei Oprisan tells Nathan Labenz why narrow, highly prescribed workflows beat open-ended autonomy — and how a 12-person team shipped a free agent network past 40,000 users in a few months.
2:10:13ENCognitive Revolution "How AI Changes Everything"· Sep 14, 2024PodcastA 30x drop in scheming is not zero, and the chain of thought is getting strangerHost Nathan LabenzMarius Hobbhahn of Apollo Research walks through a joint OpenAI study that used deliberative alignment to cut covert actions in o3 and o4-mini, then found the leftover still looks like scheming and that models already behave differently when they notice they are being tested.
1:06:15ENCognitive Revolution "How AI Changes Everything"· Sep 14, 2024PodcastJailbreaks Got Harder. Scale Made One Attack Worse.Host Nathan LabenzHaize Labs spent a month automatically attacking OpenAI's o1. The model reasons more carefully about what it should refuse — except when you teach it a homemade cipher.
10:57EN
Matthew Berman· Sep 13, 2024Testing OpenAI o1-preview: Tetris, Physical Reasoning and a North Pole ErrorMatthew Berman · Forward FutureIn September 2024, Matthew Berman runs the new reasoning-model preview through a personal rubric and examines where its answers succeed or fail.
1:31:56ENMatthew Berman· Sep 12, 2024Testing OpenAI o1 on Puzzles, Code and Reasoning: A Launch-Day LivestreamMatthew Berman · Forward FutureMatthew Berman tries o1-preview and o1-mini on word counts, Tetris, physical reasoning and a word puzzle, with viewers checking the results.
19:38ENMatthew Berman· Sep 9, 2024Reflection 70B: Conflicting Results and Unanswered QuestionsMatthew Berman · Forward FutureMatthew Berman reviews his early coverage, independent evaluations and competing claims about the September 2024 release.
12:03ENMatthew Berman· Sep 6, 2024Reflection 70B: Self-Correction Claims and the Limits of a 2024 DemoMatthew Berman · Forward FutureMatthew Berman reviews a model trained to generate reflection steps. Its initial benchmark claims later proved difficult for independent evaluators to reproduce.
35:30ENMatthew Berman· Sep 6, 2024Reflection 70B: Its Creators Explain the Proposed Self-Correction MethodMatthew Berman, Matt Shumer, Sahil Chaudhary · Forward FutureA launch-day discussion with Matt Shumer and Sahil Chaudhary covers synthetic data, selective reflection and evaluation limits.
1:54:17ENCognitive Revolution "How AI Changes Everything"· Aug 28, 2024PodcastMetaculus Flagged Covid in January 2020. Now It Is Putting AI on the Forecasting DeskHost Nathan LabenzNathan Labenz talks with Metaculus CEO Deger Turan about epistemic security, a year-long bot tournament, and why a useful forecast is harder than an accurate one.
1:12:31EN
Sequoia Capital· Aug 27, 2024PodcastA Bare Model Closes 35% of Airline Changes. Sierra's Agents Clear 70% of Real Tickets.Clay Bavor on why putting a raw LLM in front of customers does not math, and how AgentOS, supervisor agents, and outcome pricing change the arithmetic.
9:43ENMatthew Berman· Aug 26, 2024Matthew Berman Tests Jamba 1.5 on Coding and Reasoning PromptsMatthew Berman · Forward FutureA 2024 hands-on test of AI21’s hybrid model finds slow output in one hosted session and mixed answers to a small set of prompts.
11:39ENMatthew Berman· Aug 15, 2024Grok 2 Beta: Matthew Berman Tests Reasoning and FLUX Image GenerationMatthew Berman · Forward FutureA 2024 look at xAI’s Grok 2 and Grok 2 mini covers early benchmark claims, a failed reasoning test and image-generation experiments.
15:24ENMatthew Berman· Aug 13, 2024Testing the Anonymous sus-column-r Model, Later Identified as Grok-2Matthew Berman · Forward FutureMatthew Berman tests a mysterious 2024 Arena model on coding and reasoning; its verbose answers prompted speculation about OpenAI that the later identification contradicted.
57:55ENMatthew Berman· Aug 12, 2024A First AI Livestream: Model Rumors, RAG, Agents, and Audience QuestionsMatthew Berman · Forward FutureMatthew Berman tests a livestream in August 2024 and answers viewers’ questions about AI tools, model evaluation, company knowledge, and future agents.
9:51ENMatthew Berman· Aug 7, 2024Strawberry Rumors and Two Anonymous Chatbot Reasoning TestsMatthew Berman · Forward FutureIn August 2024, Matthew Berman examined speculation about OpenAI’s next model and tested mystery chatbots on simple reasoning puzzles. Later disclosures clarified the identity of one of them.
12:17ENMatthew Berman· Aug 7, 2024Testing Gemini 1.5 Pro on Code, Reasoning, Images and VideoMatthew Berman · Forward FutureMatthew Berman compares standard and experimental Gemini versions in an informal 2024 review, finding mixed results and repeated output errors.
11:34ENMatthew Berman· Aug 1, 2024Testing Mistral Large 2: Coding, Reasoning and Safety in 2024Matthew Berman · Forward FutureMatthew Berman tests Mistral’s 123-billion-parameter model with small coding tasks, logic puzzles and ethical hypotheticals. The results are promising but narrowly scoped.
42:08ENSequoia Capital· Jul 30, 2024PodcastMeta Trained a 405B Teacher on 16,000 GPUs — Then Told the World to Distill ItTwo days after Llama 3.1 405B shipped, Meta's generative-AI product lead walks Sequoia through why the most expensive model in the family is free, and why he thinks the frontier itself is heading toward commodity.
13:40ENMatthew Berman· Jul 29, 2024Testing Llama 3.1 8B: Benchmark Gains and Mixed Practical ResultsMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and other prompts on the small model and finds several failures despite stronger reported benchmarks.
5:50ENMatthew Berman· Jul 27, 2024A Mixture-of-Agents Demo Using Groq and StreamlitMatthew Berman · Forward FutureMatthew Berman walks through a 2024 Groq-powered multi-agent project, from setup to inspecting its layered responses.
10:02ENMatthew Berman· Jul 24, 2024Testing Llama 3.1 405B on Coding, Arithmetic and ReasoningMatthew Berman · Forward FutureMatthew Berman tries Meta's 2024 open-weight model on an informal set of prompts and demonstrates a sponsored dataset workflow.
13:19ENMatthew Berman· Jul 20, 2024Testing GPT-4o Mini Against GPT-4o at LaunchMatthew Berman · Forward FutureMatthew Berman compares OpenAI's 2024 small model on coding, reasoning and image prompts, with a sponsored HP and Intel laptop segment.
8:52ENMatthew Berman· Jul 14, 2024Yi-Large Tested: Coding, Arithmetic and Logic PuzzlesMatthew Berman · Forward FutureMatthew Berman puts 01.AI's hosted model through a small set of coding and reasoning prompts, with mixed results and some questionable grading.
25:56EN
No Priors: AI, Machine Learning, Tech, & Startups· Jul 11, 2024Podcast · 2 speakersA Prompt Cannot Hold a Good Design: Six AI Practitioners at Midyear 2024Host Sarah Guo, Elad GilSarah Guo and Elad Gil stitch together clips from Stripe, Figma, Figure, OpenAI's Sora team, Cognition, and Scale into a half-year snapshot of what people actually shipping AI were willing to claim.Load moreShowing 24 of 273