Results
167 articlesSort
10:57EN
Matthew Berman· Sep 13, 2024Testing OpenAI o1-preview: Tetris, Physical Reasoning and a North Pole ErrorMatthew Berman · Forward FutureIn September 2024, Matthew Berman runs the new reasoning-model preview through a personal rubric and examines where its answers succeed or fail.
13:40ENMatthew Berman· Jul 29, 2024Testing Llama 3.1 8B: Benchmark Gains and Mixed Practical ResultsMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and other prompts on the small model and finds several failures despite stronger reported benchmarks.
10:57ENMatthew Berman· Jun 24, 2024Testing Claude 3.5 Sonnet: Coding, Reasoning, and VisionMatthew Berman · Forward FutureMatthew Berman puts Anthropic’s June 2024 model through informal coding, math, puzzle, and image-understanding prompts.
17:03ENMatthew Berman· Jun 8, 2024Qwen2 0.5B and 72B: A Hands-On ComparisonMatthew Berman · Forward FutureMatthew Berman tests coding, arithmetic and reasoning prompts on the smallest and largest Qwen2 models in a 2024 demonstration.
17:12ENMatthew Berman· Apr 30, 2024Testing the Mystery 'gpt2-chatbot' Model in April 2024Matthew Berman · Forward FutureMatthew Berman tries an anonymous model on coding, arithmetic and reasoning tasks while its identity remains uncertain.
12:51ENMatthew Berman· Apr 21, 2024Llama 3 70B on Groq: Fast Inference and an Informal Prompt TestMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and repeated prompts on Groq’s April 2024 Llama 3 service, with mixed results and several scoring caveats.
1:25:10EN
Cognitive Revolution "How AI Changes Everything"· Mar 12, 2024PodcastWriting Code Is Only 30% of the Job. CodiumAI Wants the Other 70%Host Nathan LabenzItamar Friedman argues that completion is already a commodity, that code integrity is where AI still lags, and that flow engineering—not fancier prompts—is how GPT-4 doubled its score on coding contests.
13:57ENMatthew Berman· Feb 27, 2024Mistral Large: A 2024 Review of Coding, Reasoning and Safety ResponsesMatthew Berman · Forward FutureMatthew Berman reviews Mistral’s launch claims and tests the model with a small set of prompts.
11:14ENMatthew Berman· Feb 8, 2024Gemini Advanced and Ultra 1.0: A First Look and Hands-On TestMatthew Berman · Forward FutureIn February 2024, Matthew Berman reviews Google’s Bard rebrand and tests its new paid Gemini experience on coding, logic and image recognition.
1:22:59ENCognitive Revolution "How AI Changes Everything"· Dec 2, 2023PodcastIf Your Agent Is Right 80% of the Time, Ten Steps Leave You With 10%Host Nathan LabenzImbue CTO Josh Albrecht tells Nathan Labenz how a 27-person, pre-product research company is putting $200 million and 10,000 H100s toward agents that can actually finish a long job.
1:15:39ENCognitive Revolution "How AI Changes Everything"· Nov 29, 2023PodcastAfter 20 Years of Frontend, v0 Wrote Better Code Than Guillermo Rauch DidHost Nathan LabenzThe Vercel CEO tells Nathan Labenz why every product needs ghost text, why Software 2.0 will ossify the old stack, and why betting against open source is still the expensive mistake.
7:48ENMatthew Berman· Nov 26, 2023Testing Dolphin 2.2 Yi 34B on Coding and Reasoning TasksMatthew Berman · Forward FutureMatthew Berman runs a quantized Yi fine-tune through informal coding, writing and logic prompts in November 2023, with uneven results.
5:31ENMatthew Berman· Oct 30, 2023Connecting MemGPT to AutoGen AgentsMatthew Berman · Forward FutureIn a 2023 tutorial, Matthew Berman configures a MemGPT-backed coding agent alongside an AutoGen product manager and runs a short group chat.
7:02ENMatthew Berman· Oct 18, 2023Connecting AutoGen to a RunPod-Hosted Model with Text Generation WebUIMatthew Berman · Forward FutureIn a 2023 walkthrough, Matthew Berman connects AutoGen to Dolphin 2.1 Mistral 7B through an OpenAI-compatible API and tests a small coding task.
15:03ENMatthew Berman· Oct 10, 2023A 2023 AutoGen Tutorial: Local Agent Setup and Code ExecutionMatthew Berman · Forward FutureMatthew Berman builds a local assistant and user-proxy workflow, then tests how the agents generate and run Python. AutoGen’s packages and APIs have changed since this tutorial.
1:32:41ENCognitive Revolution "How AI Changes Everything"· Oct 5, 2023PodcastThere are maybe 5,000 people who can train frontier models. The rest need AI engineersHost Nathan LabenzSwyx of Latent Space tells Nathan Labenz how a new job is forming, why OpenAI will Sherlock its developers, and what still does not count as an agent.
7:32ENMatthew Berman· Oct 4, 2023MageGPT Generates a Wasp To-Do App: A 2023 DemonstrationMatthew Berman · Forward FutureMatthew Berman uses a Wasp-sponsored app generator to build, run and deploy a basic full-stack to-do app.
2:17:30ENCognitive Revolution "How AI Changes Everything"· Sep 29, 2023PodcastGemini Became the Daily Driver. A GPT-5 Worthy of the Name Would Make Agents Work.Host Nathan LabenzZvi Mowshowitz returns to Cognitive Revolution to unpack Gemini 1.5, Sora, superalignment, sleeper agents, and the live-player map of frontier labs.
9:59ENMatthew Berman· Sep 7, 2023Open Interpreter: A 2023 Local Code-Execution WalkthroughMatthew Berman · Forward FutureMatthew Berman installs Open Interpreter, analyzes local files with GPT-4 and tries a local Code Llama setup, with mixed results.
6:11ENMatthew Berman· Aug 31, 2023Running WizardCoder Locally with Text Generation WebUIMatthew Berman · Forward FutureMatthew Berman walks through a 2023 setup for a quantized Code Llama-based coding model, from the Python environment to a simple test prompt.
10:20ENMatthew Berman· Aug 30, 2023Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
1:39:53ENCognitive Revolution "How AI Changes Everything"· Aug 8, 2023PodcastUnder 100 People, 24 Million Users: Replit's Bet on Artificial Developer IntelligenceHost Nathan LabenzMichele Catasta, Replit's new VP of AI, lays out a generate-execute-learn loop he will not call AGI, a 2x-to-1000x productivity ladder, and a keystroke-level data advantage.
1:45:46ENCognitive Revolution "How AI Changes Everything"· Aug 4, 2023PodcastReplit's $10 AI Programmer Already Pays for Itself. The Ambition Is Bots Hiring BotsHost Nathan LabenzTyler Angert, Replit's first product designer, explains Ghostwriter, a virtual developer, and why the company calls itself the perfect substrate for AGI.
12:20ENMatthew Berman· Jul 10, 2023Testing UltraLM 13B: A Small Model’s Strengths and Reasoning ErrorsMatthew Berman · Forward FutureA 2023 local test finds useful writing and simple coding alongside failures in games, arithmetic and logic.Load moreShowing 24 of 167