Résultats
150 articlesTrier
10:57EN
Matthew Berman· 13 sept. 2024Testing OpenAI o1-preview: Tetris, Physical Reasoning and a North Pole ErrorMatthew Berman · Forward FutureIn September 2024, Matthew Berman runs the new reasoning-model preview through a personal rubric and examines where its answers succeed or fail.
21:21ENMatthew Berman· 12 sept. 2024OpenAI o1’s 2024 Launch: Reasoning Benchmarks and Early TestsMatthew Berman · Forward FutureMatthew Berman reviews o1-preview and o1-mini, OpenAI’s reported evaluations, and his own early coding experiments.
19:38ENMatthew Berman· 9 sept. 2024Reflection 70B: Conflicting Results and Unanswered QuestionsMatthew Berman · Forward FutureMatthew Berman reviews his early coverage, independent evaluations and competing claims about the September 2024 release.
18:39ENMatthew Berman· 17 août 2024AI Scientist, Grok-2 and Agent Q: Matthew Berman’s August 2024 AI News RoundupMatthew Berman · Forward FutureA week of model releases, web agents and AI research raises questions about benchmarks, automation and the gap between prototypes and predictions.
9:58ENMatthew Berman· 2 août 2024Abstract AI: A Proposed Routing Layer for Language ModelsMatthew Berman · Forward FutureMatthew Berman outlines a startup idea for choosing models and workflows by cost, speed and quality.
15:50ENMatthew Berman· 23 juil. 2024Llama 3.1: Meta’s 405B Model and the Open-Weight EcosystemMatthew Berman · Forward FutureMatthew Berman reviews Meta’s July 2024 release, the smaller models, reported benchmarks, synthetic data and the proposed Llama Stack.
8:52ENMatthew Berman· 14 juil. 2024Yi-Large Tested: Coding, Arithmetic and Logic PuzzlesMatthew Berman · Forward FutureMatthew Berman puts 01.AI's hosted model through a small set of coding and reasoning prompts, with mixed results and some questionable grading.
11:30ENMatthew Berman· 11 juil. 2024Testing Google Gemma 2 27B: Coding, Reasoning and FormattingMatthew Berman · Forward FutureMatthew Berman runs an informal 2024 test of Google’s open-weight model on cloud hardware supplied by sponsor Vultr.
10:57ENMatthew Berman· 24 juin 2024Testing Claude 3.5 Sonnet: Coding, Reasoning, and VisionMatthew Berman · Forward FutureMatthew Berman puts Anthropic’s June 2024 model through informal coding, math, puzzle, and image-understanding prompts.
16:11ENMatthew Berman· 22 juin 2024AI News in June 2024: SSI, Tree Search, Meta, Claude and ARC PrizeMatthew Berman · Forward FutureMatthew Berman reviews new AI research, product launches and a competition, with context on historical benchmarks and uncertain claims.
27:37ENMatthew Berman· 17 juin 2024Leo Pekelis on Extending Llama 3 to a Million-Token ContextMatthew Berman, Leo Pekelis · Forward FutureGradient’s chief scientist explains long-context training, coding use cases, inference costs and why retrieval benchmarks tell only part of the story.
17:03ENMatthew Berman· 8 juin 2024Qwen2 0.5B and 72B: A Hands-On ComparisonMatthew Berman · Forward FutureMatthew Berman tests coding, arithmetic and reasoning prompts on the smallest and largest Qwen2 models in a 2024 demonstration.
14:25ENMatthew Berman· 2 juin 2024Testing Phi-3 Vision, LLaVA-Llama 3 and GPT-4o on ImagesMatthew Berman · Forward FutureMatthew Berman compares three 2024 vision setups on image descriptions, screenshots, a meme and table extraction, with mixed results.
9:53ENMatthew Berman· 28 mai 2024Testing Microsoft Phi-3 Medium Locally: Mixed Coding and Reasoning ResultsMatthew Berman · Forward FutureMatthew Berman runs a quantized Phi-3 Medium on a MacBook and explores coding, arithmetic and reasoning prompts.
13:37ENMatthew Berman· 16 mai 2024Testing Gemini 1.5 Pro’s Long Context and Video UnderstandingMatthew Berman · Forward FutureMatthew Berman’s 2024 tests show useful image and video summaries alongside blocked code and missed details.
16:25ENMatthew Berman· 8 mai 2024Visualization-of-Thought and an Open-Source Windows AssistantMatthew Berman · Forward FutureMatthew Berman explains a spatial-reasoning prompting experiment and examines desktop automation demos from the separate PyWinAssistant project.
17:12ENMatthew Berman· 30 avr. 2024Testing the Mystery 'gpt2-chatbot' Model in April 2024Matthew Berman · Forward FutureMatthew Berman tries an anonymous model on coding, arithmetic and reasoning tasks while its identity remains uncertain.
19:09ENMatthew Berman· 28 avr. 2024OSWorld: Evaluating Computer-Use Agents in Real ApplicationsMatthew Berman · Forward FutureMatthew Berman explains the 2024 benchmark’s observation–action loop, task-specific checks and early results, with the limitations of its experimental setup.
21:47ENMatthew Berman· 24 avr. 2024Phi-3 Mini: Microsoft’s Small Model, Phone Deployment and an Uncertain Local TestMatthew Berman · Forward FutureMatthew Berman reviews Microsoft’s 2024 technical report and runs informal prompts, with an unresolved model label in the demonstration.
16:28ENMatthew Berman· 23 avr. 2024Testing Reka Core: Multimodal Benchmarks, Coding and ReasoningMatthew Berman · Forward FutureMatthew Berman reviews Reka’s reported model comparisons and tests Core on code, logic and image understanding.
12:51ENMatthew Berman· 21 avr. 2024Llama 3 70B on Groq: Fast Inference and an Informal Prompt TestMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and repeated prompts on Groq’s April 2024 Llama 3 service, with mixed results and several scoring caveats.
15:35ENMatthew Berman· 18 avr. 2024Meta Llama 3 Launch: Benchmarks, a Snake Test and the Meta AI AssistantMatthew Berman · Forward FutureMatthew Berman reviews the April 2024 release, tries a generated game and considers what downloadable models mean for developers.
14:21ENMatthew Berman· 17 avr. 2024Grok-1.5V Preview: Vision Examples and a New Spatial BenchmarkMatthew Berman · Forward FutureMatthew Berman reviews xAI’s 2024 multimodal model preview, its reported benchmark results and example image tasks.
1:00ENMatthew Berman· 2 avr. 2024LLM Colosseum: Language Models Play Street Fighter IIIMatthew Berman · Forward FutureMatthew Berman watches a short match in which language-model agents choose fighting-game actions from observations of the game.Charger plus24 sur 150 affichés