Resultados
201 artículosOrdenar
19:37EN
Matthew Berman· 13 dic 2023How Rephrased Test Data Inflated a 13B Model’s Benchmark ScoresMatthew Berman · Forward FutureMatthew Berman examines a 2023 experiment showing how reworded benchmark questions can evade simple contamination checks and inflate model scores.
20:49ENMatthew Berman· 12 dic 2023Mixtral 8x7B: Mixture-of-Experts Architecture, Setup and Early TestsMatthew Berman · Forward FutureMatthew Berman reviews Mistral AI’s December 2023 open-weight model, demonstrates a cloud setup and tests its coding and reasoning responses.
26:52ENMatthew Berman· 7 dic 2023Google’s Gemini Launch: Multimodal Demos, Benchmarks and Research ApplicationsMatthew Berman · Forward FutureMatthew Berman reviews Google’s December 2023 Gemini report and promotional demonstrations, from visual reasoning to scientific literature screening.
7:48ENMatthew Berman· 26 nov 2023Testing Dolphin 2.2 Yi 34B on Coding and Reasoning TasksMatthew Berman · Forward FutureMatthew Berman runs a quantized Yi fine-tune through informal coding, writing and logic prompts in November 2023, with uneven results.
10:37ENMatthew Berman· 17 nov 2023Testing DeepSeek Coder on Forms, Bugs and a Snake GameMatthew Berman · Forward FutureMatthew Berman puts the 2023 coding model through a small hands-on rubric, from JSON and unit tests to a playable game.
11:05ENMatthew Berman· 7 nov 2023Testing Dolphin 2.2, a Mistral 7B Fine-TuneMatthew Berman · Forward FutureMatthew Berman tries a small open-weight model on coding, formatting and reasoning prompts, with mixed results.
13:11ENMatthew Berman· 5 oct 2023Testing Mistral 7B OpenOrca: A 2023 Cloud Setup and Prompt TrialMatthew Berman · Forward FutureMatthew Berman runs a quantized OpenOrca variant on a rented GPU and reviews coding, writing and reasoning examples.
12:49ENMatthew Berman· 14 sept 2023Falcon 180B: A 2023 Review of TII’s Open-Weight ModelMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and writing prompts while examining the model’s training scale and license.
10:20ENMatthew Berman· 30 ago 2023Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
2:25ENMatthew Berman· 1 ago 2023Gorilla: Generating API Calls with Language ModelsMatthew Berman · Forward FutureMatthew Berman introduces a 2023 research project on using retrieved documentation to help language models generate tool calls.
5:34ENMatthew Berman· 26 jul 2023FreeWilly2: A 2023 Prompt Test with Uncertain FormattingMatthew Berman · Forward FutureMatthew Berman retries an instruction-tuned Llama 2 model after a poor first run, finding mixed results and a likely template problem.
9:40ENMatthew Berman· 20 jul 2023Testing Llama 2 70B on Coding, Writing and ReasoningMatthew Berman · Forward FutureMatthew Berman tries Meta’s 70-billion-parameter chat model through a hosted interface, highlighting both useful outputs and inconsistent reasoning.
9:46ENMatthew Berman· 17 jul 2023Orca Mini V2 vs. OpenOrca: A 2023 Informal Model ComparisonMatthew Berman · Forward FutureMatthew Berman tests two Orca-inspired models on coding, writing and reasoning, with results shaped by prompt setup and subjective scoring.
18:08ENMatthew Berman· 29 jun 2023Six Language Models Compared on an Informal 2023 Prompt SetMatthew Berman · Forward FutureMatthew Berman tests coding, writing and reasoning with H2O GPT, finding a tie under a small and subjective rubric.
4:36ENMatthew Berman· 20 jun 2023Testing WizardCoder 15B as a Local Coding AssistantMatthew Berman · Forward FutureMatthew Berman runs the 2023 WizardCoder model locally, testing small Python exercises and a longer Snake game.
19:38ENMatthew Berman· 9 jun 2023Orca: Teaching a Smaller Model with GPT-4 ExplanationsMatthew Berman · Forward FutureMatthew Berman reviews Microsoft Research’s 2023 Orca paper and its approach to training a 13-billion-parameter model on detailed teacher responses.
10:40ENMatthew Berman· 7 jun 2023Testing Nous Hermes 13B Locally: Mixed Reasoning and Unsafe ResponsesMatthew Berman · Forward FutureMatthew Berman compares a 2023 quantized model on hosted and local hardware, with uneven results across coding, logic and safety.
11:42ENMatthew Berman· 31 may 2023Guanaco and QLoRA: A 2023 Fine-Tuning and Model TestMatthew Berman · Forward FutureMatthew Berman reviews the QLoRA paper, explains the reported memory savings and tries Guanaco 65B on an informal prompt set.
1:25:35EN
Cognitive Revolution "How AI Changes Everything"· 21 may 2023PódcastTwo and a Half Years From 33.3% to 86.5%: Google Sent Foundation Models to Medical SchoolPresenta: Nathan LabenzVivek Natarajan walks Nathan Labenz through Med-PaLM and Med-PaLM 2: the data-light techniques, the nine-axis clinician evals, and why a high USMLE-style score is the wrong trophy.
10:16ENMatthew Berman· 21 may 2023DarkBERT: Adapting a Language Model to Dark-Web TextMatthew Berman · Forward FutureMatthew Berman reviews research on a RoBERTa-based model and its cybersecurity evaluations.
11:07ENMatthew Berman· 16 may 2023Testing MosaicML MPT-7B Chat and StoryWriter in 2023Matthew Berman · Forward FutureMatthew Berman examines MPT-7B model variants, their licenses and a mixed set of local tests.
58:24ENCognitive Revolution "How AI Changes Everything"· 11 may 2023PódcastAmong 100 Random Doctors, Zak Kohane Would Be Surprised If One Beat GPT-4Presenta: Nathan LabenzHarvard's biomedical informatics chair on a rare newborn case, why drug-style trials will not certify a general medical model, and why the AI should still work for the patient.
9:58ENMatthew Berman· 26 abr 2023HuggingChat’s 2023 Debut: Testing Writing, Reasoning and Python CodeMatthew Berman · Forward FutureMatthew Berman tries the early OpenAssistant-powered HuggingChat on writing, arithmetic, logic and a Snake game, then compares a few outputs with GPT-3.5 and GPT-4. The results illustrate both promise and limitations in this small historical test.
53:37EN
No Priors: AI, Machine Learning, Tech, & Startups· 9 mar 2023Pódcast · 2 voces30 models, 42 scenarios, 7 metrics: Percy Liang's public checkup for language modelsPresenta: Sarah Guo, Elad GilA PubMed-only model once led the U.S. medical licensing exam — until a system roughly 200 times larger overtook it.Cargar másMostrando 24 de 201