필터 결과
201개정렬
19:37EN
Matthew Berman· 2023년 12월 13일How Rephrased Test Data Inflated a 13B Model’s Benchmark ScoresMatthew Berman · Forward FutureMatthew Berman examines a 2023 experiment showing how reworded benchmark questions can evade simple contamination checks and inflate model scores.
20:49ENMatthew Berman· 2023년 12월 12일Mixtral 8x7B: Mixture-of-Experts Architecture, Setup and Early TestsMatthew Berman · Forward FutureMatthew Berman reviews Mistral AI’s December 2023 open-weight model, demonstrates a cloud setup and tests its coding and reasoning responses.
26:52ENMatthew Berman· 2023년 12월 7일Google’s Gemini Launch: Multimodal Demos, Benchmarks and Research ApplicationsMatthew Berman · Forward FutureMatthew Berman reviews Google’s December 2023 Gemini report and promotional demonstrations, from visual reasoning to scientific literature screening.
7:48ENMatthew Berman· 2023년 11월 26일Testing Dolphin 2.2 Yi 34B on Coding and Reasoning TasksMatthew Berman · Forward FutureMatthew Berman runs a quantized Yi fine-tune through informal coding, writing and logic prompts in November 2023, with uneven results.
10:37ENMatthew Berman· 2023년 11월 17일Testing DeepSeek Coder on Forms, Bugs and a Snake GameMatthew Berman · Forward FutureMatthew Berman puts the 2023 coding model through a small hands-on rubric, from JSON and unit tests to a playable game.
11:05ENMatthew Berman· 2023년 11월 7일Testing Dolphin 2.2, a Mistral 7B Fine-TuneMatthew Berman · Forward FutureMatthew Berman tries a small open-weight model on coding, formatting and reasoning prompts, with mixed results.
13:11ENMatthew Berman· 2023년 10월 5일Testing Mistral 7B OpenOrca: A 2023 Cloud Setup and Prompt TrialMatthew Berman · Forward FutureMatthew Berman runs a quantized OpenOrca variant on a rented GPU and reviews coding, writing and reasoning examples.
12:49ENMatthew Berman· 2023년 9월 14일Falcon 180B: A 2023 Review of TII’s Open-Weight ModelMatthew Berman · Forward FutureMatthew Berman tests coding, reasoning and writing prompts while examining the model’s training scale and license.
10:20ENMatthew Berman· 2023년 8월 30일Phind Code Llama 34B vs GPT-4: A 2023 Python Coding TestMatthew Berman · Forward FutureMatthew Berman compares a quantized Phind fine-tune with GPT-4 on games, short functions and refactoring, with mixed results.
2:25ENMatthew Berman· 2023년 8월 1일Gorilla: Generating API Calls with Language ModelsMatthew Berman · Forward FutureMatthew Berman introduces a 2023 research project on using retrieved documentation to help language models generate tool calls.
5:34ENMatthew Berman· 2023년 7월 26일FreeWilly2: A 2023 Prompt Test with Uncertain FormattingMatthew Berman · Forward FutureMatthew Berman retries an instruction-tuned Llama 2 model after a poor first run, finding mixed results and a likely template problem.
9:40ENMatthew Berman· 2023년 7월 20일Testing Llama 2 70B on Coding, Writing and ReasoningMatthew Berman · Forward FutureMatthew Berman tries Meta’s 70-billion-parameter chat model through a hosted interface, highlighting both useful outputs and inconsistent reasoning.
9:46ENMatthew Berman· 2023년 7월 17일Orca Mini V2 vs. OpenOrca: A 2023 Informal Model ComparisonMatthew Berman · Forward FutureMatthew Berman tests two Orca-inspired models on coding, writing and reasoning, with results shaped by prompt setup and subjective scoring.
18:08ENMatthew Berman· 2023년 6월 29일Six Language Models Compared on an Informal 2023 Prompt SetMatthew Berman · Forward FutureMatthew Berman tests coding, writing and reasoning with H2O GPT, finding a tie under a small and subjective rubric.
4:36ENMatthew Berman· 2023년 6월 20일Testing WizardCoder 15B as a Local Coding AssistantMatthew Berman · Forward FutureMatthew Berman runs the 2023 WizardCoder model locally, testing small Python exercises and a longer Snake game.
19:38ENMatthew Berman· 2023년 6월 9일Orca: Teaching a Smaller Model with GPT-4 ExplanationsMatthew Berman · Forward FutureMatthew Berman reviews Microsoft Research’s 2023 Orca paper and its approach to training a 13-billion-parameter model on detailed teacher responses.
10:40ENMatthew Berman· 2023년 6월 7일Testing Nous Hermes 13B Locally: Mixed Reasoning and Unsafe ResponsesMatthew Berman · Forward FutureMatthew Berman compares a 2023 quantized model on hosted and local hardware, with uneven results across coding, logic and safety.
11:42ENMatthew Berman· 2023년 5월 31일Guanaco and QLoRA: A 2023 Fine-Tuning and Model TestMatthew Berman · Forward FutureMatthew Berman reviews the QLoRA paper, explains the reported memory savings and tries Guanaco 65B on an informal prompt set.
1:25:35EN
Cognitive Revolution "How AI Changes Everything"· 2023년 5월 21일팟캐스트Two and a Half Years From 33.3% to 86.5%: Google Sent Foundation Models to Medical School진행자 Nathan LabenzVivek Natarajan walks Nathan Labenz through Med-PaLM and Med-PaLM 2: the data-light techniques, the nine-axis clinician evals, and why a high USMLE-style score is the wrong trophy.
10:16ENMatthew Berman· 2023년 5월 21일DarkBERT: Adapting a Language Model to Dark-Web TextMatthew Berman · Forward FutureMatthew Berman reviews research on a RoBERTa-based model and its cybersecurity evaluations.
11:07ENMatthew Berman· 2023년 5월 16일Testing MosaicML MPT-7B Chat and StoryWriter in 2023Matthew Berman · Forward FutureMatthew Berman examines MPT-7B model variants, their licenses and a mixed set of local tests.
58:24ENCognitive Revolution "How AI Changes Everything"· 2023년 5월 11일팟캐스트Among 100 Random Doctors, Zak Kohane Would Be Surprised If One Beat GPT-4진행자 Nathan LabenzHarvard's biomedical informatics chair on a rare newborn case, why drug-style trials will not certify a general medical model, and why the AI should still work for the patient.
9:58ENMatthew Berman· 2023년 4월 26일HuggingChat’s 2023 Debut: Testing Writing, Reasoning and Python CodeMatthew Berman · Forward FutureMatthew Berman tries the early OpenAssistant-powered HuggingChat on writing, arithmetic, logic and a Snake game, then compares a few outputs with GPT-3.5 and GPT-4. The results illustrate both promise and limitations in this small historical test.
53:37EN
No Priors: AI, Machine Learning, Tech, & Startups· 2023년 3월 9일팟캐스트 · 2명30 models, 42 scenarios, 7 metrics: Percy Liang's public checkup for language models진행자 Sarah Guo, Elad GilA PubMed-only model once led the U.S. medical licensing exam — until a system roughly 200 times larger overtook it.더 보기201개 중 24개 표시