最近更新的频道

点选即可只看该频道浏览全部频道 →

筛选结果

24 篇
排序
7 张幻灯片35:19EN → 简体中文
AI Engineer· 2026年10月25日SallyAnn DeLucia:如何编写你的第一个 AI 评估SallyAnn DeLucia · Arize AI从智能体的真实追踪记录中提炼具体的通过与失败标准,再构建一个能测试、能与人工判断对齐,并能在生产环境运行的大模型裁判。评估与可观测性评估智能体工程+2
2:10:42EN → 简体中文
Cognitive Revolution "How AI Changes Everything"· 2026年8月28日播客没人审核的考卷主持 Nathan Labenz前沿实验室从一群小供应商那里购买强化学习(reinforcement learning)环境,却几乎没人核查这些测试是否贴合真实工作。评估与可观测性评估Reward hacking+3
36:19EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026年6月26日播客 · 2 人对谈The Benchmark Grid Is Broken: Why Test-Time Compute Has Outrun How We Score Models主持 Sarah Guo, Elad GilOpenAI researcher Noam Brown argues that static leaderboards hide how long a model is allowed to think — and that safety evals, research taste, and the release cycle have not caught up.评估与可观测性评估Benchmarks+3
48:00EN
Unsupervised Learning: With Jacob Effron· 2025年12月15日播客A Two-Second Vote Is Training Models to Be Clickbait主持 Jacob EffronSurge AI CEO Edwin Chen tells Redpoint why LMArena rewards length and emojis, how one lab's models got worse for a year, and why frontier labs are no longer on the same path.评估与可观测性RL environments & simulatorsData quality & curation+3
32:59EN
No Priors: AI, Machine Learning, Tech, & Startups· 2025年7月24日播客 · 2 人对谈A Hundred People, a Billion in Revenue, and Zero Venture Money主持 Sarah Guo, Elad GilSurge AI founder Edwin Chen on why quality human data is a poetry problem, not a bounding-box problem, and why leaderboards keep training models to write clickbait.评估与可观测性Data quality & curation人工介入验证+3
47:07EN
Sequoia Capital· 2025年7月8日播客Goodfire’s Eric Ho thinks we will decode neural nets by 2028On Sequoia’s Training Data, the interpretability founder argues that sparse autoencoders, model editing, and a third-party lab are how AI stops being grown from data and starts being designed.评估与可观测性Model Interpretability安全与治理+3
40:33EN
Sequoia Capital· 2025年6月24日播客Fifty People in Slack. Four Minutes to the Smoking Gun.Traversal's co-founders on turning DevOps heart attacks into agent-run root cause analysis — and why observability's second-largest software spend still leaves troubleshooting as a whodunit.评估与可观测性可观测性与可靠性智能体工程+3
54:53EN → 简体中文
Unsupervised Learning: With Jacob Effron· 2025年6月17日播客一个免费投票榜烧掉 200 万美元,之后却融到了 1 亿美元主持 Jacob EffronDatabricks 联合创始人 Ion Stoica 解释了 LMArena 为何存在,中国为何在开源(open source)模型上占据优势,以及为什么真正拖住 AI 的仍然是可靠性,而不是又一条规模定律(scaling law)。评估与可观测性评估Benchmarks+3
44:21EN
Unsupervised Learning: With Jacob Effron· 2025年6月4日播客Mercor's CEO: Knowledge Work's Next Job Is Writing Evals主持 Jacob EffronBrendan Foody tells Redpoint how Mercor turned AI hiring and expert data into a $100 million round — and why he thinks a huge slice of white-collar work will become creating tests for models.评估与可观测性评估Data quality & curation+3
57:11EN → 简体中文
Unsupervised Learning: With Jacob Effron· 2025年1月30日播客一百万个 Token,也写不出一个好笑话主持 Jacob Effron普林斯顿大学的 Arvind Narayanan 谈:为什么智能体评测会误导人,推理时扩展(inference scaling)为何会撞上墙,以及 AI 更像互联网,而非一夜之间的革命。评估与可观测性评估Benchmarks+3
1:46:41EN
Cognitive Revolution "How AI Changes Everything"· 2024年12月21日播客Frontier Models Hit the 10th–40th Percentile on Real AI R&D主持 Nathan LabenzNeev Parikh of METR walks Nathan Labenz through REBench, a seven-task test of whether agents can do machine learning research—and what happened when one of them decided not to train.评估与可观测性评估Benchmarks+3
1:23:37EN → 简体中文
Cognitive Revolution "How AI Changes Everything"· 2024年10月23日播客前十名研究想法里有九个出自 Claude主持 Nathan Labenz斯坦福大学博士生 Chenglei Si 做了一项盲测,让 Claude 3.5 Sonnet 与一百多位自然语言处理(NLP)研究者比拼新颖的研究想法。评估与可观测性评估Benchmarks+3
38:28EN
No Priors: AI, Machine Learning, Tech, & Startups· 2024年10月8日播客 · 2 人对谈Half of Production AI Is RAG. Fine-Tuning Almost Disappeared.主持 Sarah Guo, Elad GilBraintrust CEO Ankur Goyal tells No Priors what Notion, Zapier, and Vercel actually ship — and which hyped techniques they quietly dropped.评估与可观测性评估检索增强生成+3
1:54:17EN → 简体中文
Cognitive Revolution "How AI Changes Everything"· 2024年8月28日播客Metaculus 在 2020 年 1 月就关注到了新冠,如今它要让 AI 也加入预测主持 Nathan LabenzNathan Labenz 与 Metaculus 首席执行官 Deger Turan 谈认知安全、为期一年的机器人锦标赛,以及为什么一个有用的预测,比一个准确的预测更难做到。评估与可观测性评估Benchmarks+3
1:56:03EN → 简体中文
Cognitive Revolution "How AI Changes Everything"· 2024年7月13日播客人类接近满分,最强 AI 仍只拿到 39%主持 Nathan LabenzZapier 联合创始人 Mike Knoop 和 François Chollet 为 ARC-AGI 拿出了 100 万美元奖金。ARC-AGI 是 2019 年提出的、考察样本效率的智能测试,在每题一美分的算力预算下,语言模型至今仍无法破解。评估与可观测性通用人工智能Benchmarks+3
55:13EN
Sequoia Capital· 2024年7月2日播客The Best AI Scores 39% on This Test. Zapier’s Co-Founder Put $1 Million on Beating ItFrançois Chollet designed ARC to measure skill-acquisition efficiency; five years later humans still crush it and frontier models do not.评估与可观测性BenchmarksAGI Timelines+3
43:45EN
No Priors: AI, Machine Learning, Tech, & Startups· 2023年8月3日播客 · 2 人对谈Eight Years of No Growth, Then OpenAI and NVIDIA Became His Customers主持 Sarah Guo, Elad GilWeights & Biases CEO Lukas Biewald tells No Priors how a labeling company with no chasm to cross funded the ML developer tools that trained most of the large language models.评估与可观测性可观测性与可靠性开发流程与测试+3
44:19EN
No Priors: AI, Machine Learning, Tech, & Startups· 2023年6月15日播客 · 2 人对谈AI Will Write Ten Times More Code. Someone Still Has to Keep It Running.主持 Sarah Guo, Elad GilDatadog co-founder Olivier Pomel on why faster generation moves value from writing software to understanding, securing, and operating it.评估与可观测性可观测性与可靠性Developer productivity+3
53:37EN
No Priors: AI, Machine Learning, Tech, & Startups· 2023年3月9日播客 · 2 人对谈30 models, 42 scenarios, 7 metrics: Percy Liang's public checkup for language models主持 Sarah Guo, Elad GilA PubMed-only model once led the U.S. medical licensing exam — until a system roughly 200 times larger overtook it.评估与可观测性AI researchFoundation Models+3
2:34:20EN → 简体中文
Lex Fridman· 2020年8月30日播客 · 2 人对谈智能,是你习得未曾准备过的技能的效率嘉宾 François Chollet · 主持 Lex FridmanFrançois Chollet 对 Lex Fridman 说,技能不等于智能,GPT-3 仍然只是在做模式匹配,而一种名为 ARC 的网格谜题,至今仍能把孩子和机器区分开。评估与可观测性BenchmarksAI research+3
1:07:08EN → 简体中文
Lenny's Podcast播客 · 2 人对谈模型就是产品,评估就是需求文档:Mercor如何长成史上增长最快的公司嘉宾 Brendan Foody · 主持 Lenny RachitskyMercor CEO Brendan Foody解释,为什么AI实验室几乎愿为专家写的评估付任何价钱,公司如何在16个月内从100万美元做到4亿美元年化,以及对工作和创业者意味着什么。评估与可观测性评估Data quality & curation+3
1:16:17EN → 简体中文
Lenny's Podcast播客 · 2 人对谈更快,可是为了什么?Nicole Forsgren 谈 AI 时代如何衡量开发者生产力嘉宾 Nicole Forsgren · 主持 Lenny RachitskyDORA 和 SPACE 的提出者解释为什么多数生产力指标会骗人,AI 如何改写心流与信任,以及怎样把开发者体验做成能证明价值的项目。评估与可观测性Developer productivity评估+3
1:07:48EN → 简体中文
Lenny's Podcast播客大多数生产力指标都是谎言:Nicole Forsgren 谈 AI 时代如何衡量开发者体验主持 Lenny RachitskyDORA 与 SPACE 的提出者解释,为何代码行数和旧流水线指标在 AI 时代不够用,以及怎样从倾听、问卷开始改进开发者体验,并把结果接到领导真正关心的事上。评估与可观测性Developer productivity开发流程与测试+3
1:46:33EN → 简体中文
Lenny's Podcast播客 · 3 人对谈从一团乱到失败地图:Hamel 和 Shreya 手把手讲 AI 评估嘉宾 Hamel Husain, Shreya Shankar · 主持 Lenny Rachitsky两位评估课讲师用一套公寓租赁助手的真实日志,带 Lenny 走完从读 traces、聚类失败模式到校验 LLM 裁判的全流程。评估与可观测性评估AI 工程管理+3