One Attacker and a For Loop: Why Email Is Still the Softest Target in an AI Arms Race

Material Security co-founder Ryan Noon on how 2016 election hacks, Google and Microsoft's email duopoly, and large language models are rewriting both sides of cybersecurity.

SGSarah Guo진행자 · Conviction
채널
No Priors
게시일
2023년 10월 26일
영상
36:22
읽기
12분
원문
영어
YouTube에서 열기
스크롤하면 영상이 오른쪽 아래로 작아져 계속 재생됩니다. 자막은 읽기 언어를 따릅니다.
읽기 언어2개 언어로 읽을 수 있음
이 글은 아직 한국어 번역이 없어 영어 원문으로 표시합니다.
요약

30초 요약

On No Priors, Ryan Noon tells Elad Gil and Sarah Guo that Material Security began as a Gmail protection trick after the 2016 Podesta leak and became a bet on defense in depth: assume the attacker is already inside the inbox. He argues LLMs are a step-function for both defenders (alert triage at a penny and a half) and attackers (one person supervising a thousand phishing campaigns), and that open societies have waved their soft underbelly at the internet. The rest of the conversation is a founder's tour of the cybersecurity industrial complex, why platforms leave security gaps, and why startups should chew off-the-shelf models before pretending they have a fine-tuning moat.

  1. 1Material's operating idea is defense in depth: assume the attacker already got over the wall, then put another wall and a machine gun behind it. The company used to call itself seatbelts for email.
  2. 2The industry's long-running 'skills shortage' has a blunt answer: an LLM is about 90% of a human for a penny and a half, and it is already good at turning noisy alerts into something an analyst can judge.
  3. 3Noon says the biggest new threat is not exotic voice clones. If attackers merely spell-checked their phishing, overall cyber risk would drop in a material way, because so much of security is still text and humans trusting humans through computers.
  4. 4At the nation-state layer, his claim is sharper than the usual nuclear metaphor: intelligence itself is now a commodity that can be arms-raced, a Red Queen contest where stopping means falling behind.
  5. 5Most officials still see models through last year's ChatGPT headlines and worry they 'make stuff up.' People have not yet realized that computers can think in a far more salient way than they ever could. Sarah Guo adds that shovel-selling is overhyped while serious talk about society in ten years is underhyped.
  6. 6Every discontinuity births new companies and makes a whole class of products obsoleteNoon once passed on Snowflake's Series B because he thought AWS would bundle security; AWS later told him tenants do not own the CPUs, so the platform can afford to be lazy.
전문

챕터별로 다듬은 전문 읽기

원본 자막만 보기
01

A dumped Gmail inbox in 2016 was enough to start a unicorn

Elad Gil introduces Ryan Noon as co-founder and chairman of Material Security, the company trying to make cloud email a safe place for sensitive data. Noon had previously started Parastructure, which Dropbox acquired, and then worked there as an engineering manager. Gil notes that Material was unusually fast to put early LLM APIs into production, but he first wants the origin story.

Noon dates the company to 2016 or 2017. He had left Dropbox, was living in Europe, and watched that year's election hacking with what he dryly calls love: ordinary Gmail accounts kept getting dumped onto the internet. He sketched a novel way to protect a personal Gmail account, coded it, and was shocked that the Gmail API actually let him do it. Friends back home saw a special case of a broader idea. Seven years later, he says, it is 'a cybersecurity unicorn thing' and the team works with some of the most interesting companies in the world.

02

A Google–Microsoft duopoly makes email a world-altering single point of failure

The product thesis is that modern work and private life hang off Google and Microsoft accounts. Email was the starting point; Material then pushed deeper into everything those accounts unlock. The commercial core is mid-size and larger companies running Google Workspace and Office 365, with modules aimed at the worries that actually keep security teams up at night.

The first big module hunts years of sensitive files sitting in inboxes and archives, redacts them, and replaces them with a clean copy. If an attacker downloads the whole mailbox, they get nothing useful; if the owner needs the original, a button plus Face ID, Touch ID, or a stricter work policy unlocks it. The company later added anti-phishing and account-takeover protection for the things attackers do after they are already in, such as resetting bank credentials.

The operating concept is defense in depth: assume the bad guy got over the wall, then ask what they want, and put another wall and a machine gun behind it. Material used to brand itself as seatbelts for email. Crashing the car is bad enough; going through the windshield is worse. Gil ties the founding impulse to the Podesta emails and Hillary Clinton's campaign. Noon agrees it matters, then points at the structural problem: Google and Microsoft have a total duopoly, so whatever little thing they miss is world-altering. Every couple of months there is a headline that another cabinet secretary's email was hacked because all the eggs sat in Microsoft's basket. Duopolies are stable in the market and extremely fragile in security. Material exists to fill the doors they leave open.

03

Feed one internet to a million GPUs and it learns some cybersecurity

Gil asks how Material started using generative models. Noon's answer is that the second you give a coder a REPL, they start iterating, and ChatGPT was a good-enough REPL. Playing with it, they found a lot of security domain knowledge already baked in. 'If you feed precisely one internet to precisely a million GPUs, it [picks] up a thing or two about cybersecurity.' The analogy he used at the time was Shang Tsung from Mortal Kombat: the model has eaten the souls of thousands of security engineers, so you might as well use it. The first production use case was operational: rarify the signal, filter the noise, then hand it to a human who knows what attackers are trying to do. LLMs turned out to be fantastic at that.

He is unimpressed by startups that fine-tune mainly to pretend they have a moat. As an engineer, he says he knows when he has headroom, and even GPT-3.5 had plenty. Things that were very hard for computers eighteen months earlier are easy for off-the-shelf models. His advice to the industry is to chew your food first. Offensive uses are what you are not supposed to talk about, but the bad guys are talking, and security has always had an arms-race quality, so the world needs security LLM companies because attackers exist.

The cliché he keeps hearing from founders is the cybersecurity skills shortage, as if America should bring back the draft and issue everyone a certificate. His zeroth-order answer is simpler: you already have 90% of a human for a penny and a half, so start there and go to Disney World after the boring work is done. The best security work, he adds, does not come from the vendor buying the CISO a steak and a Ferrari in Vegas every August. It comes from a strong engineer hacking something together. He points to Socket analyzing npm dependencies and hunting secrets dropped in codebases, messy text problems where models are already pretty good at reading code.

04

The attack does not need voice cloning when a for loop will do

Asked for the biggest cyber threat from this technology, Noon says he is talking about text models, because so much of cybersecurity is just text. There are nasty reported hacks where a voice was faked convincingly on a phone call, and APIs from shops such as ElevenLabs and LMNT make that easier. Gil notes the classic CEO-text-to-a-new-employee scam. Noon’s sharper point is that humans trust other humans through computers, and that was the key mistake. The attacks random criminals already send are not even grammatical. If all you did was spell-check the bad guys with an off-the-shelf open-source model, it would still make a material difference to cyber risk.

On how bad this gets by a hypothetical GPT-6, the analogy that helped in a conversation with someone from the White House was Bronze Age versus Iron Age. If the neighboring tribe has iron and you still have bronze, you are going to have a bad time. Talk of airstriking data centers or restricting 'super high-grade carbon steel from space' misses the point if iron is already aimed at bronze armor. Models are a step function, the flying-car kind of discontinuity people used to complain never arrived. That does not mean everyone is doomed, but it does mean the cost curve has moved.

Scope is hard to feel in this industry. A Fortune 500 can leave the front door open for a year, nobody walks in, and executives conclude hackers are fake. A no-name company gets run over and concludes the barbarians are at the gate. What is new is supervision: one human can now oversee a thousand humans. You no longer need a room full of jerks trying to hack grandma. One jerk and a for loop will suffice.

05

Open societies have waved their soft underbelly at the internet

Gil asks whether state actors and attacks on physical infrastructure should change how the West invests in LLMs. Noon’s moral basis for the whole field is unromantic: cybersecurity is the waste heat of every other computing innovation. Utility drives adoption; safety never does. Welcome to Earth. The industry is the cleanup crew. The world will take the new capability first and notice later that it messes up democracy.

He does not need hyperbolic atom-bomb language. Intelligence is now a commodity you can arms-race, a Red Queen contest like nuclear power except the thing being raced is intelligence itself. Gil notes that OpenAI’s original premise was a counterweight so Google and a few others would not monopolize the capability. Noon’s follow-on is that the West is especially exposed. Open societies put everything out, let the private sector run it, adopt every digital technology early, and are happy to wave a soft underbelly on the internet. A totalitarian state can mandate a Linux distro, lock down social media, and make 'what if I get phished' a non-question because that is not how bank accounts work there. If you can lock information, you can lock the attack surface, including elections.

What still baffles him is that cyberspace is a formal U.S. military command, William Gibson would be proud, and yet an ordinary American in that space is basically on their own. The military guards physical borders; nobody guards your inbox. He thinks the Department of Defense does impressive offensive work and that deterrence often works, and he is not taking a side on Snowden. On LLMs specifically, though, everyone is still figuring out what is going on. DARPA has interesting projects, but most people see the technology through New York Times ChatGPT coverage or a free-tier session six months ago. The worries are order-zero: make sure it does not make stuff up. People have yet to realize the computer can think in a much more salient way than it ever could. Guo adds that shovel-selling is overhyped and thoughtful talk about society in ten years is underhyped.

06

Incumbents harvest startups instead of planting, and platforms can afford to be lazy

Gil asks whether there is room for startups when security incumbents are so good at buying, bundling, and cross-selling. Noon says there is always room, then offers the cynical version. Required reading at Material, which he never actually enforces, is Ian Grigg’s paper 'The Market for Silver Bullets.' Buyers, sellers, and attackers share a market in which the buyer cannot be sure the product will save them. The truism is that every cybersecurity product is a complex insurance policy. That mushiness produced incredible distribution machines: Cisco, Palo Alto Networks, and to an extent Microsoft and Google, with sales forces, bundles, and glitzy conferences.

Their product managers, if honest, do not know what attackers will do in five years any better than anyone else. So why plant seeds when you can harvest crops that are already growing, transplant them into your yard, and water them with salespeople and market power? Cisco had just bought Splunk, and Splunk buys things; the whole market works that way. Noon once wrote an angry blog post calling it the cybersecurity industrial complex: PE firms dressed up as innovators. Creative destruction still happens, and some of the best security companies did not exist ten years ago, but many top VCs skip the category because the typical outcome is a $200 million takeout by Palo Alto. Discontinuities still breed new companies. Entire classes of product are now both necessary and obsolete. Much of the budget still goes to emitting logs, parsing twenty-year-old firewall formats, and aggregating alerts, work an LLM can eat. A lot of spend is up for grabs.

On who is actually adopting models, he sees three layers. Growth companies with nerdy founders integrated immediately. Younger public companies that still get it shipped a thinner feature later, the 'you can type in the box and typos do not matter' kind. Fortune 500s are running science projects that, he assumes, are excellent for OpenAI’s revenue. Microsoft had a top-down mandate and a year on everybody, plus a lot of marketing; he has not used the result himself. Partnerships still matter because customers hate buying a pile of point tools and also hate a big bundle that sucks. Vendors should be grown-ups and interoperate. Gil cites Material’s work with Slack around Office 365 and Google Workspace as that pattern.

Asked what has changed in seven years, Noon is almost gloomy: people still send email and still reply to texts, and a walled garden like Slack just becomes another hop in the same chain. Mobile is the tragic case. Apple locked devices down so hard that vendors can do very little, then sat on the remaining problems. Tech is better at keeping people out of its territory than at using that territory. Cloud centralization is the other unfinished story. His biggest miss as an investor was assuming AWS would bundle security away. He diligenced Snowflake’s Series B and told whoever asked him to pass, because Redshift existed and AWS was not asleep. AWS later told him they get paid either way: tenants do not own any CPUs, so the platform can be lazy. The gap is the business.

07

Build something everyone should buy, then chew your food

Founders in security and broader enterprise keep asking Noon what to build. His love-hate advice is to stop grinding a beachhead around a mediocre product that happens to be easy to sell because the category is mushy. Take a step back and build an incredibly useful thing that everyone should buy. Stop mapping CASB, UEBA, SIEM, and NDR onto a banker’s quadrant. Consumer people already make fun of enterprise diagrams; cybersecurity’s are extra mockable. If it talks to an API everyone uses, saves a pile of time, and is easier, just build that. You have about five years before you even start paying Gartner. Market is not the same thing as marketing.

The basics still come first. Figure out the team. Being a solo founder is totally okay, and far better than 'we had three coffees and we just got married.' Everything is built on the team; the saddest failure is a beautiful company with a crack in the foundation that has to be torn down. Align on risk appetite. He notes Silicon Valley is irrationally exuberant again. The six months of gloom after free money ended were too short; he wishes they had lasted a year, RIP Good Times memes and all. Now it is back to ZIRP if you are in AI.

Pick a good market: lots of dollars, and lots of weaker people you can take those dollars from. The difficulty of starting a company is the inverse of market size; play on easy if you possibly can. Guo says that is the advice she gives AI founders too: in an early industry, do the easy, high-velocity work, and save the hard problems for when the market is mature. Noon’s other pet peeve is founding teams without a real technologist. Leave MBAs alone and they will ship Casper mattresses, but for mattress pads, with razors. The best companies have a technologist somewhere, if not in the CEO seat. Technologists, though, cluster in recruiting, DevOps, and yet another monitoring tool. Get out of the house, at least on Zoom, and learn a market that is not your own. The world needs that creative energy, to paraphrase an old Dropbox slogan, but combinatorics will always invent another 'Datadog for AI, but mobile testing.' Ask how big the actual market is.

같은 채널의 다른 글

39:23EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026년 10월 15일팟캐스트 · 2명The Defenders' Window: Greg Brockman on AI-Generated Knowledge, Drug Discovery, and the New Physics of Cybersecurity진행자 Sarah Guo, Elad GilOpenAI's president explains why solving open math problems is a proof point for science, how proactive agents change product form, and why security spending must exceed software spending.보안과 안전AI security안전과 거버넌스+3
1:09:44EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026년 10월 9일팟캐스트 · 2명10,000 GB300s for Four Weeks: Reflection’s Open-Model Ledger진행자 Sarah Guo, Elad GilMisha Laskin tells No Priors how Reflection trained Beam, why reinforcement learning outspent pre-training, and why the West needs its own open weights.오픈소스와 커뮤니티Open-weight modelsReinforcement Learning+3
35:39EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026년 10월 2일팟캐스트 · 2명Compute Scaled a Millionfold. Bandwidth Only 40x. Fractile Wants to Fix That Ratio진행자 Sarah Guo, Elad GilIn the summer of 2022, Walter Goodwin founded Fractile in London to build very fast inference chips for the world's largest models.인프라와 배포InferenceAI cost & performance+3
42:44EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026년 9월 24일팟캐스트 · 2명They Paid $7.7 Billion for a Broker — and Intend to Do One Deal a Year진행자 Sarah Guo, Elad GilSequence Holdings co-founder Michael Lee tells No Priors why owning a market leader, not selling it software, is how AI actually remakes an incumbent.기업과 전략엔터프라이즈 AIAI Strategy+3
38:14EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026년 9월 18일팟캐스트 · 2명Diffusion Will Win Inference: Stefano Ermon Bets Parallel Tokens Beat Custom Silicon진행자 Sarah Guo, Elad GilThe Stanford diffusion pioneer tells No Priors why a 50-person lab is training language models that generate many tokens at once — and why that maps to GPUs the way transformers once mapped to training.모델과 학습InferenceAI cost & performance+3
45:10EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026년 9월 10일팟캐스트 · 2명76% of Agentic Commerce Is Under 30 Cents — So Coinbase Is Banking the AIs진행자 Sarah Guo, Elad GilBrian Armstrong tells Elad Gil how Coinbase became an everything exchange, why card networks cannot price agent-to-agent payments, and what his longevity company is taking into the clinic.에이전트Agentic commerce금융 AI+3