AI and the Future of Vulnerability Research

AI coding tools can help find vulnerabilities as well as write software. I worry that faster, cheaper investigation could change the balance between attackers and defenders. This video was recorded before the Mythos announcement; the examples and predictions reflect what I had seen at that time.

Some public examples involve researchers and labs reporting flaws so they can be fixed. That is encouraging, but it does not tell us how often attackers are finding or using similar issues privately. Vulnerability discovery, a working exploit and an actual compromise are distinct steps.

My concern grew at DEF CON, where I saw both cryptography puzzles and security competitions. A puzzle can test persistence and reasoning, but solving one is not the same as compromising a real system. I use the stories as signals of capability, not a direct measure of offensive success.

Gold Bug presents unusual puzzles requiring a hidden phrase. One challenge, Smugglers Manifest, occupied me for two days and involved reasoning through a fictional shipping document. At the time, the AI tools I tried did not help much; later models got further, sometimes with a little guidance.

Another puzzle, Sea Shanty, had bottles with phrases and a poem. My team spent a long time on it. I report that GPT-5.4 Pro later solved it in roughly 16 minutes, including time checking its answer. This is my account of a particular test, not a controlled benchmark of hacking ability.

I also recall organizers saying AI had contributed meaningfully to a DEF CON capture-the-flag competition. I could not find the relevant closing-ceremony recording or exact figures, so this remains my recollection rather than independently documented evidence. Competition environments differ from production systems.

In a hotel room, a security expert I knew tested an early GPT-5 model on an obscure Windows issue. The response appeared useful enough to surprise him, although it did not find or complete the issue. Later attempts failed to reproduce the behavior and he became less impressed; that uncertainty matters.

I discuss reports about how providers route or restrict some security-related requests. Those policies and model behavior can change, and my description is not evidence of a universal routing rule. The wider issue is how to enable legitimate security work while managing the risk of misuse.

I cite Anthropic's work with Mozilla and its reports of vulnerabilities found during testing. I then turn to security researcher Thomas Ptacek's essay on how agents may change the economics of vulnerability research. His claims about what will happen next are forecasts, even where they are grounded in real examples.

Ptacek argues that AI may produce a flood of discovered vulnerabilities, not only vulnerabilities introduced by AI-written code. If a tool can investigate many projects quickly, a larger number of previously unknown issues may reach maintainers. Whether that happens at the scale and speed he predicts remains uncertain.

The essay describes a shift from specialized human effort toward more parallelizable investigation. I worry that many systems contain unknown flaws that have not yet been found or fixed. That is a reason to strengthen review and response, not proof that every particular system is exploitable.

Ptacek draws on the history of exploit research, where practitioners learned obscure implementation details because they were exposed to untrusted input. His point is that weaknesses can appear far from an application's obvious authentication or security code. Understanding the whole path from input to processing matters.

A second part of his argument is that limited expert attention has historically influenced which components receive scrutiny. If agents make analysis cheaper, neglected devices and applications could receive much more attention from both defenders and attackers. This is an argument about changing incentives, not a claim that sound defenses never mattered.

I ask a model about relationships among Linux subsystems without web search to illustrate the breadth of information it can draw on. I am not a kernel specialist, and a plausible answer is not automatically correct. Domain experts still need to check claims against source code and evidence.

Ptacek argues that agents can repeatedly compare patterns, inspect code and test hypotheses without getting tired. This can speed up parts of research, but tests, tools, compute, permissions and expert judgment still matter. Finding a suspicious pattern is not the same as proving impact.

Anthropic reported more than 500 high-severity vulnerabilities found and validated during its open-source security work with Opus 4.6. Its report says findings were extensively validated, and human researchers checked initial bugs and wrote patches. The number is a report of validated vulnerabilities, not a count of unattended attacks.

Ptacek describes Nicholas Carlini using repeated agent runs across source files and then reviewing candidate reports. He reports strong results for that workflow, but the quoted near-100% figure lacks enough detail in the discussion to treat it as a general accuracy or success rate. Human validation and responsible disclosure remain essential.

The essay describes a web application flaw as an example of a finding that goes beyond memory corruption. The lesson is that established code and familiar components can still contain defects. A report needs reproducible evidence and coordinated remediation rather than confidence based only on a model's explanation.

Ptacek predicts that more targets will become economical to investigate, including devices that are difficult to update. I agree that slow patching creates risk. Keeping supported systems updated and planning for devices that cannot be updated can reduce exposure, although no single measure guarantees safety.

Open-source maintainers already face low-quality automated reports. They may also face more valid reports that require triage and fixes. Both false positives and real findings create work, so researchers should validate reports and coordinate with maintainers rather than simply increase volume.

The essay raises memory-safe languages, sandboxing and attack-surface reduction as existing defenses. It predicts that agents will eventually make chained attacks easier, but that is a forecast. Layered defenses still increase attacker effort, and the effectiveness of automated research against them requires measurement.

Ptacek also argues that keeping source closed is not a complete defense, because binaries can be analyzed. Openness and secrecy can change the effort required for different actors, but neither eliminates vulnerabilities. Organizations need patching, isolation and monitoring regardless of source availability.

A further worry is regulation that accidentally burdens legitimate vulnerability research while doing little to stop misuse. Ptacek expects political pressure after damaging incidents, while acknowledging disagreement among people who work on security policy. This is a concern about possible future rules, not a statement of current law.

One researcher Ptacek consulted believes exploit chains will remain difficult because of layered defenses such as sandboxes and virtualization, and expects specialized analysis tools to retain value. Another sees room for human researchers at the highest end, particularly where techniques are not documented. These are meaningful qualifications to the more sweeping forecast.

Ptacek expects agents to displace much human vulnerability research; I find his warning compelling and unsettling. The outcome is not settled, and the stories here do not prove that every system can be compromised by AI. What is clear is the need to support defenders, validate findings and get fixes to users.