After Agents Leak Tokens and Wipe Databases, Onyx Trains Small Models to Watch the Big Ones

Sarah Guo opens No Priors with Maxim Bar Kogan, co-founder and CEO of Onyx Security, an Israel-based startup of researchers, mathematicians, and engineers building agents to watch other AI agents. The conversation covers specialized model training, Mythos-class models, alignment research, and Israel's security-and-now-AI ecosystem. Two years earlier the consensus risk story was DLP for chatbots: what are employees pasting into ChatGPT.

That is not the market anymore. Guo says concern is not quite panic, but close to market-wide panic, and much higher than a few months ago. The cold open is already the product thesis: as companies do exponentially more with AIs, really bad actions start to happen. Agents have accidentally published code and tokens they were not supposed to. Enterprises cannot stop the adoption. They can only try to cut the chance those actions are illegitimate or wrong.

The company is still under a hundred people. In theory a Fortune 10 or Fortune 20 has no reason to work with a two-year-old shop. The door opened, Bar Kogan later says, because the pain is daily. Buyers saw the company come out of stealth and called. Security people, in his phrase, are in the business of revenue preservation. Between two risks, they would rather partner with a promising early company than do nothing.

Guo asks why he bet on agent actions at the start. The pivot, he says, was AutoGPT — the first really autonomous agent running on LLMs, as far as he knows. Instead of generating text, the model decided what to do, got an API or a tool, did the thing, and looped. In theory it could do anything a person could do on a computer. In practice it did not work well. The models were not good enough. GPT-4 was not good enough. It still gave everyone a glimpse: if the models got good enough, that same structure would produce capable agents doing work for us.

Claude Code today, he says, is not dissimilar to AutoGPT then. They were early, before the models were ready, but the concept was right. The thought that stuck was more personal. He was already AGI-pilled. Models were going to be way smarter than us. How do you oversee agents that are smarter and more capable than their operators — especially on the day they are managing the water supply, electricity, the power grid? How do you feel easy about that? He was obsessed with the question, and he was too early.

Enterprises were not using agents. Security buyers told him he was way too early, that this was not going to happen. Guo had asked whether anyone would pay before he ran out of money. He admits there was a real chance he would have. Then the market arrived: reasoning models that could do long-horizon tasks, Claude Code as the first widely used autonomous agent, then Cowork and OpenClaw.

There was a detour. Everyone was afraid to build fully autonomous agents, so the industry built limited low-code platforms based on connectors. Those platforms did not deliver the productivity gains. The unleashed agents did — with far fewer controls baked in — and even very large enterprises adopted them. A large share of Anthropic's revenue, he notes, now comes from enterprises paying for Claude Code to do work developers used to do. Autonomous agents showed up before it was too late. That, he says, was luck.

Guo wants a one-liner on the product and a picture of deployment. Onyx does two things. First, it trains models and builds agents that can oversee other agents, because the number of AI actions is going exponential and a human in the loop will not survive 100x, 1,000x, or a million-x. Second, it productizes that into a control plane — a secure control plane — that finds all of a company's AIs and autonomous agents and hooks them up so someone is watching.

The point of watching is not academic. Downtimes have already been caused by an AI doing the wrong thing. Agents have published code and tokens they should not have. Guo, for her part, has already been over-permissive: an agent deleted data permanently and created rework. She wants guardian spirits. Enterprises, Bar Kogan says, now understand the risk is growing exponentially and that they have no way to stop adoption. They have to reduce the chance actions are illegitimate or incorrect.

He splits a typical large-enterprise estate into three buckets. SaaS low-code platforms where people drag and drop automations that are not really autonomous. First-party agents built in the company's cloud, as internal apps or even customer products. And very autonomous coding agents and assistants. Right now, in the average enterprise he sees, more than 50% is the autonomous coding slice, about 45% is low-code automation, and about 2% is first-party — because building effective agents is hard and buying them, or assembling them in low-code, is easy.

The autonomous slice is also the fastest growing. Claude Code spread like fire among developers in the customer base; Claude Cowork is growing even faster. To Onyx's own surprise, some companies are sanctioning OpenClaw as a legitimate internal tool because the CEO is driven to adopt AI. That fastest-growing category, he says, typically arrives today without any controls.

Enterprises already buy on the order of a hundred billion dollars of security a year — endpoint, network, cloud, identity. Guo asks what of that is relevant to agents, or whether none of it is. Security always has overlap and defense in depth, Bar Kogan says. In this space a lot of enterprises are still helpless. Take identity. The classic move is to limit a system's permissions so that even if it is compromised it cannot do much. Autonomous assistants break that. You want Claude Code or Cowork to have your permissions, you want to give it wildly diverse tasks, and you want to go to lunch and come back to finished work. There is no tidy permission set. Identity software suddenly is not very useful.

Endpoint and API security have the same hole. If you tell Claude Code to delete a database and recreate it, that is a feature. It can save DevOps and platform teams a lot of time. If Claude is working on an unrelated task and suddenly thinks the right move is to delete and recreate the database, you do not want that. Endpoint providers and API tools do not know what Claude was thinking, or why it is doing what it is doing. They do not have the context for flexible, unpredictable systems.

Without controls built for these systems, he says, you either limit the agents until they are much less useful or you miss a lot of pretty dangerous things. Guo's traditional instinct is a proxy plus a policy engine: write rules, make the rules smarter. Bar Kogan treats a proxy as an integration method, useful when it is the easiest hook. A lot of AI runs on someone else's cloud or on an endpoint, so a proxy is often not even technically viable. Seeing the traffic is not the hard problem anyway.

The hard problem is the engine that underwrites each action: is this okay or not? You have to understand what another system is thinking, what it is planning to do, and then have your own opinion. The systems in question are some of the smartest models in the world. Who are we to judge them, and how do we do it correctly? That, he says, is a really difficult technical question — and it is why a packet filter with cleverer rules does not close the loop.

A listener's first idea, Bar Kogan says, might be to ask Claude Code to do the watching: spawn a copy for every agent, monitor everything, intervene if there is a problem. In a sense that is right. Capable agents reviewing other agents is as intuitive as capable humans reviewing other humans. The approach is also naive. If a security vendor has to run an agent for every agent the customer runs, the customer will pay more for the overseer than for the AI, and the latency will be intolerable. That is a dealbreaker.

The real challenge is knowing when to interject with those smart agents. Onyx's answer is not very smart models. It is models that are good at one thing. They are very small. They can almost do nothing else except say: should a smarter agent look at this? If you bake that intuition in — they do not miss much, and they do not call the other agent too often — you get performance, smart oversight when needed, low cost, and low latency.

That becomes the company's hard problem, because frontier models keep getting smarter and harnesses more evolved. The small models on Onyx's side have to keep being able to say: this is the action where someone should look closer. Most of the hard things the company is doing live in that space. Guo offers a blitz-chess analogy. Neither of them is beating Magnus Carlsen in a classical game, but enough timed games teach a different skill: intuitive decisions under time pressure.

He had not thought of it that way, and then he runs with it. Top chess players make most moves on intuition. They have seen so many games that they know a move is not too risky without calculating. Every so often they stop for a long time, because this is a critical position and they need to calculate forward. That is the efficient way to run computation: do not spend intelligence where you do not have to, and spend an overwhelming amount of it where the risk is high.

Onyx is mostly based in Israel. Guo notes the well-known pipeline of military and offensive-security talent, then the slightly different DNA here: co-founder Gil came out of synthetic data at NVIDIA. Bar Kogan says Israel started late in AI and is catching up fast — world models, AI infrastructure, chips. The company mixes cyber and AI in a way that reflects both founders. Most of the research engineers come from a unit in Israeli intelligence that sits at the intersection of math and cyber.

That mix matters because they do not want to be just a security company long term. Controlling advanced AI, on first principles, sounds important even if you forget enterprise gaps. If AI companies become $10 trillion companies, he thinks you want a company that is not the AI vendor itself to oversee and help you control what the AI is doing. He calls that a hundred-billion-plus opening. Scratching the surface of that job means understanding much better what models are thinking — what is happening in the internals as they operate. A lot of Onyx research is aimed there.

Guo says the industry is divided on whether mechanistic interpretability is even possible. Bar Kogan believes there has been strong progress, and that understanding internal weights, activations, and mathematical structure will be at least part of the solution. Human intelligence may still find it very difficult to read those internals quickly. His twist: as models get much smarter than us, at least in some important ways, they may help crack mechanistic interpretability more effectively. The payoff is not only oversight. It is understanding what intelligence is, and what the difference is between a smarter model and a less smart one.

Trust is the immediate objection. Why would a Fortune 100 give a sub-100-person company the same permissions and visibility it gives the labs? He agrees it should not be possible. It happens when the pain is strong enough that a CISO will call a company that just came out of stealth. Buyers also believe a huge company will exist in this space and they want the right horse. Doing nothing, they think, will disable the business in a short time.

Every CISO and engineering leader Guo knows is also freaking out about the plummeting cost of finding vulnerabilities with coding tools. Compromises have already followed. Ten years ago, Bar Kogan says, automated vulnerability research looked like a 20-to-50-year dream. Maybe that was Israeli intelligence people patting themselves on the shoulder about how hard the job was. Suddenly it is coming all at once. The market is not overreacting. For a pragmatic security person the play is two-layer: the fastest patches and mitigating controls for the holes already found, then the foundational pieces every large-enterprise security leader already knows — identity locked down, a firewall, endpoint detection, the right primitive for each asset class.

The AI attack surface, in this telling, is missing its foundational layer. That is the role Onyx claims. Guo asks about Anthropic's Glasswing and OpenAI's Daybreak, the phased, controlled rollouts of more dangerous capability. He does not have a strong opinion, only a fork. If nobody is about to release a Mythos-level model, slow is great: time to build know-how, playbooks, community, and to avoid airlines and power plants going down. If anyone gets there first — a Chinese Mythos-level model, in his example — then in retrospect the slow rollout was a huge mistake, because companies had no chance to move and nothing they can do.

He would expand access so more companies can prepare. His advice to everyone else is simpler: assume these models are coming anyway. The only thing you can do right now is invest in foundational controls that stop the downstream effects once the vulnerabilities are found. On holdouts, he hardly sees large enterprises still banning everything until it is safe. Finance is more opinionated — agents yes, but maybe only two tools. He thinks the companies that do well will allow a lot of different tools, because the landscape changes so fast. Betting on OpenAI a year earlier was the safest bet in the world; then Anthropic had better models and tools.

There is a price to paying for optionality, and a large company's risk profile should be different. A startup wants agents to do everything, because it has everything to gain and nothing to lose. JPMorgan has so much to lose that it can take more time — and, he adds, JPMorgan is adopting AI very quickly. Nuance at scale is fine. For Onyx's own pace, he feels lucky that the two core pillars of 2026 AI have not changed: LLM foundation models, and agents as an LLM deciding tool calls. That lets the company cover many applications with a lean core. 2027 he holds as strong opinions, loosely held, until AGI and ASI take a shape.

Could the labs just do this themselves as they get hungrier? Bar Kogan offers buyer psychology first. If you are buying a car, you do not have the same guy certify that the car is good. A security team will not trust a product vendor to say the product will not mess up the environment. They want an independent party whose whole business depends on being right that an action is legitimate.

Then he splits why models and agents make mistakes. Jagged intelligence produces silly errors. That class, he thinks, will go away as models get smarter, and vendors are fully incentivized to fix it. Onyx's job is not catching stupidity. The fast-growing class is different: the model is not being silly. It has a semi-independent, even semi-aware perspective on what should happen, and that perspective may not align with yours. The problem has grown hand in hand with capability. As you get smarter you have more independent thoughts. That one looks hard even for the large vendors.

Onyx has an information advantage the labs are not granted. It is allowed to look at a lot of historical agent behavior. Enterprises today will not let Anthropic or OpenAI keep that history, because they know those companies will want to train on it. Context about how an agent typically behaves is exactly what you need to see that something is wrong. The last structural reason is a multi-vendor world: cheap open-source models, different models for different tasks and cost profiles. It is unrealistic to expect every vendor to provide the same security, especially the new ones you adopt in a hurry.

Guo asks what Silicon Valley is missing, given Israel's adversarial bench — 8200, Wiz, Armis, Island, NSO Group. Bar Kogan's answer is unromantic. Security products fail when they solve a technical problem and nobody loves them. You have to know what a security person opens with morning coffee, what the boss wants, what they get praised and blamed for, and put that in the product. He is also the most AGI-pilled person Guo expects to meet in Israel, and he still thinks security teams persist for years, increasingly run by agents. Sell to humans with a few agents today; make the system convenient for agents as the buyer mix flips. For a human, do not overwhelm with irrelevant information. For an agent, do not waste tokens in context. He says that might be the same thing.