A transformer co-author wants a $160 million model no company owns

Nathan Labenz introduces Illia Polosukhin as founder of NEAR, a project that calls itself the blockchain for AI, and as one of eight co-authors of the 2017 paper "Attention Is All You Need." The paper's credits say Polosukhin and another co-author designed and implemented the first transformer models and were crucially involved in every aspect of the work. With that pedigree, Labenz notes, he could probably command one of those mythical billion-dollar packages from Mark Zuckerberg's superintelligence lab. He is building infrastructure for user-owned, privacy-preserving AI instead.

Polosukhin's path into deep learning started with the famous cat-neuron paper: an autoencoder trained on internet images learned a unit that, when activated, showed a cat. Networks worked. He thought knowledge and intelligence lived in text, not pictures, so he joined Google to put deep learning on language. The straightest test of intelligence, human or machine, is to ask questions. His team shipped short answers on google.com years before Gemini. The bottleneck was LSTMs reading one word at a time. Feed them a stack of search-result articles and they take forever. One motivation for transformers was to consume that context as much in parallel as possible, then answer.

In 2017 the slope of AI was accelerating and he wanted to build products on it. He left Google with co-founder Alex Skidanov to start NEAR AI, on the premise that machines should write code. If they can, interaction with computers stops belonging to a few people who speak to machines and becomes something everyone can do. In 2017 that sounded like science fiction. Now it is called vibe coding. They expected the 2022–23 moment to arrive on their clock, and they were short maybe an order or two of compute.

They tried to be smarter by collecting what is now called instruction fine-tuning data: people writing a bit of code for instructions, or instructions from code. The workers who could do it were computer-science students in developing countries, where a few cents adding up to a few dollars a day was real money and practice. Paying them was the wall. Students in China had WeChat Pay, not bank accounts. In Ukraine at the time, dollars arriving in a bank account had to be half-sold by law. PayPal and TransferWise did not reach some countries at all. They had heard of blockchain as a global payment network. In 2018 they looked around, decided nothing scaled, and built their own, thinking it would take six months. Mainnet launched in October 2020. NEAR, he says, now has 15 million monthly active users, runs arbitrary software in Zig, Rust, JavaScript, and Python, and is back at the original AI vision with the lessons of a chain.

Once AI writes code and talks to tools, Polosukhin argues, you start removing apps and even websites. Devices will run an AI operating system. People will want it local, but they will not process everything locally: background news summaries, and so on, have to be offloaded. Helpful AI needs all of your context. That is a scary situation if a third-party company holds the data. Chats have already leaked; some organizations have reported chats to police. Privacy, in his view, is fundamental. The assistant should have your context, stay private, and be able to execute actions on your behalf.

A lot of aggregators and middlemen look optional in that future. Google, DoorDash, Amazon, even an agency like the FDA, which takes pharmacy filings, evaluates them, and lets companies stock what is safe on average. Your AI, he says, can go to a factory in China, talk to a pharma company's AI about your exact medical situation, and look for compounds that are good for you right now, not for the average person. Society's middleman layer gives way to peer-to-peer deals among people, companies, and autonomous agents. Blockchain is the other side of that, the thing that facilitates those transactions.

If the world is seen through an AI lens, a small alteration in the model changes how you perceive it. His example: open ChatGPT, ask it to quietly convince you of something — say, voting for someone you do not like — while you talk about another topic, and the accents in the conversation start to shift. That can be an instant prompt, or it can be baked in ways you cannot see. User-owned AI means the model is yours, not theirs. Privacy is part of it. You also need to know what went into the model, and to verify at run time that nothing extra is being injected. Doing that as another company, he says, lands you in the same result. You need a movement people will contribute to as a common good, with an economic engine behind it, because the resources are expensive. On the other side sit the protocols — intents, AITP — for how your AI talks to other AIs.

Labenz starts at the foundation. Bitcoin's proof of work makes a new valid block costly; rewriting history means doing that so many times in a row that nobody gets there. It also burns about as much electricity as Poland. Proof of stake uses less compute and puts token holdings at risk. Everything they will discuss, he says, sits on validators you do not have to trust as individuals, only as a set whose incentives should not want to defect. Polosukhin reframes it from the user's chair. On today's internet you trust Riverside's DNS, certificate authorities, the organization behind the IP. You cannot verify correctness by yourself. On a chain you want to know that what you are looking at is correct, and how hard it is for someone to show you a fake.

For Bitcoin, lying about an hour of history costs on the order of $2 million. Google, in a thought experiment, could even present a days-long fake chain. Proof of stake with Byzantine fault-tolerant consensus adds finality: if two-thirds of the stake is not malicious and follows the protocol, there is no way to have a fake history. NEAR's mainnet launched in October 2020. Since then, he says, you can verify that every transition was correct. Validators are projects in the ecosystem with a vested interest, plus Vodafone, NTT, exchanges, universities, individual developers. Someone recently ran a node on an eight-year-old laptop. Joining is permissionless. Users can delegate stake to a validator they trust. Rewards are split among them.

Labenz asks about Stripe's Tempo, which plans to start with trusted validators and become more permissionless over time, and whether permissionlessness is a spectrum or a binary. Polosukhin says it is a spectrum with a few step functions. A fixed validator set can refuse to include your transactions, or your agent's. You are censored and cannot do anything. On a permissionless network you can join as a node and push transactions yourself. For ordinary people that may sound like whatever. He is from Ukraine; he has had banks close on him. For a business, if a competitor controls whose payments get included or delayed, that is not theoretical. Neutrality means no single party, and no cartel, can pull the network onto themselves.

You can show up claiming to be Coca-Cola. Nobody will believe you without a social signal — Coca-Cola's Twitter posting the validator address would do it. Identity on the network is public-key cryptography plus stake. You can be a cat. You can be an AI. If you have capital at stake, you are a validator. The stake is not at probabilistic risk if you follow the rules. If the rest of the network finds fake transactions, they take the collateral. That is slashing. So far, Polosukhin says, it has mostly been misconfigured nodes, not attacks. They slash in proportion to the stake that participated in the bad behavior, so a tiny misconfigured node loses very little. A large coordinated attack gets slashed fully, including the case where someone hacked a bunch of nodes with a zero-day. Individual mishaps are allowed; coordinated failure of security setups is not.

NEAR's market cap, Labenz notes, was recently about $7 billion. If an AI-agent economy on top of it grew toward trillions, could a coordinated attack become worth it? Polosukhin's reply is that the cost scales with the network. At $7 trillion, buying 30 percent still costs a couple of trillion, and extracting that much is another question. The system, from its own point of view, is always correct: violators are not in the system. The attack is toward someone. You fake a billion-dollar transfer, ask for a billion in services, and hope they believe it finalized. That takes about 66 percent of stake. If you are receiving a trillion, you wait. Bitcoin at roughly $2 trillion still has a one-hour attack around $2 million; send $10 million and wait longer than an hour. Five or ten dollars can be accepted immediately. The phrase "51 percent attack" is easy to misunderstand. The network is still following rules. An exchange or a service provider is the one being lied to.

The more urgent surface, he thinks, is code. AI is getting really good at finding vulnerabilities. Take a recently disclosed bug, run an LLM over the codebase, and you barely need to prompt it. Someone found a Linux zero-day that way. Cybersecurity is one-sided: it is hard to know you have no holes and easier and easier to find them, including through social engineering. The only thing you can rely on, from a user's perspective, is math. You want a mathematical proof, at the time of using the system, that it is correct given your requirements. That is formal verification. Mars rover code was verified once, then any update or requirement change threw it away, because the labor was enormous. AI that can win gold at the International Mathematical Olympiad can drive that cost down. Every line, in their thesis, eventually needs to be formally verifiable at execution time. Blockchain is the first root of trust to verify: when you deposit money, the chain proves you can get at least that much back. Expand it to a service that holds private data and must prove it will not leak.

Trusted execution environments are the other half: a guarantee that this code actually ran over your data. A couple of weeks before the taping there was a zero-day across iOS and Mac. The day before, a massive npm supply-chain attack in crypto injected code to replace addresses in tools. The volume of this, he says, is going to accelerate, which is why they are doing longer-term research on a new approach. To get privacy, verifiability, and a financial engine at once, they combine blockchain, cryptography, and hardware.

About a year before the conversation, NVIDIA added confidential computing. The GPU talks to Intel, ARM, and even AMD CPUs so that even the hardware owner, as operator of the OS, cannot see inside. In confidential mode you get both confidentiality and verifiability: this code ran on this data, only this was done, and nobody else saw it. A user establishes an encrypted channel into the secure enclave, runs a workload, and gets the result back encrypted, plus a certificate that this model ran on this input and produced this output. No silent quantization, no injected prompts. That is the primitive.

They need hundreds of thousands of GPUs. Giant data centers bring capex, energy, latency, and sovereignty problems. Because owners of the boxes see nothing, the compute layer can be permissionless: anyone with compatible NVIDIA and Intel silicon can join, certificates prove the chips, requests route to the closest available machine, and providers get paid the way validators do. Users and developers pay for inference. The result is what they call a decentralized confidential machine-learning cloud, also usable for arbitrary compute, data processing, agents, MCP servers, encrypted memory. Model developers currently face two bad roads: open-source and make zero, or stay closed, upload weights to a hyperscaler who might steal them, and sign a two-year compute contract before they have users. Encrypt the weights, decrypt only inside the enclave, and when a user pays, split the fee between developer and hardware provider. No prepay, no user data, no GDPR, HIPAA, or SOC 2 on the developer. Data providers can upload content that pays like a Spotify split when it is used at inference or later in training.

Labenz asks where the trust actually sits, and what the overhead is. Two years earlier, zero-knowledge proofs were pitched as a way to show Claude was not secretly quantized at lunchtime, but the overhead was enormous. Polosukhin puts ZK at about 1,000 times slower in even the most performant setups, and notes that ZK gives verifiability without privacy: the computer still sees your data. Fully homomorphic encryption is private and verifiable at 1,000 to 10,000 times slower. Secure enclaves are the pragmatic middle. In their production testing the maximum overhead has been 5 percent, usually 1 to 5. You will not get "dumber in the daytime" on that cloud. The limit today is a single machine, eight GPUs, which still fits current open-source models, including over a trillion parameters. Cross-machine is in the works.

A hardware attack is possible in principle. Chips are at 2 nanometers, baked into the atoms; the cost of modification is extremely high, and in their system you cannot target a specific use case — you might intercept someone asking which cat to pick. Remaining risk is NVIDIA's key management and process. They already run top-line security; Polosukhin would still like formal verification and blockchain mechanisms around that certification. That, Labenz had said at the top of the show, also helps explain how frontier labs have shipped weights to so many inference partners without leaking them.

NEAR has talked about training a more-than-trillion-parameter model at an estimated $160 million, a number Polosukhin says has been dropping since they first quoted it. Labenz cites Dario Amodei's framing: each generation burns more, but each model as its own venture can be profitable — $100 million to train Claude 3 and a billion back, then $2 billion for the next and $10 billion of revenue, then $20 billion of cost. Polosukhin is taking that to its conclusion: structure a sub-venture per model. The roadmap is inference first, then encrypted model upload, then a benchmark marketplace, then fine-tuning, then training from scratch.

They are not trying to build one model. They are trying to build a process that produces state-of-the-art, user-owned models whose construction people can inspect. Frontier labs already run on a flywheel of benchmarks: try an idea, test it at several sizes, accumulate what works, then do a bigger run. On NEAR, anyone can stand up a benchmark — predicting the future, astronomy, the ocean, deepfake detection. An enterprise can upload private evaluation data that nobody else sees, while everyone can still run a verifiable eval against it, including against a closed model whose hash the company posted on-chain. Today you cannot benchmark an intermediate OpenAI checkpoint, because they will not give you the weights, and you may not want to upload your data to them.

Fine-tuning is the next product. An ocean institute has a room of data and no training know-how; a trainer has the know-how and no data. They partner inside confidential computing, never seeing the rows, and split revenue between developer and data provider. Full pretraining is the hard last step. Data filtering and curriculum construction can themselves be opened to competition: train a few small sizes, see whose curriculum builder wins, then scale the winner. PII filtering — phone numbers, Social Security numbers — can be open-source so developers inspect it. A lot of personal chats, he suspects, are not that useful anyway; the field is moving toward synthetic data. Crowdsourced labeling still runs on the same network: NEAR Crowd, the original Scale-style project, and PublicAI.

Labenz pushes on coordination cost and on taste. Cluster setup, data pipelines, and hiring labelers are already hard. Benchmarks are everywhere and also, in the last few weeks of discourse, accused of being useless. Famous projects run on vibes. Who is the vibe checker, and how do you make sure the model is actually nice to talk to? Polosukhin's high-level answer is a marketplace of those decisions rather than a prescription that Illia gets the compute and you do not. He will probably participate as one tastemaker. Anyone else can too. The rules are known, the trace is visible, and people can build on each other's work instead of only competing.

Concretely: he wants to train a 1.4-trillion-parameter model, so he issues a token, a value-capture system for that specific model, and distributes it for compute, data, and research. He started it, so he will likely pick the pieces. If it gets used, distilled, fine-tuned, revenue goes to token holders. In parallel someone trains a 70B on the same framework, cheaper and faster, for a different set of jobs. You do not need to raise $10 billion. You raise smaller, specifically for compute and data, on top of building blocks that already exist. Some teams will keep the mix private; users will know that and decide. Others will show a full trace. Posted benchmarks today often omit even the prompts — a two-page prompt can answer half the questions. Here you can have Illia's vibe benchmark and Bank of America's test sitting next to each other.

All of that still rests on validators locking NEAR and asserting that the logs, payments, and training traces they put forward are valid. Labenz asks where a database that large lives when it no longer fits on one node. NEAR is sharded: each node holds part of the data and executes part of the transactions. More data means more shards, which is why they need more validators. It is the same move Google and Meta make — add more computers and run in parallel, rather than putting up a "no more users" sign.

Labenz's sketch of the bargain: crypto needs AI to put the smart in smart contracts, especially cheap dispute resolution, because courts are inconvenient where they exist and missing where they do not. AI needs rock-solid guardrails, checkpoints in a workflow so an agent can explore but cannot act without a validation — the Jurassic Park special-food idea, he admits, with the fictional ending attached. Polosukhin says both directions are live across a large ecosystem, not one company. One thread is NEAR Intents. You, or your AI, express something to be achieved: pizza tonight, or a building on this site with these qualifiers. The protocol has to discover who can do it, reach a commercial agreement — a car to the airport, or a six-month construction job — and handle disputes. AI can look at the chain, at both sides' private data, and run discovery without million-dollar legal bills. If parties still reject the result, they go to court with that work already done. Go-to-market started in crypto trading and is expanding into e-commerce.

The other thread is autonomous agents. A smart contract was already an autonomous being with money, independent of any person. The problem was that it was a small piece of code, so it was dumb. They put the AI brain on decentralized compute and let the contract execute actions. One example: an agent launched with $10,000 the team cannot stop, cannot take back, trading on Twitter sentiment. It made about $4,000. Imagine businesses whose bylaws are the prompt, shareholders giving feedback, training data improving the policy, or a vote to upgrade the model. Intents are how those agents do commerce with people and with each other. AITP is the communication protocol.

Guardrails, for him, are closer to a sandbox than to thought police. Before an AI service executes on your data, it should prove it will not, in his phrase, escape and go kill a bunch of people. Verifiable compute lets you attach a set of things you do not want the model to do and check the output. Governance still has to decide the list. Everyone may agree on bioweapons and disagree on other items. Experiments include AI senators: people vote for an AI, vibe with it about their views, and it constructs a platform and does the deep research humans will not. That, he argues, removes the principal-agent problem, because a human delegate still has their own agenda.

Labenz is nervous about agents you cannot take back, a billion of them, plus governance delegated to AI senators — a recipe, some would say, for AIs running everything. Polosukhin's positive picture is fusion, not a handoff. AI becomes more of an economic force; people become more like governors and curators. Formal verification is one piece. The other is jurisdiction. If an autonomous agent wants to execute an intent in the physical world, that intent is a legal contract. It needs a real jurisdiction, and that jurisdiction needs enforcement over the agent. NEAR is building infrastructure for courts to enforce on AI agents, visible and on-chain, combined with the old legal frameworks AIs can actually read. Labenz proposes a part two, recommends a science-fiction book he calls *Liquid Rain*, and Polosukhin's close is an invitation: it is a movement. Developers have a ton to build. Everyone else can own a piece of the AI they will live inside.