Snowflake's AI Stack: Small Models, the Text-to-SQL Iceberg, and Why Governance Comes First

Baris explains that Arctic, Snowflake's in-house large language model, was a relatively small effort by market standards. Work started around December, driven by customers who wanted BI-style experiences — translating natural language into SQL and querying their data without writing the query themselves. Snowflake wanted a model that was genuinely good at SQL and at following instructions, because general-purpose models struggle with the complexity of real SQL workloads. The company decided to invest in building one rather than relying entirely on third-party models.

The team that built it was small but unusually credentialed: it included the founders of DeepSpeed and vLLM, two projects at the center of efficient training and inference. With that group assembled, Arctic was built and released in roughly a three-to-four-month window. Baris frames the goal as enterprise needs rather than general capability — the pitch was not that Arctic writes the best poetry, but that it should be strong at the SQL copilot use case and at high-quality chat.

Two things drove the result. First, the team designed an innovative and efficient architecture, optimizing for efficiency in both training and inference, which let Snowflake build the model at roughly one-eighth the cost of comparable models. Second, once the architecture was settled, most of the remaining effort went into the data recipe, where the team spent a great deal of time getting the data right. Baris adds, with a nod to the poetry caveat, that the model turned out to be surprisingly good at sonnets anyway.

Snowflake serves models through Cortex, a managed service that runs Arctic alongside models from providers such as Meta and Mistral. Baris says customers do three main things with it: BI-style use cases such as SQL generation, traditional chat over unstructured data like documents and PDFs, and extraction from structured and semi-structured data. That third category includes batch text analytics on sales call logs and customer support tickets, where natural language pulls structured values out of raw text — work that is increasingly processed at scale.

BI is the trickiest of the three, and Baris calls it an iceberg problem: it is very easy to build a demo and very hard to survive the real world. Public benchmarks such as Spider are simplistic compared with production data, where customers have tens of thousands of tables and hundreds of thousands of columns with names like rev_1 and rev_2 that no model can disambiguate on its own. Snowflake's answer is Cortex Analyst, which stitches together three or four LLMs, knows when to ask for clarification, and includes self-healing systems that generate SQL and then check its validity. Baris says this got quality into the 90–95% range in Snowflake's own efforts — good for many use cases, but not something you put in front of a CFO and trust unconditionally for a revenue number.

Humans stay in the loop through a feedback mechanism: customers create verified queries, and when a question matches one, the API returns that answer with higher confidence, with new verified queries accumulating over time. Beyond that, the failures are predictable — complex joins against each company's idiosyncratic data model, hallucinated column names, and queries that simply will not execute. Baris points out that a huge amount of engineering goes into narrowing 100,000 column names down to the right few for the model, judging whether a question is even answerable, and stopping a model that will gladly answer questions it should decline.

The most telling product decision is that Snowflake consciously optimizes for precision over recall, giving up coverage in exchange for correctness. Baris says the team did not start there: the early version was broad and open-ended, able to handle every join, which is acceptable for an analyst who can spot an 80–90% right answer and correct the rest. Business users have no such luxury, so the product narrowed substantially — it should know when not to answer and when to ask a clarifying question instead.

On the extraction side, Baris thinks things are already pretty good. One example he gives is an employee survey with open-ended responses: rather than querying cell by cell, you can ask the model to summarize everything, propose categories, and pull example quotes, turning a couple of hours of spreadsheet digging into a prompt. Because Snowflake's ethos is to make things simple from the customer's perspective, it builds task-specific functions that spare customers the prompt engineering and pipeline work and can be used off the shelf.

The product surface rests on three pieces: Cortex as the inference engine running all the models, Cortex Search for high-quality retrieval, and Cortex Analyst for structured data — in Baris's framing, talking to your unstructured and your structured data respectively. Search quality is central to RAG, so Snowflake built its own embedding model, Arctic Embed, which it describes as state of the art. That model is a quarter the size of OpenAI's embedding model while scoring higher on benchmarks than some competitors.

Size matters because cost matters. Baris says cost is a big factor, and that the market has been bringing prices down largely by distilling large models into smaller and smaller ones, which are now remarkably effective and efficient. The same logic drives Snowflake's own investment in efficient architectures and in compact embedding models.

Snowflake sees three kinds of search applications. There are external, user-facing RAG applications where the focus is quality, reduced hallucination, and knowing when to abstain; internal productivity use cases, which are the most common because they carry low risk; and, somewhat surprisingly, a revival of classic enterprise search — Baris credits Glean with doing a phenomenal job and says customers want to upgrade their existing stacks. Cortex Search is a hybrid system combining vector search with lexical keyword search, which helps on the hallucination front.

The first obstacle to AI deployment at a large company is rarely technical. Baris says many large companies have AI governance boards, and the initial step is making sure the company feels comfortable with the platform of choice — that data is secure, governance is respected, and the policies are acceptable. Because Snowflake runs AI systems right next to the data, it can answer many of those questions directly and provide reassurance; only after that does the conversation move to building something high quality and starting a data flywheel.

Evaluation is the next wall. Teams build a system, ask a couple of questions, and cannot tell whether the answers are good, bad, or worse than what another model would produce. Snowflake acquired an LLM observability and evaluation company whose open-source product, TruLens, gives customers a full-stack evaluation path, including running evaluations at scale with LLMs as judges. Baris notes that until recently almost everything was a proof of concept, and evaluation was the part people could safely skip.

The transition from proof of concept to production is real but uneven. Internal productivity and analysis use cases are moving, while external-facing deployments still need more confidence before they scale. Baris thinks both product innovation and better technology are required: end users need to become comfortable with answers that are not always right, and they need mechanisms to check them. Hallucination remains a live risk, and in regulated industries a wrong answer can be catastrophic.

Running models next to the data is Snowflake's core value proposition. All of its models run inside Snowflake for customers by default, and customers who want external models can configure that, but it is not the default path. Governance is built from the ground up, with granular access controls that let customers grant or restrict access to essentially any database object. Baris argues that having this in place before AI arrived is a major advantage, since many customers spent a couple of years getting governance right and can now build AI on top of it.

He uses an HR chatbot to explain why governance has to be native. If someone asks who got promoted last week, the correct answer depends entirely on who is asking, and there is very little room for hallucination or data leakage. That example, rather than the more familiar concern about one colleague reading another's SQL, is what makes granular, platform-level access control essential for enterprise AI.

The new engineering work was extending those controls into retrieval. Snowflake had to build access filtering deeply into search so that a query only surfaces documents the asker is permitted to see, and data questions fall back on the existing object-level grants: if you cannot access the data, you do not get the data. Baris frames this as a compounded advantage — the governance investment customers already made now pays off again in AI, and it would be very hard for a newcomer to recreate.

Snowflake's advice to customers is to start with large models and build proofs of concept, since a large model plus a RAG solution usually gets you quite far. Optimization comes later, once the system is headed for production: fine-tuning a smaller model can deliver latency and cost advantages at scale, and companies with large amounts of unique data may justify a custom pre-trained model. Baris notes that Arctic itself shows building a targeted custom model does not have to be outrageously expensive. Regulated industries often want full control over what data goes into the model, and healthcare is a good example because of its distinctive vocabulary — the model does not need to write a French sonnet, but it does need to understand the business.

Inference, not training, is where the volume sits, which is why Snowflake's messaging centers on running any model well. Fine-tuning is supported and made simple for customers who want to fine-tune and then use that model in place. A small number of customers go further and work with Snowflake to train their own models and run them inside Snowflake, but Baris is clear that this is the exception rather than the default.

Asked what patterns he sees in model selection, Baris admits something somewhat surprising: customers start choosing models based on brand rather than capability. Very capable models go underused because their brand recognition is not there. Customers ask for help selecting models, and Baris expects that making LLM evaluation frameworks easily accessible will push those decisions toward measured quality rather than reputation.

Supporting new models is mostly about keeping the inference stack current. When Llama 3.1 405B arrived, Snowflake built a series of optimizations into vLLM, including multi-node inference, for both inference and fine-tuning, and upstreamed the work with the vLLM team — whose founders are now Snowflake employees. Baris says the 405B model is very capable but slower, though still not unreasonable on cost, and it is being used for fine-tuning, distillation, and synthetic data generation. On cost more broadly, he is not seeing enterprises blocked from production by price: most current use cases are internal, volumes are not massive, and model costs have been falling rapidly.

Snowflake has no ambition to build a fully general-purpose model that competes with the frontier labs. Baris says the goal is to serve Snowflake customers' specific needs, which means focusing on SQL generation and RAG quality and doubling down where Arctic can be measurably better for the data work customers actually do.

Guardrails are a live topic with customers, and Baris describes an ongoing debate about whether open-source or proprietary models ship better alignment. In his view Meta has done a great job aligning Llama, and offering Cortex Guard with Llama Guard underneath as an additional layer resonates with the market because this is uncharted territory for most businesses. Model responses are completely unscripted, so companies want reassurance that behavior lines up with both policy and brand.

Asked what is still missing, Baris lists hallucinations, immature measurement, quality in complex scenarios such as text-to-SQL, and agentic systems. Evaluation remains a weak point for many companies, and agents are uncharted territory with a lot of potential that is only just getting started.

On build versus partner, Baris points to the Neeva acquisition as instrumental — it brought in Snowflake's CEO as well as technology that underpins Cortex Search and its embedding model, along with engineers who became core to the team. Snowflake also partners with model providers such as Mistral, Reka and AI21, invests in some of them, and works with companies like Landing AI that bring full end-to-end solutions. Those partners can run their applications inside the Snowflake boundary and access Snowflake data directly, so customers who do not want to move data out of the platform can still use capable third-party products.

Internally, Snowflake uses LLMs for sales teams summarizing why deals are won or lost, for employee assistants that answer questions over internal data and documents, and for documentation-oriented chatbots. AI has also become a core enabling technology inside the product itself, from optimizing the SQL engine to bringing AI capabilities into the Marketplace. Asked to contrast Snowflake's strategy with Databricks, Baris returns to the single-product ethos: everything works well with everything else, which he admits makes development difficult but makes the product delightful to use. He points to the quality of Cortex Analyst's text-to-SQL and Cortex Search's retrieval, and notes that Snowflake was one of the first companies to integrate SQL AI into SQL as the first thing it showed.

Baris is bullish on startups in the AI infrastructure stack. The market is massive and still growing, he has already seen impressive infrastructure startups, and Snowflake talks to and works with them despite the pull toward end-to-end platforms from buyers whose governance boards want a single vendor. He is also excited about agents plugging into the system and about the data-sharing ecosystem: because data sharing is easy on the platform, an asset manager can access S&P or FactSet data, and as those companies turn their data into AI-ready data, a firm like Morgan Stanley can build a chatbot on top of all of it. On the inference stack, he expects continued innovation because every improvement translates immediately into lower cost, though he sees long-term inference businesses as a scale play with room for only a few winners.

Baris's own background is in assistants: he started Google Now, an early AI product that morphed into Google Assistant, and jokes that after a decade the product was essentially a series of handcrafted use cases until LLMs made it general purpose. Assistants have so far taken narrow actions like setting alarms, timers and playing music; now that LLMs can call APIs, he thinks they become far more powerful. He favors the vision of many small, specific agents created in natural language and running on your behalf, because everyone's needs are unique — an agent that watches a website for something to become available is valuable to one person and irrelevant to everyone else. Agents are already appearing in practice: Cortex Analyst behaves agentically, deciding when to generate SQL, when to evaluate it, run it, and reason about the data that comes back.

In the closing round, Baris names evaluation as the most underhyped thing in AI and agents as the most overhyped today, though he expects the agent hype to eventually be matched by reality. His biggest surprise building Snowflake's AI was text-to-SQL: the team knew it would be complex, but finding the exact problem to solve turned out to be the core of the solution — the classic machine learning lesson still applies. He credits Meta and Mistral with showing the market that it is not a two-player industry, names Mistral as the startup he is most excited about outside his own space, and says that if he built an application tomorrow it would be an assistant. The catch, in his view, is that assistants want to be platforms while a startup cannot start as one, so the only viable entry is to nail a couple of deep use cases the way Devin did for engineering — and he expects the application space to move from selling building blocks to selling end-to-end products over the next two years, with customer service as the most obvious example.