Douwe Kiela on Data, Retrieval and Enterprise AI

Douwe Kiela says enterprise language models face hallucinations, weak attribution, difficulty updating or removing information, privacy and compliance concerns. He is co-founder and CEO of Contextual AI, which aims to address these problems; host Harry Stebbings discloses at the end that he invested in the company.

Kiela learned to code as a teenager in the Netherlands and built an operating system at 16. He then studied philosophy, including questions about mind and language, before returning to computer science at Cambridge and working in natural language processing.

An internship with Léon Bottou led to research at Facebook AI Research, where Kiela worked with Yann LeCun and others. He says the experience taught him to focus research on clear real-world applications rather than pursue work too far ahead of the field.

Stebbings asks whether Meta captured the economic value of open-source work such as React. Kiela says it may have captured less direct revenue, but argues that React and PyTorch advanced the company’s broader mission and had a large impact on the field.

At Hugging Face, Kiela found an organization aligned with his interest in open science and was struck by its marketing, brand and community. He and co-founder Amanpreet Singh, who had worked together at Meta and Hugging Face, later saw an opening after ChatGPT’s launch: enthusiasm was high, but enterprise deployment still had major obstacles.

Kiela lists confident fabrication, lack of traceable sources, difficulty revising or deleting model knowledge, privacy, compliance and latency as problems. He presents Contextual AI’s planned architecture as an enterprise-oriented response, not as a verified demonstration that all these problems are solved.

He describes retrieval-augmented generation, which he says he and colleagues at FAIR introduced in 2020: separate a retrievable memory from the language model’s generation and ground responses in retrieved information. He expects this to improve attribution, reduce hallucinations, allow memory updates and support privacy and efficiency, though those are claims about a proposed approach.

Stebbings asks whether an open or closed model is still a black box. Kiela agrees that the internal operation of a neural network at this scale cannot be fully understood, but says training it to rely on retrieved information can provide stronger attribution for its outputs.

On the claim that hallucinations are a feature, Kiela distinguishes creative writing from business-critical tasks. A user seeking imagination may welcome invented material and revise it, while an enterprise process requiring reliable answers should seek more grounding.

The field was moving quickly enough that Kiela expected many new models within a year. A startup able to switch models could benefit, he says, but would also depend on other providers. He believes room remains for new model companies, including those focused on specialized enterprise problems rather than a broad pursuit of AGI—a view connected to his own startup.

Stebbings asks whether specialized intelligence can be achieved sooner than general intelligence. Kiela agrees that a bounded business problem is easier than AGI, a term he says has long been poorly defined. Specialized systems can improve as underlying models improve.

On model size, Kiela argues that training data can matter more under a fixed compute budget. He points to Meta’s Llama work as evidence that training a smaller model longer on more data can offer better value than simply adding parameters. With unlimited compute and data, he says, a larger model could still yield more capabilities.

Training time and cost depend on the number of GPUs and how resources are allocated. A large model trained on too little data can be undertrained; he argues the field had underestimated how much data is useful relative to model size. This is his interpretation of the trade-off, not a universal guarantee.

Incumbents can benefit from data, but much useful training material is public, he says. He speculates that special data may help explain the strength of GPT-4 and mentions audio transcription as a possible route to podcast text. He does not establish that OpenAI trained GPT-4 on all podcasts.

Asked whether startups need proprietary data, Kiela says the answer depends on the business. A deep-tech model company benefits from a data flywheel, while an application built on capable language models can sometimes start with relatively little data. He also describes the possibility of using a stronger model to generate data for cheaper specialized models.

He sketches three stages of building a chat system: pretraining on next-word prediction, supervised fine-tuning to follow instructions, and reinforcement learning from human feedback on response quality. The outline omits many implementation details, which he acknowledges matter.

Kiela admires OpenAI’s data acquisition and says ChatGPT’s popularity gives it insight into user needs and a potential data advantage. He disputes the claim that leading labs have no moat, also citing serving economies of scale. He says he likes open source but considers it naive to assume open models will automatically keep pace with heavily resourced closed models.

Evaluating models is difficult, Kiela says. Static tests can be contaminated when a model has seen evaluation material during training, and using one model to judge another raises its own questions. He sees an opportunity for independent evaluators serving particular use cases; he does not establish that any specific model’s benchmark score is contaminated.

He points to HELM and established NLP benchmarks, then argues static test sets can be overfit. His Dynabench work instead emphasizes dynamic human adversaries trying to break a model; the difficulty of finding failures can help track progress. That approach complements a broader evaluation problem rather than explaining every capability.

Kiela expects prompt injection and other attacks to create demand for security and auditing. As models generate code or take actions, a manipulated system could do damage, such as deleting a database. He predicts external specialists may provide this layer, while acknowledging that startups and existing security firms are both exploring it.

He pictures model offerings as a pyramid: costly frontier models at the top, small open models at the bottom, and midsize models in between that may suit businesses seeking value for money. He expects different applications to use different models rather than a single winner.

Kiela expects open models to retain a role in research and deployment but doubts they will reach the frontier without sufficient resources. He attributes much of the contemporary open-model activity to Meta making Llama available and says the cost of training very large models is a barrier. This is his 2023 forecast, not an established outcome.

Discussing a proposed six-month AI pause, Kiela jokes about competitive motives and argues that some public narratives are driven by self-interest. He acknowledges sincere concern and a nonzero chance of extinction risk, but personally considers it extremely small and compares it with other risks such as nuclear war, pandemics and climate change.

He argues that incumbents may benefit from regulation that smaller firms struggle to meet and that alarming narratives can benefit some participants. He does not identify an individual or substantiate a general claim about motives; Contextual AI itself has an interest in how regulation treats startups.

Kiela says the AI community should better educate regulators and the public about what systems can and cannot do. He worries that European overregulation could harm innovation and favor incumbents, while saying he is not an expert on the details of EU rules. Stebbings also voices concern; these are their policy opinions.

On enterprise adoption, Kiela expects a wave but says experimentation and some production deployment were already happening in 2023. He expects adoption to be gradual as companies identify suitable use cases and address hallucination, attribution, privacy, latency and other obstacles. Stebbings cautions that adoption may take longer than excitement suggests.

A bank’s unwillingness to send transaction data off-premises raises a hard deployment question. Kiela describes separating a customer’s data plane from a model plane, potentially retaining data in the customer’s virtual private cloud while deploying a model elsewhere. He says decoupled retrieval and generation could enable such hybrids; actual privacy depends on the implementation.

Asked about large AI fundraising rounds, Kiela says the potential upside may justify them, though some companies must now meet high expectations and generate revenue. If enthusiasm turns to disillusionment, he warns, funding could dry up. As a funded AI founder, he has a stake in that market.

He praises Microsoft’s partnership with OpenAI as a strong strategic move and says he is curious what Apple will do with assistants such as Siri. He thinks the field is still early and that important technical questions remain unresolved.

AI companies do not have to be in Silicon Valley, he says, though a local office can help recruit from universities. He also wants the community to temper hype, focus on technology that works and discuss ideas more deeply than social-media sound bites allow.

Kiela sees demand for companies helping enterprises implement AI, whether those providers are startups or existing cloud companies. His philosophy background helps him reason about questions at different levels of abstraction, including AI questions that are not yet sufficiently defined for scientific testing.

He admits he underestimated the value of scale and had dismissed work that used more compute and data rather than new algorithms. In hindsight he regards that judgment as wrong and credits OpenAI with pursuing scaling effectively.

Asked about superintelligence, he says the term is poorly defined and notes that computers surpassed human calculation long ago. Using a definition of AGI centered on performing many economically valuable human tasks, he predicts substantial economic displacement in five to ten years. The timeline is his speculation, not a demonstrated outcome.

Kiela says an early lesson was to put people ahead of ambition and develop more empathy for colleagues. Looking ten years ahead, he hopes Contextual AI’s approach could become a leading generation of language-model technology; he offers that as an aspiration, not a result. Stebbings then thanks him for allowing him to invest.