In 30 seconds
Zhang Xiaojun and Saining Xie revisit his academic journey and vision research, explore representation, self-supervision, prediction and planning, and discuss his plans for AMI Labs with Yann LeCun. Research judgments and future applications are presented as his views.
- 1Representation as a central threadXie traces how vision models can learn useful internal representations.
- 2Prediction supports decisionsHe uses control theory to connect state, action and prediction to planning.
- 3Simulation and understandingHe distinguishes generating visible scenes from forming a model usable for reasoning.
- 4Balancing research and businessAMI Labs aims to pursue foundational research while finding a sustainable business model.
- 5Rethinking intelligenceAnimal and robotics examples remind him that familiar symbolic tasks are not the only measure of intelligence.
Takeaways
- Useful representations preserve information needed for particular tasks and decisions.
- Generating realistic video and reliably understanding and predicting the world are distinct challenges.
- Data, training methods and applications for world models remain open questions.
- AMI Labs aims to balance long-term research, partnerships and commercial sustainability.
- Animal behavior and everyday life inform the guest’s view of intelligence.
Read the edited text by chapter
The normal one and growing up
In New York, Zhang Xiaojun interviews Saining Xie about his upbringing, vision research and world models. Recorded in early 2026, the conversation also covers his subsequent founding of AMI Labs with Yann LeCun; technical and business forecasts are time-specific.
In New York, Zhang Xiaojun interviews Saining Xie about his upbringing, vision research and world models. Recorded in early 2026, the conversation also covers his subsequent founding of AMI Labs with Yann LeCun; technical and business forecasts are time-specific.
The conversation opens with New York neighborhoods. Xie says he likes the city’s everyday and artistic life; NYU’s AI research, Yann LeCun and its film tradition also attracted him.
This is his first podcast-style interview. He enjoys listening to podcasts and talking with friends and students, but previously worried about what was worth saying publicly.
Recalling childhood, he says his mother often took him traveling and his father loved books. Seeing places outside and reading at home gave him two different ways to explore.
He got his first computer at about nine, initially using it for games and later discovering the internet. Forums and blogs let him express ideas as well as read.
He recalls posts on early social platforms that now seem immature, but prefers to leave them as part of his personal and internet history.
No one in his family worked in a purely technical field. He credits a relatively relaxed upbringing with helping him approach the world openly; this is his own interpretation.
On school, he does not describe himself as following a perfectly optimized meritocratic path. Compared with peers with exceptional competition and academic records, he sees chance in his own choices.
In high school, competition experience gave him an opportunity to enter Shanghai Jiao Tong University early. Teachers encouraged him to pursue the entrance exam, but he liked Shanghai and wanted to study computing.
Entry to the ACM class required additional selection. He says he would choose the same path again and remembers spending the summer before classes playing games in the dorm.
He does not deny the opportunities he received and appreciates the university and alumni network. Whether a path looks smooth, he says, depends heavily on the comparison.
He recalls being asked about a favorite book during the ACM-class interview and naming What Is Mathematics? and Richard Courant. Years later he joined NYU’s Courant Institute, a coincidence he finds meaningful.
Early in college, he encountered books on brains, consciousness and images in a neuroscience-and-AI lab and met older student Xiaodi Hou.
He admired Hou’s early computer-vision publication and sought advice on research direction and writing papers. At the time, he says, such results were unusual for undergraduates.
Books and conversations helped him identify an interest in vision research, while boundaries among image processing, machine learning and computer vision were changing.
The ACM class arranged internships, often at Microsoft Research Asia. Unable to find a vision group taking undergraduates, he did not want to switch to an unrelated field.
He contacted Shuicheng Yan’s lab at the National University of Singapore himself and told his teacher after arranging an internship. The teacher agreed, and the option later became available to other students.
He explains his attraction to vision through films, games and seeing the world. That personal motivation later connected with research on perception and intelligence.
He mentions a hypothesis connecting the evolution of vision with Cambrian diversification as an illustration of perception’s importance, not a settled account of evolutionary causation.
He sees vision as more than an isolated sensory problem: it relates to how organisms understand their environment. That view recurs in his discussion of representation learning and world models.
Xie says each person reaches research questions through experiences, reading and conversations. He encourages students to see themselves as potential sources of distinctive contributions.
In Singapore he worked with Jiashi Feng and others on an early paper. Completing a research project and writing it up gave him a first understanding of the process.
The experience coincided with AlexNet and growing attention to deep learning. He sees it as a starting point for his own research as well as a turning point in the field.
He identifies curiosity and the desire to exchange ideas with strong researchers as motivations for further study, leading him to apply for doctoral programs abroad.
Comparing Jiao Tong’s ACM class with Tsinghua’s Yao Class, he notes he did not attend the latter. He recalls courses encouraging broad interests and presentations outside required academic subjects.
He says he does not fear competition but dislikes excessive competition and favors multiple standards of evaluation. He values pursuing an interest over chasing a single ranking.
Vision research and academic journeys
While applying for a PhD, he did not initially receive the vision opportunity he wanted and considered other fields. He emailed Zhuowen Tu and received a response near the deadline.
While applying for a PhD, he did not initially receive the vision opportunity he wanted and considered other fields. He emailed Zhuowen Tu and received a response near the deadline.
Because of the time difference, they spoke by phone in the middle of the night. He explained his experience and interest in working together and received an opportunity to join Tu’s group.
Before he enrolled, his adviser decided to leave UCLA without a confirmed destination. Xie chose to wait and follow him rather than decide solely by university ranking.
Tu later moved to UC San Diego. Xie says who he worked with and whether he could pursue questions he cared about mattered more to him.
He recalls close supervision during the PhD, including going through code line by line. That hands-on teaching differed from his earlier short internships.
He admires earlier researchers who built systems before today’s mature tools and open-source libraries, saying they opened a path for later students.
He rejects a simple story of being ordinary and then suddenly famous. To him research is long-term construction aimed at understanding problems.
Early in his PhD, he and collaborators studied Deeply Supervised Nets. Deep learning was expanding quickly, though some vision researchers still preferred traditional methods.
Deep supervision adds training signals at intermediate layers so gradients need not come only from the final output. He says this helped address some difficulties in training deep networks.
Holistically-Nested Edge Detection used information from multiple layers for pixel-level edge detection. Combining different scales of information helped produce edge predictions.
The work showed him that even a neural network called a black box has intermediate layers that can be studied and connected to new tasks. It changed how he understood network structure.
The paper received a Marr Prize honorable mention. He recalls the excitement of the stage while cautioning that reviews and awards involve luck and are not identical to long-term value.
His adviser encouraged exploration, and he completed five internships during his PhD. He wanted to see what different organizations studied and whether he was missing other interesting questions.
He recalls driving from Southern California to the Bay Area with his belongings for summer internships, then finding housing again on return. The teams exposed him to different research cultures.
Not every internship produced a result. Some disappointing projects discouraged him, but taught him that an unsuccessful attempt is not the end of a career.
Late in his Meta internship, Kaiming He joined FAIR and they began collaborating. Xie learned research from him and helped him adapt to a new environment.
Meeting Kaiming He and other researchers
They entered the ImageNet challenge and developed ResNeXt over roughly a month. Xie praises He’s ability to turn an ordinary idea into a clear research direction.
They entered the ImageNet challenge and developed ResNeXt over roughly a month. Xie praises He’s ability to turn an ordinary idea into a clear research direction.
He explains that ResNeXt uses parallel groups to change network structure and study scaling at similar compute. He draws an analogy to later mixture-of-experts ideas, though their routing mechanisms differ.
The collaboration reinforced for him that research is not always linear: after a long period without results, a new combination can suddenly clarify a problem.
He advised students to try different environments. Xie later studied video at Google and explored reinforcement learning and embodied tasks at DeepMind in London.
He recalls the London internship as difficult and learned that he did not necessarily enjoy the reinforcement-learning problems he tried. Observing DeepMind’s organization was nevertheless instructive.
He describes a combination of exploratory small teams and organized execution: ideas can emerge bottom-up, then require coordination and clear execution as they mature.
He recalls Demis Hassabis’s ambitions for long-term scientific impact and seeing the AlphaFold team develop from exploration into a coordinated project.
He ultimately connected the diverse experiences through representation learning. He compares a representation to a tree’s roots and vision, video and embodied tasks to branches.
He defines representation learning as learning a mapping from raw data into a space with useful properties for downstream tasks. The mapping can be hierarchical and nonlinear.
Representation, self-supervision and vision models
He describes his field as representation learning because specific technical fashions change while forming useful representations remains a more enduring question.
He describes his field as representation learning because specific technical fashions change while forming useful representations remains a more enduring question.
He recalls an early paper being rejected amid a formula typo, later published elsewhere and recognized years later with a test-of-time award. The experience reinforced his focus on long-term impact over a single result.
He regards a single paper result as one point on a timeline and sees research contribution as something accumulated over longer periods. It is his personal metaphor for a career.
Before graduation he considered different labs. Despite other opportunities, he preferred FAIR and the vision researchers he already knew.
He recalls declining an OpenAI offer and a call with Ilya Sutskever. The choice reflected his preferences for research environment and direction at the time.
The FAIR interview did not go entirely as planned: he finished his talk early and researchers extended the discussion with questions. Such imperfections need not obscure a longer-term goal, he says.
Years later Sutskever contacted him again. Xie says work on language models and other directions need not negate one another; different teams can advance different problems.
He sees reliability, safety and control as technical questions and connects them to later world-model research. The proposed approach remains a research idea requiring validation.
He values trust built through research: understanding a paper and being inspired by it can connect people even when they do not regularly live or work together.
He is uneasy about expressing research goals solely as personal “impact,” preferring work others can understand and build upon.
On communicating research, he supports explanation and discussion but resists promotion centered only on a lab leader. He wants students who do the work to receive visibility.
He describes NYU’s data-science environment and sees AI as a bridge among computer science, mathematics, statistics and other disciplines. He admires LeCun’s early support for that interdisciplinary structure.
Xie describes Fei-Fei Li’s support, including sharing experiences and helping him navigate difficulties. He chooses not to disclose some private conversations.
He argues that Li’s contribution includes defining an important research problem as well as building ImageNet. This is his assessment of her research leadership.
He explains his collaborations with prominent researchers as growing from shared questions and mutual understanding rather than deliberately cultivating status.
Returning to technique, he says architecture, data and objective all matter. A network without suitable data and training objectives is like an engine without fuel.
He recalls Kaiming He’s early interest in larger data and models and the vision community’s exploration of self-supervised learning. Scale can help, but results still depend on method and task.
Self-supervised learning can construct training signals from the data rather than requiring a manual target label for every example. Different methods construct those signals differently.
He discusses limits of classification labels: a model may exploit incidental correlations such as background instead of learning an object’s structure. Evaluation affects whether such shortcuts are detected.
Early self-supervised vision research tried pretext tasks such as predicting rotations or reconstructing masked content. The goal was transferable representations rather than memorizing specific labels.
He recalls many early methods lagging supervised ImageNet pretraining. Reaching useful performance required repeated benchmark comparisons and training improvements.
Contrastive learning seeks representations that bring related views closer and distinguish different samples. Sampling and training design shape the actual result.
He discusses Kaiming He’s focus and describes strong researchers as combining attention, research taste, persistence and engineering ability.
A broad direction can come from reading and discussions with peers and advisers, but he doubts that the best specific idea always comes from sitting and thinking. Experiments provide new input.
He recommends starting with a clear problem and baseline, then reproducing or extending existing work. Good and bad results alike can indicate where to explore next.
He calls surprising observations important research signals. The most valuable idea may emerge during experimentation rather than in the original plan.
He suggests that a project with no difficulties or changes may lack exploratory value. This is his research preference, not an objective rule for every project.
He stresses the need to pivot. After a winding path, a researcher can connect the key decisions in retrospect, but cannot know every step at the outset.
Defining a problem is creative work in itself. He again cites Fei-Fei Li, arguing that identifying the right question can matter more than a small improvement on an existing metric.
He sees research impact as unevenly distributed: a few influential works can redirect a field. Counts and average performance do not fully capture long-term influence.
He contrasts a chess player minimizing mistakes with an inventor seeking a long-term breakthrough. The metaphor does not mean failure is costless or every breakthrough rests on one success.
He worries that academia may only chase results from better-resourced organizations without defining its own questions. Reproduction has value, but he also wants researchers to find new directions.
He explicitly says his experience does not represent the full diversity of research. His judgments about field-changing papers are likewise subjective.
On large language models, he expects them to remain useful but doubts they form the entire foundation of general intelligence. This is his research position; others disagree.
He revisits visual contrastive-learning work such as MoCo and describes scaling training and improving representations. His attribution of contributions is his recollection.
Masked Autoencoders learn representations by masking part of an image and reconstructing it. Their objective differs from contrastive learning but can also yield useful results.
He mentions extending self-supervised learning into areas such as 3D and regards the papers as steps in exploration, not proof that the problem is fully solved.
Compute and engineering infrastructure constrained experiments. The team built TPU training infrastructure that let subsequent work iterate on a reliable baseline.
He emphasizes baseline quality: a new method can appear better merely because the comparison is weak. Strong systems and controls make improvements easier to interpret.
The team tracked experiments in spreadsheets, deciding which metrics, changes and results to record. The tool is less important than extracting a clear signal.
He says negative results can provide direction: if a change hurts performance, researchers can analyze why and design alternatives rather than recording only successes.
He shares Kaiming He’s interests outside work and memories of playing games together, reminding listeners that researchers are more than their papers and experiments.
Asked about research taste, Xie says the difficult-to-define idea includes judging questions, designing experiments, communicating work and recognizing what matters.
Research taste and experimental practice
Using philosophical language, he argues that readers should look past a paper’s surface presentation to its substantive contribution. He presents this as personal understanding, not a philosophical conclusion.
Using philosophical language, he argues that readers should look past a paper’s surface presentation to its substantive contribution. He presents this as personal understanding, not a philosophical conclusion.
He values writing early because organizing a story can expose gaps and force researchers to consider what readers need to understand.
His team also attends to paper layout, figures and websites. Presentation is no substitute for research, he says, but clarity helps others understand and use the work.
He compares research with filmmaking: both involve finding a theme, exploring a path and deciding how to present it to an audience.
Beyond the final result, he wants papers to explain key decisions. Understanding the process may let readers apply lessons to different questions.
Choosing one’s own research interests can be difficult because following a crowd feels safer. Personal judgment in uncertain areas must still be tested experimentally.
He emphasizes questioning consensus: start from a widely accepted explanation and design controls to see which components actually contribute.
In ConvNeXt work, the team studied design differences between convolutional networks and vision transformers. He says results prompted them to reconsider assumptions about the role of self-attention.
That conclusion comes from particular experiments and does not make attention unimportant in all tasks. The broader method is to control variables and examine design decisions.
He recalls top-down research-alignment meetings that struggled to choose future projects. For exploratory work he prefers directions emerging from questions, experiments and feedback.
The team began comparing representations learned by diffusion models with supervised and self-supervised models. Initial experiments did not immediately yield a decisive answer.
They then found that using a transformer as a core component of a diffusion model could yield a simple, scalable design. This led to Diffusion Transformers, or DiT.
He treats simplicity and scalability as part of research taste: for the same goal, a clearer and more testable solution can be more valuable.
DiT initially faced reviewer skepticism, partly because its structure seemed insufficiently complex. He says acceptance in one review cycle should not be the sole measure of value.
The project spanned his move from FAIR to NYU. He recalls authorship, collaboration and organizational decisions as personal experience.
He discusses later video-generation work using DiT ideas and says the key is not only an architecture but also a team’s willingness to invest and build a system around it.
He uses “antifragile” to describe research that learns and improves from rejection or unexpected outcomes rather than treating shocks only as losses.
He introduces the Cambrian series, examining visual encoders and representations in multimodal systems. He wants systematic comparisons to identify which design choices matter.
He questions assumptions about commonly used visual encoders: a widely adopted component is not necessarily best for every task, and alternatives should be evaluated experimentally.
The NYU team faced compute and infrastructure constraints. Students and collaborators had to build systems, debug and choose worthwhile experiments under limited resources.
He discusses pressure from research funding and compute costs in US academia. This reflects his environment; funding varies across programs, institutions and time.
He compares securing resources to an aspect of entrepreneurship: contacting collaborators, explaining research value and allocating scarce capacity.
Cambrian-S extends attention to video and ongoing events. Xie argues that visual understanding should consider change over time, not only isolated images.
He uses a long take in film to illustrate continuous observation and participation in an environment. Relationships and events in video are not fully captured by independent static frames.
He proposes a progression from indirect textual knowledge and image question-answering to streaming event cognition. This is his research framework, not a universally accepted standard.
The team hopes its framework helps define a north star for vision research, but he acknowledges there is no final answer yet.
He treats vision as a perspective on continuous, high-dimensional, noisy signals rather than only a fixed set of tasks. Many objects and changes in an environment must be handled together.
He traces vision tasks from digit recognition to more complex visual question-answering. Language makes questions flexible but may mask weaknesses in visual representations.
He worries that overreliance on language models could act as a crutch and reduce effort to improve vision. This is a critique of research direction, not proof that language cannot help vision.
Robotics exemplifies continuous physical environments, but he does not want research limited to one hardware form. He seeks perception and prediction foundations usable in multiple settings.
He discusses differences in scaling language and vision, noting that people have long organized internet text for communication while raw visual data has different structure and uses.
He says language compresses and communicates information while omitting many physical details. “A cup broke” need not describe every change during the break.
He invokes Moravec’s paradox: perception and movement that feel easy to people may be difficult to automate, while some difficult symbolic tasks may suit machines.
He recalls academic research inspiring industry models but worries commercial secrecy may weaken open exchange and attribution. This is his observation about the research ecosystem.
He introduces research using stronger representations in generative models. The question is how generation can use meaningful internal state rather than only optimize surface pixels.
He argues that high-dimensional spaces can provide useful structure in machine learning, while higher dimension alone does not guarantee improvement; training, data and evaluation remain important.
His long-term vision is to use representations as an important foundation for a world model, then decode them into language, video or actions. The implementation and effectiveness remain to be demonstrated.
He worries the commercial value chain around language models may draw academic resources too strongly into one direction. He wants other paths preserved, not to deny language models’ value.
What is a world model?
Xie starts with a basic world-model formulation: a system has a current state, receives an action or intervention, and predicts possible subsequent states. He does not require every world model to predict only through time.
Xie starts with a basic world-model formulation: a system has a current state, receives an action or intervention, and predicts possible subsequent states. He does not require every world model to predict only through time.
Prediction can support decisions: foreseeing that putting a hand in a fire would hurt helps one avoid it. Control theory has long studied how models can predict the consequences of actions.
He illustrates planning with model predictive control: compare candidate action sequences, choose one with lower cost, execute its first step, and plan again from the new state.
In model-based reinforcement learning, a predictive model can also help a policy learn from simulated experience. He mentions Richard Sutton’s Dyna work as an example of combining models and learning.
What counts as a state depends on the problem. A useful state retains enough information for prediction and action rather than recording every detail of the environment.
For example, judging whether a table can support a microphone requires the support relationship, not necessarily every detail of its texture and lighting. Representation is therefore task-dependent.
He compares this with fluid mechanics: researchers need not simulate every molecule individually when macroscopic variables and equations describe useful regularities at an appropriate scale.
He invokes Sutton’s “bitter lesson” to discuss limits of hand-designed structure: search and learning may discover structures people did not specify. He sees language itself as useful but limited structure.
Safety and controllability also involve anticipating consequences. With a knife as an example, he argues a system should understand the danger an action may create as well as follow textual rules.
Asked how teams’ definitions differ, he distinguishes a world simulator that generates controllable video from a model that learns abstract state for understanding and planning.
A generative simulator can present a coherent environment, but visual quality, long-term memory and precise spatial consistency are different problems. Attractive images alone do not guarantee them, he argues.
He expects world models to work with associative memory, reasoning and planning, first improving understanding of the physical world and potentially helping game or other digital agents.
Data, perception and potential applications
He questions whether turning video frames into tokens for a language model is enough for direct visual understanding. This is his research judgment, not a settled conclusion.
He questions whether turning video frames into tokens for a language model is enough for direct visual understanding. This is his research judgment, not a settled conclusion.
Language usually serves communication between people, carrying intention and selective expression. He compares language models to tools for search and symbolic reasoning, arguing they cannot alone encompass all experience.
He discusses conditional video generation: producing images from text requires learning which visual phenomena might occur, but does not mean the generated result already has reliable world understanding.
Pixels are also an interface for people. He wants research to look beyond visible images to whether machines form abstract representations suited to prediction and decisions.
In his account, a world model is a research objective rather than one algorithm. Better representations might help language, multimodal understanding and video generation; he sees RAE as an early exploration.
On scaling laws, he argues that remembering knowledge in language and compressing high-bandwidth perceptual data have different requirements; data, parameters and tasks cannot simply be related in the same way.
He contrasts rich sensory input with limited attention: much perceptual information never enters conscious description, yet some of it still guides action.
He is confident in the vision for visual world models but acknowledges it may fail: how much data, which data and how to train effectively remain open questions.
He suggests starting with internet video while noting that access involves rights, partnerships and social choices; abundant data does not automatically settle authorization.
He imagines wearables continuously capturing bodily and environmental information to help users examine associations, such as coffee timing and sleep. This is a proposed application, not validated health advice.
Such devices still face practical questions about power, hardware, data handling and privacy; the idealized continuous perception discussed is not an existing product.
Robotics is another application he finds promising, but he acknowledges that a general-purpose robot able to care reliably for an older person remains far away.
He sees representation learning, world models and AI development as a continuous evolution, while worrying that “world model” has become an overly broad buzzword.
Founding AMI Labs with Yann LeCun
The conversation turns to founding AMI Labs. Xie recalls friends and mentors encouraging him to seek an environment for long-term research, leading to discussions with Yann LeCun about a shared direction.
The conversation turns to founding AMI Labs. Xie recalls friends and mentors encouraging him to seek an environment for long-term research, leading to discussions with Yann LeCun about a shared direction.
He wanted to pursue research-intensive questions with sufficient resources and felt academia and large companies faced different constraints. This is his personal account of the career choice.
He argues that competition over public leaderboards and short-term product metrics can shape resource allocation, making it harder to pursue questions without established benchmarks.
He believes defining important questions takes time and accepting exploration without immediate results. He wants the team to formulate its own questions rather than only execute a set roadmap.
LeCun described a company that was not simply a nonprofit research institute and would need a business model. Xie stresses this is his interpretation, not necessarily a verbatim quote.
Drawing on advice from other founders, he says products should start from real needs: world-model research requires understanding what people want solved as well as technical work.
He pairs “the world needs a world model” with “world models need the world”: the latter means obtaining questions, data and collaboration with external partners.
The team hopes to balance foundational research with applications. Xie says the roadmap still requires substantial research and a commercial balance that can sustain it.
He wants young researchers to gain visibility and room to grow instead of becoming easily replaced parts of a large organization; he also acknowledges that excessive individual ego can hurt collaboration.
For hiring, he favors capable people with strong reputations who may not yet be widely known. He values the maturity and research taste students demonstrate through complete projects.
He calls one proposed approach a “reverse OpenAI”: start from industry partners’ real problems, build models together, and use resulting value and further data to improve them.
This data-and-value loop remains a business and research proposal; the required data types, quantities and partnership terms still need to be determined.
He uses a historical narrative about payment networks to explain the appeal of open collaboration: multiple participants building an ecosystem can reduce dependence on a single center.
He mentions young founders working directly in factories and says teams of different ages and backgrounds can find opportunities by engaging with real settings.
Xie describes LeCun as a calming presence when the team faces difficulties. He says LeCun can change his mind in response to evidence, contrary to an image of absolute stubbornness.
He recommends LeCun’s position paper on autonomous machine intelligence, saying its architecture and objectives are more concrete and layered than simplified public summaries.
He recounts LeCun’s management analogy: trust team members to do their work and adjust direction together when needed, like sailing a boat.
Moving from research collaboration to founding a company lets him bring friends and students of different backgrounds around one goal. He hopes to preserve a mission-driven culture while taking the business model seriously.
He says funding can ease immediate pressure, but startup capital remains precious and must be allocated carefully. Delivering research breakthroughs and a viable business will be the longer-term test.
Silicon Valley, funding and founding
On not choosing Silicon Valley as the sole center, he says concentrated competition around large language models can shape the research culture. This reflects his personal experience and preferences.
On not choosing Silicon Valley as the sole center, he says concentrated competition around large language models can shape the research culture. This reflects his personal experience and preferences.
He shares LeCun’s interests in photography, astronomy and jazz, suggesting researchers’ lives and tastes can influence how they formulate questions.
He mentions Tarkovsky’s Solaris, connecting its themes of memory and the subconscious with generated images, while noting the film is also concerned with human suffering.
Asked whether prominent founders and large funding make a startup privileged, he acknowledges that LeCun’s standing makes fundraising easier, but says seed funding is not itself proof of success.
He likes working with a challenger’s mindset and compares it to skiing: one must find balance and sometimes face the slope rather than instinctively lean away.
When assessing researchers, he values courage, persistence and a desire to understand hard questions, not just existing achievements. He sees these qualities as useful in both research and founding a company.
Founding a company lets him focus on assembling a team and setting priorities for the coming months. He acknowledges he may be wrong and hopes the team can adjust together.
He recalls hesitating about starting a company before being drawn to LeCun’s vision. The team also includes people with operations, business and research experience, rather than researchers alone.
He says the team will explore consumer products, but the interview does not provide a validated product plan or delivery date.
He divides his view of JEPA into questioning, understanding and embracing it: initially he saw another self-supervised algorithm, then came to appreciate its architectural ambitions for prediction and planning.
He sees JEPA’s broader goal as learning useful representations that support understanding of the world, while the proposed path toward general intelligence still needs research validation.
Human and animal intelligence
Founding has brought long hours, legal work and coordination across offices, as well as satisfaction in seeing a team take shape. He describes practical ups and downs rather than a smooth growth story.
Founding has brought long hours, legal work and coordination across offices, as well as satisfaction in seeing a team take shape. He describes practical ups and downs rather than a smooth growth story.
Discussing human arrogance, he reminds listeners that human perception and consciousness also have limits, and argues against judging all intelligence solely by familiar human achievements.
He cites examples from animal-behavior research involving mirrors, self-recognition and chimpanzee social relationships to highlight different abilities across species.
He also discusses observations of food-caching birds and whale communication. Whether animals possess abilities equivalent to human language is not settled by the interview; he uses the examples to challenge a human-centered view.
He recounts Sutton’s view that building a system with squirrel-like adaptive abilities may be harder than some symbolic tasks humans find difficult. It is an argument about how to evaluate intelligence.
There remains a large gap between everyday household tasks children can handle and robot capabilities, he says. Progress is uneven, and skill at a particular movement does not imply general common sense.
Life, creativity and 42
He discusses collecting data through real robot deployments, simulation and imitation learning, mentioning research-oriented robotics companies. Their costs and generalization still need testing.
He discusses collecting data through real robot deployments, simulation and imitation learning, mentioning research-oriented robotics companies. Their costs and generalization still need testing.
Xie says he is not uniquely chosen, mentions long supporting Liverpool, and likes a coach’s emphasis on being an ordinary person. He wants to remain aware of his own limitations.
He says a leader sometimes needs to act like a battery for the team. Although he often feels discouraged, he hopes to pass on the confidence he feels from LeCun.
Discussion on social platforms can sometimes relieve isolation but also add pressure. He distinguishes confidence in a research direction from trust in a person and says both can involve doubts.
He recalls revisiting LeCun’s research ideas: mapping earlier arguments onto current multimodal work helped him see how they could guide specific questions.
The people in New York’s Washington Square Park remind him of lives outside work: music, dance, chess and rest all show that the world contains more than research.
He appreciates the possibilities New York offers even if he lacks time to try every interest. Skiing is one recent experience he mentions.
They discuss science-fiction portrayals of AI, including systems that expand from language or code into perception of the world. He sees the works as metaphors worth discussing.
He notes that some works do not depict optimistic endings and connects a film’s theme of alienation through work and technology to current debates. This is his interpretation of the film.
Asked about influential books, he recalls works read as an undergraduate and recommends Zen and the Art of Motorcycle Maintenance, with its attention to values, quality and lived experience.
He says sincere communication between people matters. Research and creative work can also communicate across time, allowing later readers to understand their creators’ thinking.
On video generators such as Seedance, he praises their capabilities while explicitly treating rumors about model architecture as unconfirmed.
The host mentions the studio name “Language and World.” He cautions against using a Wittgenstein quotation as a simple proof of a modern model architecture: the philosophical context differs from machine-learning terminology.
He invokes later Wittgenstein’s language games, emphasizing meaning in use. His interpretation raises the question of how language, action and the physical world relate.
He resists equating complex creative activity directly with a single generative model’s training loss. Algorithms can be tools, while creation also involves context and action.
The interview ends with the science-fiction answer “42” to life, the universe and everything: the answer itself cannot replace the work of understanding the question.
More from this channel
1:59:52EN
2:51:18EN
2:51:18ZH → English
2:34:18EN
2:34:18ZH → English
1:09:50ZH → English