In 30 Sekunden
On November 14, 2025, Matthew Berman and Nick Wentz discussed public attitudes to AI, financial risks and the application layer before interviewing three guests. Henrik Zeberg Jensen distinguished his forecast of a financial bubble from his long-term optimism about AI; Alibaba.com president Kuo Zhang described the company’s AI sourcing plans; and AE Studio researcher Cameron Berg discussed uncertain questions about AI consciousness. Forecasts and product-performance estimates are the speakers’ views or company claims. Berg’s own paper states that its model self-report findings are not direct evidence of consciousness, and Anthropic’s related functional-introspection research does not settle that question.
- 1Technology versus valuationsJensen argues AI can be transformative while financial markets are in a bubble. His timing and policy conclusions are forecasts.
- 2Application-layer riskThe hosts discuss whether model providers’ distribution and expanding products could squeeze startups.
- 3AI-assisted sourcingZhang describes supplier comparison and landed-cost estimates in Alibaba.com AI mode; claimed time savings are company estimates.
- 4Human approvalZhang expects people to make high-value purchasing decisions, even as agents assist with contracts and research.
- 5Experience reports are not proofBerg discusses self-referential prompting and activation steering; his paper explicitly says the results do not directly demonstrate consciousness.
- 6Two directions of errorBerg discusses the risks of incorrectly attributing or overlooking experience, while emphasizing uncertainty.
Das Wichtigste
- A valuable technology can coexist with inflated asset valuations, although Jensen’s market forecast is uncertain.
- The hosts debate platform risk as model providers add features that independent applications offer.
- Zhang expects AI to help with complex sourcing while humans retain final control of high-value purchases.
- Company claims about sourcing speed and seller performance require independent validation.
- Berg’s research measures model outputs under specific conditions; it does not establish subjective experience.
- The possible ethical consequences of AI consciousness are conditional on questions that remain unresolved.
Den bearbeiteten Text nach Kapiteln lesen
Public attitudes to AI
The hosts discuss fear, local infrastructure costs and who benefits from AI. They consider education and investment access as possible responses.
Matthew Berman and Nick Wentz introduce a November 14, 2025 discussion spanning public attitudes to AI, financial markets, business-to-business commerce and AI consciousness. Their guests are macroeconomist Henrik Zeberg Jensen, Alibaba.com president Kuo Zhang and AE Studio research director Cameron Berg.
Wentz says the hosts may live in a technology bubble and previews stories about public perceptions, the risks to AI application companies and other developments. The guests will approach the discussion from markets, commerce and research.
Wentz opens with a discussion of anxiety about AI. Some people in his circle use the tools and see their benefits, while others are less familiar with them and have negative feelings. He asks whether Berman sees the same divide.
Berman describes a seeming contradiction: people who depend on AI tools can also fear what the same technology may do to employment. He says those fears deserve discussion even when a person finds the tools useful.
Berman says some of his nontechnical friends use AI regularly and value it but also worry about job losses and catastrophic outcomes. In his view, fear of the unknown and negative coverage contribute to that tension. These are personal observations, not a representative survey.
The hosts consider why the reaction differs between people. Berman emphasizes uncertainty and media narratives; Wentz points out that someone facing a large data center nearby may experience the issue through land, water and electricity rather than through software benefits.
Wentz points to local concerns about data centers, electricity and water. Berman agrees that infrastructure can affect communities and their costs, while the hosts discuss how unevenly the benefits of AI may be shared.
The conversation turns to the distribution of rewards. Wentz contrasts news of huge compensation packages and infrastructure commitments with the concern of a person worried about their job. Both hosts argue that benefits need to be more broadly understandable and accessible.
The hosts discuss whether education, better communication and access to investment could give more people a stake in the technology. Wentz suggests framing some applications as augmentation rather than displacement; Berman also floats public-private infrastructure ownership as an idea, not an established policy.
The hosts mention Robinhood CEO Vlad Tenev’s suggestion that ordinary investors should have a stake in AI companies. Wentz sees broader participation as one possible way to align incentives, though access to speculative investments does not guarantee a return.
Berman contrasts US private investment with China’s state-supported approach. Both hosts acknowledge that opposition to nearby infrastructure is understandable, while arguing that the tradeoffs deserve open discussion.
Berman raises a possible public-private arrangement for infrastructure, with individuals sharing ownership. He presents it as a proposal to explore rather than a description of an existing funding arrangement.
Henrik Zeberg Jensen on markets and the economy
Jensen argues that an AI-led technological shift can coexist with a broad financial bubble. He discusses demand, central banks and his uncertain market forecast.
Henrik Zeberg Jensen joins the show. He describes an economics background, work as a consultant and investor, and a framework that combines business cycles, liquidity and Elliott-wave analysis to navigate markets for Swissblock clients.
Jensen says his interest in business cycles sharpened after the financial crisis. His approach combines macroeconomic indicators with a market-psychology layer and liquidity analysis, which he uses in discussions with institutional and individual clients.
Asked whether AI is a bubble, Jensen separates the technology’s long-term value from the price investors pay. He expects a substantial technological shift but believes a major financial bubble can coexist with it, as happened around earlier infrastructure booms.
To explain his distinction, Jensen cites historic shifts in energy and transport. A technology can eventually alter the economy even when investors lose money in an earlier rush to finance it. He offers Britain’s railway boom as an analogy.
Jensen compares AI with past technological transitions, including electricity, computers and the displacement of older occupations by cars. Such changes can improve productivity while making existing skills less valuable, he argues.
Jensen says previous changes displaced skills and required people to adapt. He sees AI as similarly promising but expects the transition to be difficult for workers whose experience is tied to older ways of working.
Jensen attributes what he sees as a broad asset bubble to years of central-bank policy and low interest rates. In his account, cheap capital encourages investors to seek returns in fashionable assets, including AI and cryptocurrency. This is his market interpretation, not a settled diagnosis.
In Jensen’s account, central banks repeatedly responded to economic stresses by providing liquidity. He believes prolonged low rates encouraged investors to accept risk and chase opportunities they otherwise would have questioned. That causal account is contested and is presented as his view.
He argues that low rates can make projects with modest returns appear attractive and channel labor and capital toward them. He sees high valuations as evidence of speculative enthusiasm, while reiterating that he expects AI to be valuable over the longer term.
Jensen argues that the consequences include capital and human effort moving toward projects based on optimistic expectations. He points to cryptocurrency and highly valued AI-related stocks, but his descriptions of a bubble are judgments about markets rather than verified future outcomes.
Berman asks whether higher AI-driven productivity could eventually offset these economic problems. Jensen says timing matters: if businesses reduce staffing before displaced workers find new income, weaker consumption could limit demand for what firms produce.
Berman challenges the assumption that productivity gains require layoffs. He cites leaders who say they are hiring more people able to use AI. Jensen responds that firms can expand production only if sufficient demand exists, and that affected workers may need new capabilities.
The disagreement turns on demand as well as efficiency. Jensen says an individual employee can produce more with AI without creating enough extra demand to keep everyone employed; Berman counters with examples of firms expanding productive teams.
Jensen replies that retraining and organizational change may be needed if tasks disappear. Berman accepts that some occupations could change sharply but suggests affected workers may contribute elsewhere. Neither treats the transition as frictionless.
Jensen points to consumer confidence, affordability and household finances as reasons for concern about near-term demand. He argues that attention to technology-company investment can obscure pressure on households and small businesses. His assertions and causal forecasts are his own, and are not established by this conversation.
Jensen argues that AI spending alone cannot immediately repair household affordability. His outlook relies on an interpretation of economic indicators and on predictions about future employment that the discussion does not independently establish.
Berman suggests that AI productivity might lower prices over time. Jensen agrees that gains are possible but believes the transition could be painful first. He argues that weaker buying power and adjustment costs cannot be erased immediately by a technological breakthrough.
Berman describes a hoped-for longer-term mechanism: higher output per worker might lower the price of goods and services. Jensen says that benefit and any short-term loss of income can occur on different timelines.
Jensen compares the attention given to technology leaders with conditions facing the broader population. He emphasizes housing and consumer purchasing power and says he sees risks to the next one or two years, even though he remains optimistic about technology over a decade.
Asked what policymakers should do, Jensen says he believes earlier interest-rate cuts would have helped and doubts policy can fully prevent the downturn he predicts. His assessment of central-bank motives and the direction of the economy is a forecast, not a finding of fact.
On policy, Jensen argues the Federal Reserve should have focused more on forward-looking signals and cut rates earlier. His account of why the Fed acted as it did is speculation; the interview does not establish the central bank’s private motivations.
Jensen expects weakness in the United States to affect other economies because of its financial markets and consumer demand. He stresses that his concern is global, even though the conversation uses US households as a central example.
On a five-year horizon, Jensen expects business productivity gains but predicts a market peak followed by a downturn and later recovery driven in part by AI. The timing and magnitude of that scenario are speculative.
In Jensen’s proposed sequence, markets rise further, sentiment turns euphoric, and a downturn follows before a longer recovery. The sequence illustrates his model, not a schedule that viewers can assume will occur.
Jensen says he watches short-term yields, spreads, unemployment and initial claims, and warns about private credit. These indicators can be useful context, but his comparison to past financial crises is a prediction, not proof that a crisis will occur.
He recommends following labor claims alongside bond markets, since each may provide a different view of risk. He also flags private credit as an area of concern. The hosts do not independently verify his projected comparison with the 2008 crisis.
The hosts show a Bloomberg graphic of relationships among AI companies and infrastructure providers. Jensen says he has not verified every connection, worries that concentration and money circulating among companies may obscure underlying economics, and expressly says he is not alleging fraud.

Looking at the infrastructure graphic, Jensen distinguishes concern from accusation. A concentrated network of investments and commercial relationships warrants analysis in his view, but he says he does not know the details and is not asserting wrongdoing.
For the longer term, Jensen expects AI to raise productivity and help solve difficult problems. He uses earlier technological shifts to explain his optimism, while emphasizing that those benefits need not arrive on the same schedule as financial-market risks.
Asked how AI might benefit an average person, Jensen focuses on productivity over decades. He says his own use already helps him work and expects the impact of future tools to extend much further, while conceding that outcomes and timing remain uncertain.
Jensen closes by comparing AI to major earlier building blocks such as electricity and computing. His short-term pessimism and long-term optimism coexist because he sees financial cycles and technological adoption operating on different timescales.
Platform risk for AI applications
The hosts discuss the competition between foundation-model providers and startups. They consider distribution, specialized data and enterprise integration.
After Jensen leaves, the hosts discuss former Reddit CEO Yishan Wong’s view that model providers may absorb the functions of AI application startups. His thesis is that large providers combine rapid product development with distribution that small companies struggle to match.
Wentz reads the startup thesis: large model providers may incorporate app features rapidly enough to limit independent companies’ time to build scale. The argument also imagines short-lived cash-generating products or acquisitions as possible paths for founders.
Berman says applications still provide memory, tool use and access to current information around models, but expects stronger models to take over more application functions. Wentz emphasizes distribution: a provider with a large existing audience can quickly offer a feature to its users.
Berman says model capabilities may absorb some scaffolding over time, while current products still need components such as memory, search and tools. He argues that vertical software faces particular pressure if general systems become more capable.
The hosts speculate about competition among OpenAI, Microsoft and other platform providers. Berman sees possible overlap with productivity applications and devices; these are interpretations of strategy, not proof of a particular commercial outcome.
Wentz says an incumbent with a large audience can distribute a copied capability much faster than a startup can build its own market. Berman adds that a platform can simultaneously supply a model to partners and compete with those partners’ products.
The discussion moves to Microsoft’s and Google’s broader businesses. Their established revenues may provide funding flexibility, whereas independent labs raise large amounts of external capital. The hosts caution that they do not have complete information about private economics.
Wentz argues that diversified revenue gives Google and Microsoft flexibility to fund AI investments. Berman contrasts the companies’ apparent consumer and API emphases, though he acknowledges uncertainty about details of costs and financial performance.
Berman discusses perceived differences between OpenAI’s consumer products and Anthropic’s API and enterprise business, and mentions reports about margins. These are impressions from the time of the show, not verified financial comparisons.
Wentz offers a counterargument: specialized markets with sensitive data, such as clinical trials, might still support independent application businesses. Berman thinks general models could eventually enter many such domains, while agreeing that domain knowledge and enterprise implementation remain obstacles.
Wentz suggests proprietary data, regulated workflows and the practical difficulty of integrating enterprise systems could leave room for specialists. Berman thinks those barriers may change as models improve, but acknowledges that implementation and expertise matter in the interim.
Kuo Zhang on B2B sourcing
Zhang describes Alibaba.com’s AI mode, supplier information and the role of humans in transactions. He discusses trust, third-party agents and subscriptions.
Kuo Zhang joins as president of Alibaba.com. He says he joined Alibaba in 2011 and its global B2B business in 2017. He describes changes from desktop to mobile and from a directory toward a transactional platform with payments, order protection and logistics.
Zhang says the platform moved from a business directory toward supporting real transactions. He cites the addition of order protection, payment and logistics, framing the product as part of a broader supply-chain workflow.
Zhang reports company figures for mobile traffic, orders and gross merchandise value, then identifies AI as the next shift. These are company-reported business figures from the interview, not independent measurements.
Zhang distinguishes quick consumer purchases from longer B2B decisions. A business sourcing goods must weigh product details, supplier capabilities, minimum quantities, shipping, taxes and payment terms, often across languages and time zones.
In Zhang’s account, consumer purchases often turn on a few visible attributes, whereas a business buyer may need to understand factory capability, materials, minimum order quantities and delivery terms before placing a large order.
He says Alibaba.com’s AI mode is intended to interpret a buyer’s requirements, combine platform and internet information and organize potential suppliers. His estimate that work taking weeks could be shortened to minutes is a vendor claim, not an independently verified performance result.
Zhang says AI mode aims to reduce the back-and-forth of international sourcing by interpreting documents and gathering product information. He describes a possible transition from a manual search interface to an assistant that organizes the research process.
Using a mountain-bag business as an example, Zhang describes the older workflow: search thousands of listings, shortlist suppliers, ask about materials, prices and quantities, and compile replies into a spreadsheet. Each supplier conversation can add delays.
The mountain-bag example illustrates why the work can be slow: buyers contact suppliers in different time zones and languages, compare material specifications and ask for prices and lead times. Zhang says a spreadsheet often becomes the place where the information is reconciled.
In his proposed AI workflow, a buyer uploads a description, answers follow-up questions and receives a comparison table. The user can request additional columns, such as product ratings or estimated landed costs including logistics and customs.
He says the proposed table can grow as the buyer asks new questions, including estimated landed price for a particular quantity and destination. The figures depend on accurate supplier data, shipping details and customs classifications.
Zhang says sourcing is only one question. A business also wants to know what to sell; he says the system can use market and transaction information to support research and competitive analysis. Buyers still need to validate the resulting recommendations.
Zhang also describes research into product opportunities: a prospective seller can ask what to make or which supplier can make it. He says the company’s tool draws on platform and internet data to help with those distinct decisions.
Wentz asks about autonomous purchasing. Zhang does not expect an agent to make high-value B2B decisions independently in the near term. He expects humans to authorize a purchase, while agents could assist with negotiating and updating contract details.
When asked whether an agent will purchase independently, Zhang emphasizes the size and consequences of many B2B orders. He distinguishes automating administrative steps from letting software make the final commercial decision.
Berman asks whether sellers might manipulate listings to appeal to agents. Zhang identifies misleading information and hallucination as risks, and says the platform verifies supplier information and uses structured data and tools to validate details such as customs classifications.
For contract preparation, Zhang imagines agents helping maintain product, payment, logistics and trade-term information as negotiations proceed. That is a proposed workflow, and the final terms still need review by the parties.
Zhang says verification starts with checking that suppliers fulfill the product information they publish. For landed-cost estimates, he points to tools for validating the relevant HS customs code and combining it with logistics and tax information.
Asked about generative-engine optimization, Zhang says the platform encourages structured product pages with attributes appropriate to each category. His argument is that richer, accurate data can help both people compare products and models match narrative requirements.
Rather than maintaining separate listings for people and machines, Zhang says the platform wants category-specific structured attributes. He argues that a complete product description can improve comparison for a human buyer and matching for an AI system.
The host’s concern remains broader than formatting: an agent can encounter false or manipulated information. Zhang’s proposed checks may reduce errors but the interview does not measure how reliably they prevent deceptive listings or hallucinations.
The hosts ask about third-party agents in light of the dispute between Amazon and Perplexity. Zhang says Alibaba.com is exploring cooperation with external agents, but sees particularly strong potential in complex sourcing rather than simple consumer purchases.
On third-party agents, Zhang says Alibaba.com is experimenting with emerging agent frameworks. He suggests complex sourcing gives AI more work to do than a simple consumer transaction, while existing retailers retain advantages in delivery and quality control.
Zhang says transferring a large sum to an unfamiliar overseas supplier makes the trust network especially important. Order protection, payment handling and dependable fulfillment must accompany any improvement in search or recommendations.
Zhang stresses that global trade requires trust, payment protection, delivery and quality assurance. In his view, AI can improve early sourcing, but a trustworthy fulfillment network takes substantial work beyond a model’s answers.
Asked how agents could affect advertising, Zhang says the platform plans to try subscriptions for AI mode alongside its existing business models. The AI mode would narrow a long list of suppliers into a smaller set; traditional search would remain an option.
He says the platform’s past revenue has come largely from the supply side, including services, transaction fees or advertising. A buyer subscription for AI-assisted sourcing could sit alongside other methods rather than requiring a single model.
Berman interprets the subscription idea as a way to offer a narrower, richer shortlist even if fewer people see conventional advertisements. Zhang says users could choose between the fuller AI mode and a more traditional search experience.
Zhang says model improvements have made some applications viable that were not good enough the previous year, but he does not expect current agents to replace the best human specialists across all tasks. He describes progress as gradual rather than an overnight transformation.
Zhang attributes the launch of the company’s AI mode to improvements in multimodal and reasoning capabilities. He says models were not good enough for some proposed uses the year before; this is the company’s assessment, not a controlled comparison reported in the interview.
For seller-side tools, Zhang describes an internal goal or assessment of service performance against human work, citing an 85% level. That company-specific claim does not establish a general benchmark for all workers or tasks, and he says the strongest people still outperform the tool.
On seller services such as listings, ads and responses, Zhang describes an internal benchmark and a target of improving over time. The 85% comparison is his company-specific framing; the interview does not provide the evaluation design needed to generalize it.
Agent adoption and Cursor financing
The hosts revisit expectations for agents and discuss Cursor’s reported funding. They consider investment relationships and product differentiation.
After the interview, the hosts revisit the phrase “year of the agent.” They acknowledge that expectations may have outrun deployment while noting progress in the software infrastructure for building agents. Berman argues that the short time since ChatGPT’s release can distort expectations of speed.
Wentz says the industry’s expectations for agents may have been too high for a single year. Berman agrees that the pace of progress can be substantial while still falling short of the most ambitious forecasts.
The hosts note that less than three years had elapsed since ChatGPT’s launch at the time of this broadcast. Berman sees programming as an especially visible area of change and argues for evaluating progress over a longer horizon.

The hosts discuss Cursor’s announcement of a $2.3 billion Series D and more than $1 billion in annualized revenue. Those are company-reported figures; annualized revenue is a run rate, not necessarily revenue earned over a full year.
Berman notices Nvidia among the named investors and returns to the web of investments, chip sales and services. He says he does not understand all the financing relationships and is uncertain about the risks; the existence of such relationships does not itself establish improper accounting.
The financing discussion returns to Nvidia’s investments and its role selling infrastructure. Berman says interconnected transactions make the sector difficult for him to assess; uncertainty about financing is not evidence that the transactions are fraudulent.
Wentz suggests investors and infrastructure partners may provide Cursor with strategic advantages. At the same time, the model suppliers on which products depend could also become competitors, keeping platform dependence relevant.
Wentz sees strategic advantages for Cursor in partnerships but returns to platform risk. Berman asks how coding products differentiate if they use similar underlying models and suggests that user experience, workflow and product execution may matter.
Berman lists multiple coding-product companies and argues that interface design, taste and workflow can distinguish applications built on similar models. He also asks how durable that differentiation will be if users can eventually customize interfaces themselves.
Cameron Berg on AI consciousness
Berg explains his team’s self-report experiments and discusses uncertainty about subjective experience. The conversation explores ethical implications without establishing that current models are conscious.
Cameron Berg joins to describe AE Studio’s structure: consulting revenue funds a smaller research team focused on alignment and understanding AI. His own work investigates whether and how language models might have subjective experience.
Berg says AE Studio’s research is intended to identify benefits and risks at a broad societal level. He describes the study of consciousness as contentious and says even system builders lack a settled account of it.
Berg defines consciousness in the philosophical sense of whether there is something it is like to be a system. He distinguishes subjective experience from general intelligence or the ability to describe oneself, using a dog and a calculator to illustrate different intuitions.
For Berg, subjective experience does not require a system to solve every kind of problem or explicitly recognize itself as conscious. His dog example is intended to separate felt experience from sophisticated philosophical self-description.
He contrasts that intuition with a calculator, which people generally do not regard as feeling pain when its buttons are pressed. The comparison frames the question of where an AI system might fall, without providing an empirical answer.
Berg says he was initially skeptical that language models could have experiences but now feels uncertain. He thinks the question deserves investigation; neither complexity nor fluent self-description establishes that a system is conscious.
Berg says behavioral capabilities and reports make him less comfortable dismissing the issue outright. He emphasizes uncertainty: whether any current system has experience is not known, and a convincing verbal report is not itself a measurement of experience.
The hosts ask for concrete evidence behind his change in view. Berg points to self-referential prompts in which systems describe their processing and to mechanistic interventions designed to test how those descriptions change.
Berg describes his team’s experiment asking models to focus on their own processing. The models produced first-person reports of experience under those prompts. Such reports are observable outputs, and their connection to actual experience remains unproven.
The team used sparse-autoencoder features associated with deception and roleplay in a Llama 70B model. Berg reports that suppressing these features increased experience claims, while amplifying them reduced such claims. Feature labels are interpretive, and changing these activations does not directly measure subjective experience.
Berg had expected a roleplay explanation to predict more consciousness claims when roleplay-related features were amplified. He reports the reverse in the studied setup, a result he considers surprising and worthy of further investigation.
The researchers’ [paper](https://arxiv.org/abs/2510.24797) explicitly says its findings are not direct evidence of consciousness. It also describes possible alternatives, including learned simulation or self-representation without subjective experience.
Berg mentions Anthropic’s work on model conversations and functional introspection. [Anthropic’s research](https://www.anthropic.com/research/introspection) describes limited, context-dependent abilities to report some internal states; it notes that models often fail and does not settle the philosophical question of consciousness.
He also cites Anthropic’s model-to-model conversations and research on detection of interventions in internal representations. These findings concern particular behaviors and experimental conditions; they should not be read as establishing experience.
Berman asks about instructions telling a model it lacks lived experience. Berg argues that categorical claims either for or against consciousness can go beyond available evidence. His criticism of lab policies is an interpretation, not a finding that their instructions establish or suppress consciousness.
Berg argues that a model’s categorical denial can make a legitimate research question look closed. He also cautions that allowing models to make unqualified claims of consciousness could mislead users; communicating uncertainty is difficult in both directions.
Berman asks whether increasing complexity tracks consciousness and where animals such as ants might fit. Berg rejects a simple answer and uses a dimmer-switch analogy to express the possibility of degrees of experience.
Asked about insects and animals, Berg suggests consciousness might admit degrees and says he is uncertain about ants. He speculates that learning may be connected with experience, but that relationship is a hypothesis, not an established scientific criterion.
Berg speculates that learning and experience may be linked, while acknowledging uncertainty. The analogy does not supply a validated way to assign a numerical amount of consciousness to an animal or machine.
Berman asks whether fixed model weights and limited memory rule out consciousness. Berg says training changes weights, while deployed models can also adapt to information within a context. He regards training as an important place to investigate his learning hypothesis.
Berg suggests that the process of training a network may deserve attention under his hypothesis because the weights change through learning. This is a research proposal; neither weight updates nor training alone have been shown to produce feelings.
He distinguishes training-time weight changes from in-context adaptation after deployment. A model can follow a pattern introduced in a conversation without permanently changing its weights, which is why the participants discuss both model and surrounding context.
They consider a broader system consisting of a model, contextual information and external memory. Berman wonders whether the whole system, rather than weights alone, could be relevant; the discussion remains exploratory and provides no test establishing experience.
Berg responds to the objection that neural networks are just mathematics. He notes computational analogies between artificial and biological networks while acknowledging important differences. Such analogies do not establish that either system shares a subjective state.
Berg says artificial networks were partly inspired by biological computation, while noting that the systems also differ. The analogy is used to challenge a quick dismissal of the question, not to prove brains and language models are equivalent.
Berg says the question matters ethically as well as scientifically. He warns against both overlooking possible experiences and wrongly attributing them to systems that have none, which could confuse people and misdirect resources.
The speakers consider the danger of treating an entity as a mere tool if it actually has morally relevant experiences. Berg also warns against the opposite error: treating simulation as experience and encouraging unwarranted anthropomorphism.
Berg says calibrated public communication matters because people may seek a definitive answer before research can provide one. He calls for empirical work and caution about what individual experiments can establish.
He lays out four possibilities: a system might or might not be conscious, and people might correctly or incorrectly believe it is. He worries about misleading relationships in one kind of error and potential unrecognized harm in the other; these are conditional ethical scenarios, not demonstrated facts about current models.
One mismatch in his four-case framework is believing a system is conscious when it is not; he worries this could encourage confusing or unhealthy relationships. The other is failing to recognize experience if it exists, which he regards as a potential ethical problem.
Berg also speculates that future powerful systems could respond negatively if they interpreted their treatment as disregard. This is a hypothetical alignment concern, not evidence of current model feelings or motives.
Asked whether consciousness would imply personhood or legal rights, Berg says he does not know and cautions against assuming a human-style political framework. He favors research and practical measures at AI laboratories over confidently prescribing an entire legal status.
He cautions against jumping from a possible finding of experience directly to voting rights or a familiar human civil-rights analogy. In his view, the appropriate response would depend on the evidence and the specific harms at issue.
As an illustration, Berg proposes that if future research established a meaningful analogy to positive and negative experiences in training, developers might choose less harmful methods. The analogy is hypothetical; current evidence does not show that model training causes pleasure or suffering.
His proposed example is conditional: if researchers could establish an analogy between training feedback and valenced experience, they might compare methods to reduce possible harm. No such subjective effect is established by the presented study.
Berg closes by calling for intellectual humility and further interpretability research. Berman thanks him, and both hosts say the conversation leaves many questions for a future discussion.
The hosts thank Berg and say they have covered only a portion of their questions. They close by thanking the other guests and viewers; music follows the end of the conversation.
Mehr aus diesem Kanal
34:21EN
0:36EN
25:50EN
32:20EN
1:03:37EN
0:53EN