Sam Altman, Gary Marcus and IBM Testify on AI Risks: Matthew Berman Reacts

Matthew Berman reviews selected excerpts from the May 16, 2023 Senate hearing on AI oversight, adding his own reactions. The witnesses are OpenAI CEO Sam Altman, IBM chief privacy and trust officer Christina Montgomery, and NYU professor emeritus Gary Marcus. The hearing was held by a Judiciary subcommittee, despite the video title’s reference to intelligence.

Chair Richard Blumenthal opens with a recording that sounds like him, then reveals that AI cloned his voice and ChatGPT wrote the text. He warns that the same techniques could impersonate public figures and spread false statements. Berman finds the demonstration impressive and the impersonation risk frightening.

Blumenthal compares the state of generative AI to early mobile phones and discusses potential benefits in science and medicine alongside disinformation, discrimination, fraud and job displacement. Berman thinks the technology could be at least as consequential as the internet and says both job losses and new jobs are possible.

Senator Josh Hawley compares the potential of generative AI to transformative inventions, asking whether its effect will resemble the spread of knowledge or a dangerous breakthrough. Berman agrees that AI could change the world and notes that any US restrictions must be considered in an international context.

Altman says OpenAI was founded to pursue AI’s benefits while managing serious risks and describes its nonprofit governance and mission. Berman questions the contrast between the organization’s early openness and the limited disclosure surrounding its later models.

Altman points to uses in learning, productivity and accessibility, including Be My Eyes using GPT-4’s multimodal technology to help visually impaired people. He says current systems cannot yet solve problems such as curing cancer, but sees future potential.

Altman describes pre-release evaluations, external red teaming and dangerous-capability testing, saying GPT-4 spent more than six months in evaluation. He proposes government licensing and testing requirements for models above a capability threshold.

Berman worries that licensing might raise barriers for startups and open-source projects while benefiting companies already at the frontier. He says safety concerns may be genuine but argues that the incentives of companies and advocates also matter.

Montgomery calls for “precision regulation”: rules focused on how AI is deployed in particular contexts, with stronger requirements for higher-risk uses. Her proposed elements include defining high-risk uses, disclosing AI interaction and conducting impact assessments for bias and other harms.

Berman agrees people should know when they are interacting with AI but emphasizes the tradeoff between regulation and the burden on smaller organizations. Montgomery argues that a risk-based, use-specific approach can protect people without unnecessarily impeding innovation.

Marcus warns that generative systems could create persuasive falsehoods at scale, affect elections and markets, and shape opinions through training-data choices. Berman considers misinformation and deepfakes among the more immediate risks and notes that bias can be subtle rather than simply partisan.

Marcus raises examples of harmful chatbot interactions and worries about systems connected to the internet and tools, including Auto-GPT. Berman says the tool was not especially effective in his experience at the time, but more capable autonomous iterations could pose risks.

Blumenthal asks about independent testing and model “nutrition labels.” Altman supports companies reporting strengths and weaknesses as well as independent assessments, and says users should verify model outputs. Berman welcomes transparent benchmarks and outside audits.

Asked about jobs, Altman says effects are hard to predict but expects new and better work. He distinguishes tasks from jobs, while acknowledging that some jobs could be automated and that government and industry must address the transition.

Montgomery says AI will transform many jobs, create others and eliminate some, and urges preparation of the workforce. She points to new AI governance roles at IBM. Berman sees the response as sensible, though prepared.

Marcus supports disclosure but says testing generalization is difficult without knowing what is in the training data: a benchmark may not measure novel performance if similar material appeared during training. He calls for scientists and greater transparency.

On employment, Marcus cautions that past technology-driven job creation does not guarantee the same outcome this time. He uses a hypothetical 50-year horizon for AGI and its potential labor effects; Berman notes that it is unclear whether this was a prediction or an illustrative example.

Marcus presses Altman to state his worst fear. Altman answers that the industry could cause significant harm to the world and wants to work with government to prevent it. Berman finds the answer broad and wishes it were more specific.

Hawley cites research on models trained on media diets predicting survey responses and asks whether such capabilities could be used to tailor political influence. Berman distinguishes predicting opinions from the prior influence of the media people consume, while recognizing the potential for manipulation.

Altman says personalized persuasion and election misinformation are major concerns. He supports disclosure when people interact with AI or encounter generated content, alongside company policies, regulation and public education. Berman agrees disclosure is useful and says synthetic imagery is becoming harder to distinguish.

Marcus says opinionated models might influence users directly and argues that training-data composition affects model behavior. He calls for transparency and scientific analysis. Berman adds a concern that coordinated actors might deliberately seed material into future training data.

Hawley asks whether systems trained on personal data could target individuals’ attention and responses with unprecedented precision. Berman compares the concern to existing recommendation algorithms, while worrying that more targeted outputs could shape opinions without users noticing.

Altman agrees the risk deserves attention and says OpenAI was not pursuing an advertising-based model or building user profiles for that purpose at the hearing. Berman is skeptical of the company’s incentives and of the durability of that position.

Asked about liability and Section 230, Altman says he does not yet know the right legal answer and wants to work on a new framework. He says model providers, users and intermediaries all bear responsibility. The exchange is a policy discussion, not a settled legal conclusion.

Senator Amy Klobuchar raises election misinformation, including false polling information. Altman says the industry and government should work together and describes refusal policies and monitoring for large-scale generation. He distinguishes generation tools from the social networks that can spread content; Berman observes that both stages matter.

Klobuchar asks about compensating news producers and the risk to local journalism. Altman hopes AI tools can help news organizations and notes that the version of GPT-4 discussed was trained on data through 2021. Berman argues that browsing-enabled systems can access more recent information.

Senator John Kennedy asks whether powerful AI tools should be licensed and overseen by a capable agency. Altman and Marcus support that direction; Montgomery favors building on existing regulators. Berman again worries about the effect of licensing on startups and open source.

A senator asks how to decide if a model is safe and mentions both human-labeled harmful content and constitutional AI. Altman responds by explaining iterative deployment: he wants the public and institutions to learn about systems’ limits and benefits while models are still comparatively weak, subject to a safety threshold before release.

Montgomery says the proposed EU approach resembles her risk-based, use-specific framework, with safeguards and disclosures for election uses. She argues existing regulatory authorities should address current harms, while a senator notes that those bodies may lack resources and powers.

Asked about international coordination, Marcus mentions the UN, UNESCO and OECD but says international politics is not his specialty and he cannot prescribe the exact institutional model. Berman notes that no single national government can regulate every country.

Senator Mazie Hirono asks who decides what counts as a harmful request. Altman cites violent content, encouragement of self-harm and adult content, explaining that the company refused a broad category where it could not reliably distinguish associated risks. Berman questions who should make these decisions.

Hirono asks how a licensing scheme could address the breadth of potential harms. Altman suggests generated images should be clearly identified, and says his proposed licensing system is aimed at the more powerful systems that may emerge on the path toward AGI. Berman questions how durable watermarks would be.

Another question concerns underrepresented languages. Altman describes OpenAI’s partnership with the Icelandic government and says GPT-4 was better in many languages than previous models. Berman welcomes efforts to include more languages and perspectives.

Senators ask about the danger that a regulator could become overly burdensome. Altman cites slowing US development relative to other countries and imposing costs that small startups and open-source projects cannot bear. Berman welcomes that acknowledgment while remaining skeptical about implementation.

Altman describes OpenAI’s data practices at the time: API inputs were not used for training and were retained for 30 days for trust and safety, while ChatGPT users could opt out of training. Berman discusses the implications of personal data for AI businesses. These are historical statements, not current policy guidance.

Asked when self-aware AI might arrive, Marcus says he does not know and that self-awareness itself is not well understood. Berman agrees that defining consciousness is difficult and suggests a hypothetical experiment involving training data that excludes discussion of consciousness.

Hawley asks about the open letter calling for a pause. Marcus, a signatory, clarifies that it focused on systems more powerful than GPT-4, not all AI research, and supported safety research. He says his own emphasis is on pausing deployment until there are good safety cases, especially for systems reaching millions without external review.

Altman says OpenAI had waited more than six months after GPT-4 training before deployment and was not then training GPT-5 or planning to do so in the next six months. He favors audits, red teaming and safety standards over a fixed calendar pause, while allowing that specific discoveries might warrant stopping.

Montgomery says ethics and responsible technology should be prioritized, but questions whether a development pause is practical. Berman agrees that a pause would be hard to enforce globally, and the excerpt turns to the risk of regulators being captured by the industries they oversee.