Reddit’s API Changes, AI Training Data and the Open Internet

SUMMARY

In 30 seconds

In June 2023, Matthew Berman responded to Reddit’s API pricing controversy and the protest by many communities. He argued that valuable training data could give platforms an incentive to restrict access, reducing the range of third-party services, and called for users to share more of the value they create. His account contains an unverified theory about Reddit and OpenAI and an overbroad statement about content ownership. Reddit’s contemporaneous policy retained some free API access and said users own their content; later deals do not prove the alleged earlier coordination.

1Reddit’s API disputeBerman describes the 2023 protest and the Apollo developer’s estimate that the new API fees would cost roughly $20 million annually.
2AI and data incentivesHe argues that access to distinctive data can be valuable to model developers and may lead platforms to charge for access.
3Users create the contentBerman calls for users to have more control over, and compensation for, their contributions. Ownership and licensing should not be treated as identical.
4An unverified theoryBerman discusses a writer’s speculation about Reddit and OpenAI. He presents no evidence of the alleged coordination or exclusive access.

Takeaways

Reddit’s 2023 API pricing changes prompted protests and led the Apollo developer to say the app would shut down.
Berman argues that demand for distinctive AI training data may encourage platforms to restrict or monetize access.
Reddit said qualifying noncommercial and lower-volume uses could continue to use its API free, so the policy did not end all third-party access.
Berman advocates stronger user control and compensation for contributions, while rights and licenses differ by platform and jurisdiction.
The suggested preferential-access theory was speculation; later partnerships do not establish an earlier secret arrangement.

TopicsOpen-source ecosystems Data quality & curation Privacy & data governance AI Strategy Creator Economy

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

The API pricing controversy

Berman recounts Reddit’s 2023 API changes and the Apollo developer’s cost estimate. He sees a broader shift in access to platform data.

I think the Reddit upheaval of June 2023 signals a change in the open internet, and I connect it to artificial intelligence. Reddit announced charges for high-volume API access, making some third-party applications hard to operate. Apollo’s developer estimated that the proposed price would cost the app about $20 million a year and said he would shut it down.

Twitter had also started charging for API access. I expect more companies holding valuable data to restrict access. But the Reddit change was not a complete shutdown: [Reddit said in June 2023](https://redditinc.com/news/apifacts) that qualifying noncommercial and lower-volume uses, including moderator tools, could continue with free access under its rules.

AI training data and user value

He argues that demand for unique datasets may encourage platforms to charge or restrict access. He calls for stronger user rights and compensation.

My explanation is that large language models require substantial training data, and distinctive, high-quality datasets can give developers an advantage. As AI companies seek new sources, I think platforms such as Reddit, Stack Overflow and Twitter have more reason to protect or monetize their data. That is an argument about incentives, not a measured proof that every dataset has increased exponentially in value.

For years, platforms opened APIs so independent developers could build applications, creating benefits for users, developers and the platforms themselves. A new player now wants access to those conversations for AI. I worry that platforms will sell access or build their own models instead, leaving fewer opportunities for independent developers and fewer useful features for consumers.

Reddit’s anticipated IPO and its interest in monetization, in my view, helped make its data more valuable. I support AI and believe it can bring enormous benefits, but I did not expect this possible side effect. The relationship between API rules, data licensing and access to public websites is more nuanced than saying an entire site has closed.

Users write the posts and comments that create much of these platforms’ value. I think they should have greater control over that material and be paid for their contributions, and I hope the Reddit blackout leads to meaningful change. I am not sure it will, but users collectively create the value that makes these platforms useful.

I say that Reddit and Twitter own everything users post, but that is too broad. Reddit’s [April 2023 explanation of its terms](https://redditinc.com/news/2023apiupdates) says user content is owned by the people who created and submitted it; its user agreement grants Reddit a license to use content. Ownership, contractual licenses and control of access are distinct, and rules on another platform may differ.

I point to revenue sharing with creators on Twitter and YouTube as steps toward rewarding contributions, and argue there is room for a platform to challenge services that depend on user work without paying contributors. My proposal is a business model where users receive a share of the value of the content and data they create.

A theory and proposed responses

Berman raises an unverified theory about relationships between Reddit and OpenAI. He concludes with proposals for new business models, protest and open-source competition.

I discuss [Dmitry Mazin’s 2023 essay](https://www.cyberdemon.org/2023/06/14/reddit-moat.html), which speculates that ties among investors and executives might advantage OpenAI’s access to Reddit data. I go further in suggesting that OpenAI may already have had preferential access and influenced Reddit’s API decision. Neither the video nor the essay provides evidence establishing that alleged arrangement, exclusive access or coordination; the essay itself describes its argument as a theory.

Later context: [Reddit and OpenAI announced a partnership in May 2024](https://openai.com/index/openai-and-reddit-partnership/) under which OpenAI would access Reddit’s Data API, and [Reddit separately announced a Google partnership](https://redditinc.com/news/reddit-and-google-expand-partnership) in February 2024. These later announcements do not establish the alleged secret or exclusive 2023 arrangement. OpenAI disclosed that Sam Altman was a Reddit shareholder and said its COO led the deal, which its independent board approved.

I see an opportunity for entrepreneurs to build services that reward users for their contributions. I support the Reddit blackout, though I am uncertain whether it will produce change, and I think a thriving open-source ecosystem can help provide competition to closed AI companies. I want readers to consider what kind of internet and compensation model they want in the future.