Bridging the Divide: How Community Notes Built a Trusted Fact-Checking System on Disagreement

Keith Coleman, product lead for Community Notes, describes the basic loop: someone on X sees a post they think is misleading and proposes a note, and other contributors rate it. If people who normally disagree with each other find the note helpful, it shows to everyone on X. The goal, he says, is simply to give people more information so they can make better decisions.

Jay Baxter, the founding ML engineer, says a common misunderstanding is that notes appear immediately or by majority vote, both of which would invite bias. Instead the algorithm looks for agreement among people who have disagreed in the past, which he calls bridging. Before the project began in 2020 he says a room of ML engineers would have insisted it needed closed source and fact-checker labels, yet bridging made bootstrapping possible without external ground truth.

Contributors are ordinary people who must meet a few checks, such as a verified phone number, and are randomly selected rather than curated. Coleman recalls colleagues suggesting that journalists go first, but the team deliberately moved away from editorial decisions. Baxter adds that notes add context rather than fact-check, since a true post can still mislead without context.

Coleman says every post can get a note, including those from Elon Musk, government figures and advertisers, and that you earn the ability to write notes through your ratings. On scale, he cites hundreds of notes shown per day, against a Berkeley estimate of roughly 10 traditional fact checks a day. In 2024 about 95,000 notes were seen around 30 billion times, more than double the prior year's 37,000 notes and 14 billion views, with roughly 950,000 contributors.

Baxter says notes on images and videos are automatically matched to other posts with the same media, giving a single note reach across thousands of posts, and the team keeps those matches precise because people love that notes are written for the specific claim. Coleman adds that people who liked a post are notified when a note later appears, which closes the gap between posting and correction.

The helpfulness threshold, fitted with matrix factorization and gradient descent, is 0.4 on an arbitrary scale, which in practice means a sizable majority on both sides of a divide must agree. Only about 7 to 11% of proposed notes are shown, and both guests say the bar is deliberately conservative. Writers whose notes are found not helpful by people who normally disagree can lose their ability to write and must earn it back.

Asked about extreme or highly polarized users, Baxter says if no cross-divide agreement exists, a note may be correct yet unhelpful to show, since entrenched readers will not change their minds. He notes that independent studies found that seeing a note changes people's agreement with a post's claims. The algorithm also stops counting raters who consistently go against the bridging consensus. Coleman adds that the team wants all of humanity participating so it has the data to know what is helpful to humanity.

Contributors are unpaid volunteers motivated by wanting better information in the world. Coleman points to the 2022 US-wide launch, when a note on a White House tweet led the White House to delete it and reissue a statement, and imagines how a contributor with a dozen followers felt about that.

Baxter reports that although the ranking algorithm does not use whether a post is noted, user behavior changes: A/B tests showed 30 to 40% drops in likes and reposts, and external research found 50 to 60% drops in total reposts. Coleman says compounding that drop each generation quickly takes virality to zero. Authors also become 80% more likely to delete a noted post, which Baxter feels mixed about because the best notes then disappear.

Coleman traces the idea to 2016, when he considered joining Twitter during the Trump–Clinton election and saw that people were sharing both good and questionable information there. He joined in December 2016 and spent three years helping turn the company around. He then concluded that existing approaches to misleading information, namely fact-checkers and internal trust and safety teams, could not handle the scale or speed and were not widely trusted.

While managing a large PM team and seeing little of the output he wanted, he came back from paternity leave and asked his boss Kayvon if he could stop doing his job and work on the problem instead. Kayvon was enthusiastic. He started by reading research, then prototyping, then piloting what became Community Notes.

The project began as Birdwatch. After Musk acquired the company he DM'd Coleman to say he loved this Community Notes thing, and Coleman told him that was the very first name on his earliest Figma mockup. The name changed the next day, and Coleman says the team liked the clearer name even though Jack Dorsey joked it was the most boring Facebook-style name.

On leaving a large management role, Coleman says he has made that sawtooth leap several times and rejects the myth that managing more people means more impact. He asks what 16 more pages of OKRs would have produced compared with Community Notes.

Coleman explains that Thermal was an internal Twitter program, proposed by Kayvon, for teams isolated from planning cycles and arbitrary reassignments. Community Notes was the first such project. Its attributes were one clear driver who is effectively a founder, one senior decision-maker outside the team, 100% focus from everyone, and freedom to use its own processes.

With total focus, Coleman says, an idea suggested in the morning could show results the same afternoon, whereas a week's delay from a distracted colleague kills momentum. The team skipped OKRs and set goals for the next meaningful milestone, whether two weeks or three months. Baxter jokes that the OKR planning process took longer than picking a goal and finishing it.

The team started at about five people, with one each for backend, client engineering, ML, design and research. Baxter says that one ML engineer was right while requirements were unclear, but once the goals were clear he ran a Kaggle-style bake-off to bring in other engineers. His first production algorithm was a PageRank variant aimed at manipulation, which could amplify bias when one side had more users.

Real pilot data showed that the main issue was polarization rather than voting rings, so bridging-based agreement won out. The idea drew on prior work that looked for content liked across political sides. The team coordinates through a long-running Google Doc and daily meetings rather than heavyweight task tools, so irrelevant items simply fall off.

Coleman credits the structure, autonomy and speed for the project's existence and says X as a whole retains many of those attributes. Baxter says the mission plus the way of working is powerful for recruiting, recalling that he gave up a safer path in a push-notification project to join. Coleman adds that nobody was assigned to the team: people chose to join and he and the team interviewed every one.

He sees the same opt-in logic in Musk's fork-in-the-road email after the acquisition, asking people to click yes to stay for a hardcore Twitter 2.0. He adds that an experience on a project like Spaces convinced him he would run any future company even leaner. A small team scaled Spaces in two or three weeks, work he says would have taken a year at Twitter 1.0.

On the layoffs, Coleman says the smaller team and reduced process raised the pace of launches and experiments, and remaining people act like owners. Baxter says deleting code is often more valuable than writing it, since small A/B wins can add lasting maintenance costs, and the company had to audit systems after the layoffs. He says you do not need to rewrite everything, only remove the cruft.

Coleman says the team required the product to earn each next stage. They first showed mockups to people across the political spectrum and found readers were open to notes even when critical of their own side. An internal pilot with MTurk-type participants then showed that normal people could write some good notes.

The public pilot began with 1,000 people, and the team did not know whether it would be a dumpster fire. They mocked up a dumpster fire GIF to set expectations but dropped it, since the page already had a lot to explain. The notes turned out to be a mixed bag with gold in it, so the challenge became sifting the gold from the rest.

In 2020, when a usertesting participant sent a prototype screenshot to an NBC reporter, Musk, then just a Twitter user, replied that it was definitely worth trying. Coleman says his support began long before he was involved with the company.

Coleman lists the principles that seemed crazy at first. The product is the voice of the people, not the company, so there is no button to change a note's status. If a problem with a note is bad enough that you want to act, he says, it is a problem with the system. Contributor selection is open and random, with only a verified phone number as a basic criterion.

The second principle is transparency: the code that decides what shows, and all ratings data, is public so anyone can replicate and audit the service. Scoring changes are posted on GitHub and data is published daily. Baxter says this is costly, because it required unusual architecture so the algorithm runs as a script on downloaded data rather than a hosted service.

Baxter says you currently need about 500 GB of RAM, and it takes about a day on one machine, but it is possible and people, including Vitalik Buterin, have verified it. Coleman says breaking any principle, such as adding a black box, would make the product harder to trust.

Coleman names the start of the Israel–Hamas conflict in October 2023 as the biggest stress test. In the first three days there were about 500 notes covering out-of-context imagery and fake battle footage made in a video game, and the median time from post to note was five hours, compared with days for typical fact checking. He says media matching and a speed-up that had cut three hours off the time-to-publish arrived just weeks before.

Baxter says the conflict also answered the criticism that a bridging algorithm would produce no notes in polarized settings. There were plenty, and he suggests there is more agreement across divides than conventional wisdom says. Coleman says people are willing to cross partisan lines to reach accurate information, and Lenny Rachitsky notes that this implies facts still matter.

Coleman says one big lesson from the pilot was anonymity. The team first assumed contributors should use real names, but people feared harassment on controversial topics, and people are more willing to cross partisan boundaries when pseudonymous. Many quality mechanisms in the system keep standards high even with pseudonyms.

On overcoming the existing trust and safety apparatus, Coleman says people were open-minded because everyone knew the existing approach was not working. The team made the product prove itself at every step and brought data for each proposed expansion. Baxter adds that outside research had suggested laypeople could match fact-checkers, but many people did not believe it until it worked.

Asked how the project survived Jack, Kayvon, Parag, Elon and Linda, Coleman says the product's nature helps, since leaders on any side are likely to find output that people who disagree rate as helpful. The team also kept shipping every week and stayed focused despite distractions during the acquisition.

He stresses that cost savings never came up and were never the reason for the project, since the aim was an internet-scale solution to an internet-scale problem. Asked about low ego, he says he sees the work as community service and is attached to the product being helpful.

Coleman says the roadmap is more, better and faster notes, including a recent update to the request-a-note feature that lets anyone add a source explaining why a post needs one. He sees AI and LLMs as a big frontier. Baxter describes the outside Supernotes work, in which an LLM generates variants of proposed notes and a simulated jury of contributors predicts how each would be rated, so that notes are likely to be rated helpful rather than merely generated.

Baxter also imagines agents checking whether a note is supported by its sources, while worrying that raters must still verify the output themselves. Coleman says the open source dream is that the public could build the product, perhaps eventually writing much of the scoring algorithm.

Baxter says users have taken notes in unexpected directions, such as debating Messi versus Ronaldo, which the system models as a polarized topic, and flagging spam. Coleman closes by arguing that notes show people can agree on much more than society assumes, and wonders whether the same approach could find agreement on legislation and policy. He says the team is hiring an ML engineer.