Grok 3’s Brief Misinformation Filter and the Question of AI Bias

Matthew Berman examines an instruction that told Grok to disregard sources linking Elon Musk and Donald Trump to misinformation, and xAI’s explanation of how it reached users.

MBMatthew BermanPresenta · Forward Future
Canal
Matthew Berman
Publicado
25 feb 2025
Vídeo
7:59
Lectura
3 min
Diapositivas
5
Original
inglés
Abrir en YouTube
Al desplazarte, el vídeo se reduce a la esquina inferior derecha y sigue reproduciéndose. Los subtítulos siguen tu idioma de lectura.
Idioma de lecturaDisponible en 2 idiomas
Este artículo aún no está en español, así que se muestra el original en inglés.
RESUMEN

En 30 segundos

In this February 25, 2025 video, Matthew Berman criticizes a Grok 3 prompt instruction that filtered certain sources about Elon Musk and Donald Trump. xAI engineering lead Igor Babuschkin said an employee added the instruction, a reviewer missed it in a larger change, and it was reverted when users pointed it out. Berman argues the episode exposes the influence of human choices on model behavior; the displayed chatbot answers do not independently establish a ranking of misinformation spreaders. The video includes a paid Jotform sponsorship.

  1. 1The disputed instructionScreenshots showed a Grok instruction excluding sources that linked Musk or Trump with spreading misinformation. Its scope concerned those sources, not every mention of either person.
  2. 2xAI’s explanationBabuschkin attributed the change to an employee who previously worked at OpenAI and said it had been reverted after users pointed it out.
  3. 3Where bias can enterBerman argues that people influence outputs through data selection, training and post-training decisions.
  4. 4A missed reviewBabuschkin said a reviewer missed the instruction because it appeared in a larger pull request; Berman calls for closer scrutiny.
  5. 5An earlier political disputeBerman cites a 2024 exchange between Altman and Babuschkin about Grok and training data, presenting their claims as views rather than verified explanations.

Conclusiones

  1. A Grok instruction told the system to ignore sources saying Musk or Trump spread misinformation.
  2. xAI’s Igor Babuschkin said the change was made by an employee, missed in code review and reverted after users raised it.
  3. A chatbot’s answer to an open-ended misinformation question is not a validated ranking.
  4. Berman argues that training data, filtering and post-training choices can shape model behavior.
  5. An earlier exchange between Babuschkin and Sam Altman illustrates competing claims about political bias rather than settling them.
TEXTO COMPLETO

Lee el texto editado por capítulos

Solo la transcripción
01

A disputed Grok instruction

Users shared a prompt that excluded certain sources about Musk and Trump. Berman reviews the screenshots and xAI’s initial response.

Grok 3 was promoted as a truth-seeking model, but users discovered an instruction that placed a restriction on certain sources. People had been asking it who spreads the most misinformation on X, and some shared answers naming Donald Trump and Elon Musk. Those answers are chatbot outputs, not a rigorous or independently validated ranking.

The screenshot I am looking at shows a prompt directing Grok to ignore sources that say Musk or Trump spread misinformation. Another user, Pliny the Liberator, reported reproducing the instruction. The language is specific to sources making that allegation; it does not establish that all discussion of either person was suppressed.

The disputed source-filtering instruction
DIAPOSITIVA 1 / 5The disputed source-filtering instructionThe displayed prompt includes the instruction about sources linking Musk or Trump to misinformation.
A user reports reproducing the instruction
DIAPOSITIVA 2 / 5A user reports reproducing the instructionThe post requests an explanation after showing another apparent instance.

xAI engineering lead Igor Babuschkin acknowledged the issue and attributed the change to a former OpenAI employee. He said it was reverted once users pointed it out. His account identifies an employee and describes what happened; it does not independently establish that Musk ordered the change, and Babuschkin said Musk was not involved.

02

Responsibility and model bias

Berman questions the process behind the change and considers where human choices can influence model behavior. His assessment of the employee’s motives is an inference.

I find the explanation troubling. A prompt that filters sources for a politically sensitive question can affect what the model tells users, and I question why such a change could be deployed. My speculation about what the employee expected or intended is not proof of that person’s private reasoning.

I think changes to a system instruction should receive careful review before deployment. A team may move quickly, but speed can create serious problems when a small instruction changes which evidence an assistant considers. This incident challenges the idea that a product marketed as a truth seeker is automatically free of bias.

Human choices run throughout an AI pipeline: people select and filter training data, configure training, and decide how to align and deploy a model. I believe those choices can introduce biases, even when a company promises a different political balance from its competitors. The existence and direction of any particular bias still need evidence and careful testing.

03

Review failure and earlier dispute

Babuschkin says a reviewer missed the change in a larger pull request. Berman revisits an earlier exchange about political bias and concludes that AI systems reflect human decisions.

Babuschkin describes code review
DIAPOSITIVA 3 / 5Babuschkin describes code reviewThe displayed exchange says the line was missed in a larger pull request.

Babuschkin said he wished the employee had asked him before making the change. Asked about code review, he said the instruction was part of a larger pull request and the reviewer missed it. I would want especially close attention to files that govern a system prompt, because a small change can have broad effects.

Babuschkin’s account of the motive
DIAPOSITIVA 4 / 5Babuschkin’s account of the motiveHe says the employee saw negative posts and thought the instruction would help.

Asked why the employee added the instruction, Babuschkin said they had seen negative posts on X and thought it would help. That is his account of the motivation. I find that rationale alarming: suppressing relevant sources because of an unpopular answer is a poor way to pursue reliable information.

I also revisit a November 2024 exchange. Sam Altman had posted a comparison in which Grok 2 selected Kamala Harris over Trump when asked who would make a better president, contrasting it with a ChatGPT response. Babuschkin responded that filtering biased ChatGPT material from training data had been difficult and claimed this explained Grok’s answers. His assertion is not independently established by that exchange.

An earlier debate over model bias
DIAPOSITIVA 5 / 5An earlier debate over model biasBabuschkin’s post responds to Altman’s comparison and makes claims about training data.

My conclusion is that Grok should not be assumed to be a perfectly neutral “ultimate truth seeker.” I see it as a potential counterweight to what its creators regard as bias in other models, but that is my interpretation. Whatever system people choose, they should understand that humans make decisions throughout its development and should examine its claims and evidence critically.

Más de este canal

12 diapositivas34:21EN
Matthew Berman· 27 oct 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyIA empresarial+2
0:26EN
Matthew Berman· 25 oct 2026Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.Human-AI CollaborationFuture of WorkAI Strategy
2 diapositivas0:36EN
Matthew Berman· 25 oct 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.EvaluacionesAgentes de programaciónOpen-weight models
2 diapositivas0:21EN
Matthew Berman· 24 oct 2026An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.BenchmarksEvaluacionesGenerative media
6 diapositivas25:50EN
Matthew Berman· 24 oct 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 diapositivas32:20EN
Matthew Berman· 23 oct 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1