GPT-4 Before Its Release: Rumors, Expectations, and What Happened Next

SUMMARY

In 30 seconds

Before GPT-4’s March 2023 release, Matthew Berman discussed rumors about its size, possible modalities, Microsoft’s investment, and Bing. He also played excerpts of Sam Altman cautioning against inflated expectations. Subsequent official sources confirm that GPT-4 accepted images as well as text, and Microsoft said the new Bing used GPT-4; the exact model parameter count and the video’s asserted investment terms were not established by those announcements.

1Parameter rumor rejectedAltman dismisses a widely circulated size rumor; GPT-4’s actual parameter count was not disclosed in its technical report.
2A text-only predictionBerman predicts text-only operation, but OpenAI later announced image and text inputs with text output.
3Microsoft’s investmentThe video cites particular financial terms; Microsoft’s public January 2023 announcement described a multiyear, multibillion-dollar investment without confirming those terms.
4Bing speculationBerman suggests Bing was likely based on GPT-3.5. Microsoft later said the new Bing used GPT-4.
5Managing expectationsAltman says the GPT-4 rumor cycle has inflated expectations and cautions that OpenAI did not have AGI.

Takeaways

The video was published on February 28, 2023, before GPT-4’s March 14 release.
Sam Altman rejected a widely circulated parameter-count rumor; OpenAI’s technical report did not disclose GPT-4’s parameter count.
Contrary to Berman’s text-only prediction, GPT-4 accepted image inputs as well as text, though image input was initially a research preview.
Microsoft later confirmed the new Bing used GPT-4; the video’s GPT-3.5 suggestion was speculation.
Microsoft announced a multiyear, multibillion-dollar partnership in 2023, without confirming the video’s specific 49% stake claim.
The video’s cost, training-time, and economic projections should be read as historical claims rather than established technical measurements.

TopicsOpenAI Foundation Models AI Strategy History

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

Parameter size and efficiency

Berman reviews rumors about model size and Altman’s response. He speculates about the economics of training and serving a model.

In late February 2023 I collect what I can find about the upcoming GPT-4 release. At the time, claims about its size, abilities, and launch date are circulating widely. This is a pre-release snapshot: later official announcements answer some questions and contradict several predictions.

GPT-3 was described as having 175 billion parameters, and one rumor placed GPT-4 at 100 trillion. In an interview excerpt, Sam Altman rejects a viral graphic suggesting a dramatic size increase. I infer that OpenAI may prioritize efficiency and a similar or somewhat larger model, but that was speculation: the [GPT-4 technical report](https://cdn.openai.com/papers/gpt-4.pdf) did not disclose its parameter count.

I discuss the cost of training and running large models, and suggest efficiency will matter if a chatbot is to compete economically with search. I also assert that GPT-3 took a year to train and was trained only once, and cite Megatron-NLG as a comparison. Those specific training and performance claims are not established by the evidence presented in the video; parameter count alone does not determine capability, cost, or training duration.

Modalities and the Microsoft partnership

He predicts text-only operation and discusses Microsoft’s investment and computing infrastructure. Later official announcements clarify the model’s visual input and leave the stated stake unconfirmed.

I predict that GPT-4 will remain text-only and say video generation will not be included. The later [OpenAI launch announcement](https://openai.com/index/gpt-4-research/) contradicts the text-only prediction: GPT-4 accepted image and text inputs and produced text, with image input initially a research preview. Accepting an image is distinct from generating an image or video; the launch description did not make GPT-4 a text-to-video model.

I discuss Microsoft’s partnership with OpenAI, asserting a $10 billion investment for a 49% stake and suggesting much of it came as Azure credits. Those specific terms should not be treated as confirmed: [Microsoft’s January 2023 announcement](https://blogs.microsoft.com/blog/2023/01/23/microsoftandopenaiextendpartnership/) described a multiyear, multibillion-dollar investment and expanded collaboration, without confirming the asserted stake or credit breakdown. I also note Microsoft’s earlier supercomputer work with OpenAI.

Coding and release timing

Berman discusses potential coding applications and speculation about Bing and GPT-4’s release. Microsoft later identified GPT-4 as part of the new Bing.

I expect code generation to be an important application and point to GitHub Copilot as an existing example of AI-assisted programming. That is a forecast about how investment might affect tools, not proof that every feature discussed was part of the yet-unreleased GPT-4.

I speculate that Bing Chat is likely a customized GPT-3.5 system and say the GPT-4 release date is unknown. After the video, [Microsoft said](https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2023/04/RAI-for-the-new-Bing-April-2023.pdf) the new Bing included GPT-4, and OpenAI announced GPT-4 on March 14, 2023. The video’s assessment of Bing was therefore an incorrect pre-release inference.

In interview excerpts, Altman says the model will be released when OpenAI is confident it can do so safely and responsibly, and warns that the rumor cycle is creating inflated expectations. He says OpenAI did not have an actual AGI. I interpret the remarks as an attempt to temper anticipation and close by saying I look forward to reporting on the release.