OpenAI DevDay 2023: GPT-4 Turbo, Custom GPTs and Developer Tools

OpenAI’s DevDay brought a long list of announcements. Matthew Berman starts with GPT-4 Turbo, which he describes as faster, cheaper and available with higher rate limits than the earlier GPT-4 offering. This article describes the November 2023 event; its product availability, terms and prices are historical.

OpenAI CEO Sam Altman says GPT-4 Turbo supports a 128,000-token context window, roughly 300 pages of text, and claims improved accuracy over long contexts. Berman welcomes the larger window, especially for coding, but notes that models can miss information in the middle of long inputs and plans to test the claim.

Altman introduces JSON mode for syntactically valid JSON and improved function calling, including multiple function calls in one message. Berman sees these as useful for applications and agents. A beta seed parameter was also announced to make responses consistent most of the time; it is not a guarantee of identical output, and OpenAI also planned log-probability support.

OpenAI introduces retrieval to bring external documents and knowledge into applications. Berman thinks this creates competitive pressure for startups that built retrieval-augmented generation products around the API. The [2023 developer announcement](https://openai.com/index/new-models-and-developer-products-announced-at-devday/) describes retrieval as an Assistants API tool; it does not mean retrieval or application design becomes unnecessary.

Altman says the model’s knowledge cutoff moves to April 2023, an improvement on the earlier cutoff. Berman compares that with the then-new Grok product and its access to information from X, and expects competition around up-to-date information.

The keynote announces API access to GPT-4 Turbo with vision, DALL·E 3 image generation and text-to-speech. Berman is enthusiastic about image understanding and the release of the open-source Whisper large-v3 speech-recognition model, which turns speech into text. These were separate capabilities and interfaces, rather than a single model doing every task.

OpenAI expands GPT-3.5 fine-tuning to the 16K version and invites eligible users to apply for experimental GPT-4 fine-tuning. A separate Custom Models program pairs selected organizations with OpenAI researchers on domain-specific training and post-training. Altman says the program will initially be limited and expensive; Berman sees a potentially substantial enterprise business.

The company announces doubled tokens-per-minute limits for established paying GPT-4 customers and requests for increases through account settings. Berman welcomes this because he has encountered rate limits when using the API with AutoGen. Those limits describe the offer at the time, not necessarily current account limits.

OpenAI also announces Copyright Shield: it said it would defend eligible customers and pay costs for covered copyright claims involving generally available ChatGPT Enterprise and developer-platform features. Altman says API and Enterprise data are not used for model training. The protection has a defined scope and should not be interpreted as a guarantee covering every copyright issue.

For the 2023 launch, GPT-4 Turbo was priced at $0.01 per 1,000 input tokens and $0.03 per 1,000 output tokens: one-third and one-half of the comparable earlier GPT-4 rates, respectively. OpenAI also lowered the new GPT-3.5 Turbo’s rates. Berman suggests routing easier tasks to the cheaper model and using GPT-4 for more complex work.

Microsoft CEO Satya Nadella briefly appears to discuss the partnership. Berman speculates about Microsoft’s ownership and contrasts the partnership with its other open-source relationships; his on-air estimate that Microsoft owned half of OpenAI is not substantiated here. He expects improving open models and orchestration tools to put pricing pressure on proprietary providers.

The keynote says ChatGPT is incorporating newer GPT-4 capabilities and bringing browsing, code execution, data analysis and image tools into a more unified interface. Berman welcomes the removal of the need to choose a tool manually and hopes the system can select an appropriate capability as needed.

Altman frames GPTs as a first step toward assistants that can perform tasks: customized versions of ChatGPT with instructions, additional knowledge and actions. Berman sees a direct competitor to some third-party agent products. A custom GPT is a configured assistant, however, not necessarily a multi-agent team or a separately trained model.

A presenter demonstrates a Zapier GPT connected to a calendar, asking it for the day’s schedule and conflicts. Berman notices another browser window visible behind the demo and wonders whether actions were being prepared in the background. The visible window does not by itself establish deception or explain how the demonstration was staged.

Altman demonstrates GPT Builder by describing a startup mentor in conversation. The interface generates instructions and a name, previews the result, creates an image with DALL·E and offers configuration options. A document can be uploaded as extra knowledge, illustrating a retrieval-based use case that Berman finds straightforward.

OpenAI says users can keep GPTs private, share them by link or make them available within an Enterprise workspace, and announces plans for a GPT Store. Berman compares the idea with an app store and predicts paid distribution and revenue sharing; those details are his expectations at the time, not established terms for every creator.

Berman’s main missing feature is cooperation among specialized agents. He sees systems such as AutoGen as useful for dividing complex tasks among agents and expects the area to develop. He closes by praising OpenAI’s pace of releases while cautioning developers that building entirely on one provider exposes them to platform risk; diversifying models is one way to manage it.