OpenAI Departures, Developer Tools and Synthetic Media: An August 2024 AI Roundup

In this August 2024 news roundup, I set aside the week’s speculation about OpenAI’s “Strawberry” project and cover leadership changes, developer tools, model releases and synthetic media. These announcements and comparisons reflect their original publication date, not a current ranking or feature list.

OpenAI cofounder Greg Brockman said he was taking a sabbatical through the end of 2024; he had not resigned. Cofounder John Schulman left for Anthropic, and product executive Peter Deng also departed. The reports did not establish that all three events had the same cause.

Schulman said he wanted to focus more deeply on AI alignment and return to hands-on technical work. I find his move to a competitor notable, but he explicitly said a lack of support for alignment research at OpenAI was not his reason for leaving. My questions about his motivation are speculation, not evidence contradicting his statement.

Schulman also thanked colleagues for their support, and Sam Altman publicly thanked him. I do not have evidence of a particular dispute behind the move. Brockman’s announced leave and the two departures are distinct developments, although together they drew attention to leadership turnover.

A 404 Media report described internal Nvidia materials about collecting online video for AI training, including from YouTube. Nvidia defended the practice as consistent with copyright law. The legal status of a particular use depends on facts and applicable law; a company’s defense and a news report are not a court ruling.

As a creator, I find it uncomfortable when a company uses work that took substantial time and money to make to develop another product. I understand the comparison with people learning from publicly available material, but I distinguish that from corporate-scale training. Viewers may reasonably disagree about the analogy and how rights should work.

I would at least like creators to have a meaningful choice about use of their work. Websites can use robots.txt to signal preferences to crawlers, although that mechanism is not a universal legal or technical solution for every AI-training use. The question of consent and compensation remains contested.

LangChain announced LangGraph Studio, an interface for visualizing and debugging agent applications. Its demonstrations show a graph of steps, messages and execution, with the ability to inspect or adjust a run. I like the visual approach because it makes complex workflows easier to examine.

OpenAI also announced Structured Outputs in its API, allowing developers to specify a JSON schema and use strict function definitions. Structured format makes it easier to connect generated content to applications. It is different from merely asking for JSON in natural language: the developer supplies the schema.

OpenAI reported a 100% result for its specified complex-schema evaluation using GPT-4o-2024-08-06 with strict Structured Outputs. That is a result on that evaluation, not a guarantee that every request completes, that every schema is supported, or that values are factually correct. Refusals and incomplete responses still need handling.

One possible use is generating interface elements from user intent, such as a signup screen or a stock widget. I find the idea of interfaces created when needed interesting. A schema can help structure the result, but generated code and data still require validation.

Mistral announced customization on La Plateforme for models including Mistral Large 2 and Codestral, using prompts, examples or fine-tuning with a dataset. It also introduced an agent interface and a stable client SDK. I see the agent interface as closer to predefined assistants than to the more elaborate agent frameworks shown elsewhere.

The Qwen team released Qwen2-Math open-weight models. Its chart reports 84.0 on the MATH benchmark for Qwen2-Math-72B-Instruct under the stated zero-shot evaluation setup. That is the team’s specific 2024 comparison, not proof the model is best at every mathematical task or a current leaderboard.

The team also compares model sizes and other open-weight systems. I have not tested it myself. Reports and rumors about “Strawberry” provide context for interest in mathematical reasoning, but they are not evidence of that unreleased project’s actual capabilities.

A GPT-4o voice clip attracted attention after the model unexpectedly shouted and then appeared to imitate a user’s voice. OpenAI’s 2024 system card documented rare unintended voice-generation behavior during testing and described safeguards. The clip is an example of a particular failure, not evidence that ordinary users’ voices were routinely cloned.

I also show an image circulating online that I describe as AI-generated using FLUX. It looks convincing to me, including text and details, and another example animates an image with video-generation tools. I worry about synthetic media misleading people during elections; visual realism alone cannot establish the origin of any particular image.

Finally, I discuss Deep-Live-Cam, a project demonstrated using a single reference image for live face swapping. Examples use the faces of public figures. The output is synthetic, and matching a face to a moving body can still produce visible inconsistencies; I had not independently tested the tool at the time.

I find the live demonstrations striking and consider making a tutorial. Alongside creative uses, face and voice imitation can mislead viewers about who actually appeared or spoke. The roundup ends with the need to assess these examples as demonstrations rather than assume that each result is flawless or representative.