AI News in August 2023: CAPTCHAs, LK-99, Medical Models and Generative Agents

In this August 2023 roundup, I start with a study of CAPTCHAs: those image selections, puzzles and other tests designed to distinguish people from automated traffic. The researchers compared human and automated performance on several challenge types and found that bots could be faster and more accurate on some of the tested tasks. That result concerns the evaluated systems and challenges, not every CAPTCHA or every anti-abuse defense.

Elon Musk argues that subscription payments can make large-scale bot activity more costly on X. In my own experience running companies, raising the cost of spam can help discourage it. This is an economic argument, not proof that a credit card or subscription by itself eliminates automated abuse.

LK-99, a material that a South Korean team claimed could superconduct at room temperature, attracted enormous attention. By the time of this video, independent work discussed in Nature had found strong reasons to doubt that interpretation. The partial levitation in the original demonstration was not enough on its own to establish superconductivity.

Researchers also examined the reported drop in resistivity. An impurity, copper sulfide, undergoes a transition at a similar temperature and can explain that feature without requiring a superconductor. Although the initial claim was disappointing, I enjoyed watching so many people become interested in materials science and the process of checking a remarkable result.

Unitree introduced a humanoid robot at a quoted price around $90,000, while I compare Boston Dynamics' Spot at $75,000 with a Unitree quadruped advertised from about $1,600. These were 2023 figures for different products and configurations, not like-for-like performance comparisons. Unitree attributed some cost reduction to its own lidar; I am unsure how much that alone explains the difference.

OpenAI acquired Global Illumination, the team behind a Minecraft-like open-source game project. OpenAI said the team would work on core products including ChatGPT. I wonder whether the acquisition is primarily about bringing in talent or whether its game-building experience may have broader uses; that is my speculation, not the company's announced reason.

Reports at the time estimated that operating ChatGPT cost $700,000 a day and speculated about OpenAI's finances. The figure is an outside estimate, not independently established here, and does not establish that bankruptcy was imminent. I am skeptical of the bankruptcy prediction because of ChatGPT's popularity, OpenAI's backers and the company's continued investment; that is my judgment, not a financial forecast to rely on.

Turning to health, I discuss Microsoft research on GPT-4's ability to structure clinical-trial criteria and answer medical questions. Such tasks could eventually help researchers work with complex studies or recruitment. Performance on a dataset or a narrow research task does not show that a model can diagnose, treat patients safely, or replace a clinician.

I also mention DoctorGPT, a Llama 2 7B-based project whose creator claimed it could pass the US medical licensing examination and run locally. The claim needs independent evaluation, including exactly which questions and conditions were used; it does not mean the model is a licensed doctor. I am optimistic that AI could improve access and cost, but those outcomes require real clinical validation.

Stability AI introduced StableCode, a coding-oriented model with roughly three billion parameters and a version supporting a 16,000-token context. The intended use includes integration with development tools such as Visual Studio Code. I like the possibility of making developers more productive, although generated code still needs to be checked and tested.

OpenAI described using GPT-4 to help build and apply content policies. Policy specialists first label a small reference set, then GPT-4 independently labels the same examples after reading the policy. Experts examine disagreements, ask about the model's reasoning, clarify ambiguous rules and repeat the process.

I see potential for moderation teams, but the claim that a tool is universally the best is not established by this example. OpenAI's own discussion notes bias and the need for monitoring, validation and human involvement; experienced reviewers outperformed GPT-4 in its reported comparison. The workflow is a way to refine a policy, not a guarantee of correct moderation.

The US Department of Defense announced a generative AI task force to assess and support responsible use of the technology across the department. I point out that military applications can include both defensive and offensive uses, though the announcement itself does not establish that this one group controls every such application.

I discuss Microsoft Azure ChatGPT as an enterprise deployment approach for organizations concerned about sending private information to a public chatbot. An Azure-hosted solution can be configured with enterprise controls, but it is still a cloud deployment: privacy and security depend on architecture, access controls and service terms, rather than following automatically from the product name.

Runway updated Gen-2 with an extension feature for generated video. The demonstration starts from a text prompt, an image, or both, and adds four seconds at a time. The narration calls the upper length 18 seconds, while the embedded tutorial says three extensions yield 16 seconds; the figures are inconsistent in the source, and these are historical product details.

For the AI video of the week, I show a clip suggested by gingerbot 5000 and posted by fugitive. I think it may have been made using WarpFusion, though that is my inference rather than a confirmed production credit. The example illustrates the creative effects that video tools were beginning to make accessible.

A report says Google and Universal Music were in early discussions about licensing artists' voices for AI-generated songs, potentially allowing fans to create music with participating artists' permission. These were talks, not a finalized universal licensing scheme. I contrast the idea with the removal of a viral song using AI-generated Drake and The Weeknd vocals, and hope artists can choose whether to participate.

I expect AI-generated music to enable more people to create, even if much of the resulting volume is low quality. A few exceptional pieces could still make the technology worthwhile. That is a prediction about a developing creative field, not a measurement of how it will turn out.

The Generative Agents research project from Stanford and Google simulated characters that plan routines, interact and form apparent social relationships in a small virtual environment. Its authors released code, and a16z published a related simulation inspired by the work. These are designed simulations of behavior, not evidence that the agents experience relationships as people do.

I can imagine such simulations making game characters more dynamic: an NPC in a game like Skyrim or Grand Theft Auto might continue activities outside the player's immediate view. That is a possible design direction, not a feature established by these research demonstrations.

Google was adding generative search features such as explanations and summaries of web pages. I can see the convenience for users, but I worry that answers presented directly in search could reduce visits and revenue for publishers. Similar concerns could arise for AI summaries of YouTube videos; the scale of any effect would have to be measured.

A separate report discussed possible AI writing features for ChromeOS, including rewriting selected text, preset prompts and continuing a user's writing. At the time this was reported as an unannounced product, so it should be treated as a proposal rather than a confirmed release or promise about current availability.

Finally, an Associated Press report on red teaming highlights concerns that competition could push AI systems to market before their risks have been adequately tested. Language models learn patterns from large datasets rather than following only hand-written rules, which makes their behavior difficult to exhaustively predict. I believe security, training-data manipulation and other safety issues deserve serious attention, even as companies invest in safeguards.