AI News in June 2024: SSI, Tree Search, Meta, Claude and ARC Prize

In this June 2024 AI news roundup, I discuss Ilya Sutskever's new company, an AI infrastructure announcement, a tree-search paper, several Meta research releases, Groq speech transcription, Claude 3.5 Sonnet and the ARC Prize. These announcements and performance comparisons reflect the state of the field at the time.

After leaving OpenAI, co-founder Ilya Sutskever announced Safe Superintelligence Inc. (SSI) with Daniel Gross and Daniel Levy. The company said it would pursue safe superintelligence as its single focus, pairing capability and safety research. I welcome another team making safety central to its stated mission, though a mission statement is not proof of a particular safety outcome.

I read SSI's claim that its business model will insulate safety, security and progress from short-term commercial pressures, and contrast that with my concerns about OpenAI's incentives. Former OpenAI safety researcher Jan Leike had criticized the balance between product work and safety and said promised computing resources did not materialize; these are his reported concerns, not independently established details of every internal decision.

SSI said it would operate from Palo Alto and Tel Aviv. I look at the co-founders' backgrounds: Gross has invested in technology companies and worked at Apple and Y Combinator, while Levy previously worked at OpenAI and other major technology companies. I am curious how a company with such a long-term mission will align its investors and business model.

Michael Dell announced a Dell AI Factory with NVIDIA to power Grok for xAI. I show images discussed alongside the announcement and focus on the cooling challenge posed by large numbers of GPUs. The images and commentary alone do not establish the location, complete design or ownership of the planned xAI system.

A separate research paper from Fudan University and the Shanghai Artificial Intelligence Laboratory explores Monte Carlo Tree Self-Refine, or MCTSr. Some commentators compare it to the rumored OpenAI project Q*, and I find that possibility interesting. The public paper does not establish that it implements Q* or reveal what OpenAI's internal project is.

The method applies Monte Carlo tree search to candidate answers: it selects a solution, generates feedback and refinements, scores alternatives and updates the search tree. I see this kind of iterative search as a promising way to augment a language model. It is an inference-time procedure and should not be confused with evidence that the model has autonomously trained itself through self-play.

In the researchers' [reported results](https://arxiv.org/html/2406.07394), LLaMA-3 8B went from 74.07% with zero-shot chain-of-thought to 96.66% on GSM8K with eight MCTSr rollouts. The video repeats a claim that this beat GPT-4; the paper's own comparison table lists GPT-4 Turbo at 97.1% on GSM8K. It also reports 45.49% on GSM-Hard with eight rollouts, so the results depend on the benchmark and extra search compute.

Meta announced Chameleon models in 7B and 34B sizes, supporting mixed-modal inputs and text outputs in the release described here. The appeal is integrating modalities within the model rather than treating vision as a separate add-on. The [Chameleon research license](https://github.com/facebookresearch/chameleon/blob/main/LICENSE) limits use to noncommercial research; it is not an unrestricted open-source release.

Meta also presented multi-token prediction for code completion. Instead of training only to predict the next token, the method predicts several future tokens, which Meta says can improve training efficiency and speed. The announced pretrained code models were under a noncommercial research license, a restriction I would prefer not to see.

JASCO is Meta's text-to-music work, with additional conditioning inputs to give users more control. At the time, the paper was available and the model was promised for later release. I play samples that transform musical ideas into different arrangements, including an electronic pop sound, a Chinese traditional arrangement, country and psychedelic trance.

Meta also announced AudioSeal, a watermarking model intended to detect and localize AI-generated speech, plus research data and code addressing geographical and cultural representation. I see watermarking as a potentially useful tool as synthetic audio spreads, though a watermarking system alone is not a universal detector for every deepfake.

Groq announced fast speech-to-text using Whisper large-v3. While recording, I also saw Anthropic launch Claude 3.5 Sonnet, the first release in its 3.5 family. These were 2024 announcements, so availability, model options and prices should not be assumed to be current.

Anthropic's [launch benchmarks](https://www.anthropic.com/news/claude-3-5-sonnet) reported 96.4% on GSM8K, 59.4% on GPQA Diamond and 92% on HumanEval for Claude 3.5 Sonnet under the listed prompting conditions. Anthropic also claimed it ran twice as fast as Claude 3 Opus at lower cost. These are company-reported results; the model did not lead every row, and comparisons depend on evaluation settings.

Anthropic also introduced an Artifacts preview: Claude could generate documents, code, Mermaid diagrams, vector graphics and simple games in a workspace beside the conversation. I am excited by competition that improves capability and brings prices down, and I am interested in testing the models myself.

The ARC Prize launched a public competition around ARC-AGI, a benchmark of visual pattern tasks. Its organizers use the benchmark as a measure of progress toward general intelligence; I describe a million-dollar prize, but that is the broader prize fund rather than a simple payment for proving AGI. The 2024 [technical report](https://arcprize.org/media/arc-prize-2024-technical-report.pdf) describes a $600,000 grand prize for reaching 85% on the private evaluation set, alongside other awards and conditions.

ARC examples give input-output grids and ask the solver to infer a transformation. I solve one in which colored pieces are completed into squares and describe another in which colors move downward. Such puzzles test a particular form of abstraction; success on this benchmark alone would not settle the broader question of whether a system has achieved AGI.

At the time I look at an early leaderboard showing MindsAI around 38%, well below the 85% goal. That is a historical snapshot, not the final competition result or a present-day ranking. The 2024 grand prize ultimately went unclaimed, according to the organizers' report.