AI News in September 2024: Advanced Voice, Llama 3.2 and Orion

SUMMARY

In 30 seconds

In this September 2024 roundup, Matthew Berman tries ChatGPT Advanced Voice and discusses Llama 3.2, Meta’s Orion prototype, AI funding and personnel news, robotics and multimodal models. He distinguishes his early impressions and predictions from company announcements and reports available at the time.

1Advanced Voice arrivesBerman demonstrates interruptions and accents, and describes limitations he encountered in early use.
2Llama 3.2Meta announced 11B and 90B vision models and 1B and 3B models for devices.
3Orion prototypeMeta showed AR glasses with an EMG wristband and a wireless compute unit; access was limited.
4Microsoft AI changesA report described Sébastien Bubeck moving back to Microsoft Research and offered an interpretation of the strategy.
5Molmo modelsAI2 showed multimodal models, benchmark comparisons and a visual assistant demo.
6Murati departureThe OpenAI executive said she was leaving to make time for her own exploration.

Takeaways

Berman found Advanced Voice impressive but less reliable in his early tests than the earlier demo.
Meta announced Llama 3.2 vision models and smaller models intended for on-device use.
Orion was an AR prototype with glasses, an EMG wristband and a separate wireless compute unit, not a consumer product at the time.
Reports about funding, organizational strategy and unreleased products were not completed transactions or confirmed outcomes.
AI2 presented Molmo benchmark and preference results alongside a visual assistant demonstration.
Mira Murati announced her departure from OpenAI, citing a desire for time to explore.

TopicsFoundation Models Multimodal AI Voice AI Robotics AI Strategy

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

Advanced Voice and Llama 3.2

Berman tests the voice feature and compares it with earlier demonstrations. He then introduces Meta’s new model sizes.

Advanced Voice Mode is rolling out to paid ChatGPT users after a long wait following OpenAI’s original demonstration. I have tested it for about a day. In my demonstration, I interrupt it, ask it to speak in a sad voice, and try accents; it can stop and adopt some accents but refuses the requested speaking style.

My early impression is that it is impressive but does not work as smoothly as the earlier demo. It can get confused by pauses or repeat itself, and I have had trouble getting it to sing. At the time I report that the feature is unavailable in the EU and apparently has a daily quota; availability and limits are specific to that rollout.

Meta has announced Llama 3.2. The 11B and 90B versions add vision capabilities, while 1B and 3B versions are aimed at running on devices such as phones and computers. I describe the larger models as replacements for Llama 3.1 with vision added, and I plan to test them further.

SLIDE 1 / 10Llama 3.2 model sizesMeta’s announcement distinguishes the on-device models from the larger multimodal versions.

Orion and AI hardware

He examines Meta’s prototype glasses, wristband and compute unit. He also discusses Jony Ive’s AI hardware work.

Meta has also unveiled Orion, an AR glasses prototype with AI features. The thick frames look closer to ordinary glasses than a headset, and a demo shows virtual content placed in the real environment. I suspect tracking may not match a larger Apple Vision Pro, although that is my impression from the video rather than a comparative test.

SLIDE 2 / 10Orion opticsThe displayed prototype uses micro-LED projectors and optical-grade silicon carbide.

Meta describes micro-LED projectors and optical-grade silicon carbide for a roughly 70-degree field of view, custom chips, magnesium frames and miniature cameras and sensors. An electromyography wristband offers an additional input method, including when the hands are out of view. A separate wireless compute unit fits in a pocket.

SLIDE 3 / 10Orion wristbandMeta describes electromyography as an additional way to control the glasses.

Orion is a prototype, not a product I can buy at the time of this report. Meta says employees and selected external audiences will get access while it works toward consumer glasses, sharper displays, smaller form factors and affordability. I guess a future product could cost more than $1,000; that is my estimate, not an announced price.

SLIDE 4 / 10Orion’s prototype plansMeta outlines limited prototype access and its priorities for future consumer glasses.

Designer Jony Ive has confirmed he is working with Sam Altman and OpenAI on AI hardware. After leaving Apple, Ive started LoveFrom; I also comment on an expensive Moncler jacket associated with the design firm. The eventual AI device remains unspecified in this discussion.

Funding, teams and creative tools

The roundup turns to reported Anthropic funding talks, Microsoft personnel changes and Stability AI’s board. Berman considers their possible implications.

The Information reports that Anthropic has held funding discussions that could value the company at $30 billion to $40 billion. It also reports earlier projections of roughly $800 million in annualized revenue by year-end and $2.7 billion in cash burn. These are reported talks and projections, not a completed financing or confirmed financial results.

Another Information report describes a reshuffle at Microsoft AI under Mustafa Suleyman. Sébastien Bubeck, a lead researcher on Phi, is moving back to Microsoft Research after working on distilled models. The suggestion that this could signal a change in emphasis on synthetic data is an interpretation of personnel moves, not a confirmed change in strategy.

SLIDE 5 / 10Microsoft AI personnel changesThe displayed report describes Bubeck’s move and an interpretation of its strategic significance.

Filmmaker James Cameron is joining Stability AI’s board. The company says his perspective will help its work on visual media tools. Alongside Lionsgate’s partnership with Runway, I see this as evidence of some filmmakers exploring ways to incorporate AI into the creative process.

Robots, models and products

He reviews Boston Dynamics updates, Gemini releases, infrastructure reporting, Sora and Notion AI. He then examines AI2’s Molmo demos.

Boston Dynamics has shown updates for Spot and Orbit, including ultrasonic inspections, autonomous laser scanning and improved navigation. Spot can use multiple docking stations in an environment, and the demonstration shows it moving around obstacles and climbing stairs. I am also looking forward to further humanoid robot iterations.

SLIDE 6 / 10Spot laser scanningBoston Dynamics demonstrates an autonomous scanning capability.

Google announced updated Gemini 1.5 Pro-002 and Flash-002 models. Its announcement cited more than 50% lower Pro input and output prices for prompts under 128K, higher rate limits, faster output and lower latency. I welcome the improvements, though I find them less dramatic than some other announcements that week.

SLIDE 7 / 10Gemini 1.5 updatesGoogle’s announcement lists pricing, rate-limit and latency changes.

OpenAI reportedly discussed energy and data-center development with the White House, including a possible facility of up to five gigawatts. I speculate about a future of large data centers and independent energy generation, potentially including small nuclear plants. The proposal and my view of that future should not be mistaken for an operational facility.

The Information also reports that OpenAI is training a new Sora version intended to generate higher-quality, longer clips quickly. Sora had not been publicly launched when I recorded this video. I am eager to try it, although other video products have advanced since its first demonstration.

Notion has introduced deeper AI features, including access to external systems, writing assistance, and PDF and image analysis. I use Notion daily and have not yet had time to test the new functionality. Much of it is familiar elsewhere, but I welcome having it built into the product.

AI2 has introduced Molmo, a family of open multimodal models. Its displayed chart compares average scores across 11 academic benchmarks and human preference ratings: Molmo 72B leads the shown academic average, while GPT-4o is slightly ahead on the displayed preference rating. These are the presented comparisons, and I want to test the model myself.

SLIDE 8 / 10Molmo comparisonsAI2’s chart presents academic benchmark averages and human preference ratings.

A demonstration places Molmo in an Apple Vision Pro workflow: a user asks about a coffee machine, and the system describes it aloud. A second question asks for an overview of drinks, with the demo highlighting choices as it speaks. It is an example of visual assistance rather than a comprehensive evaluation of reliability.

SLIDE 9 / 10Visual assistant demoThe demonstration uses a coffee machine’s menu as the subject of a visual question.

Mira Murati’s departure

Berman reads Murati’s announcement and reflects on OpenAI’s leadership departures.

Just as I finish the roundup, Mira Murati announces that she is leaving OpenAI after six and a half years. She says she wants time and space for her own exploration and is focused on a smooth transition. I recall her interim leadership during the 2023 OpenAI upheaval and speculate about her role in it; that speculation is not established by her departure announcement.

SLIDE 10 / 10Murati’s departure noteThe displayed announcement explains her decision and transition focus.

I note that Ilya Sutskever has left and Greg Brockman is on extended leave, while OpenAI continues to ship products rapidly. The executive departures are another significant development in a busy week for the company.