In 30 seconds
At Tesla AI Day in 2021, engineers said Dojo’s multi-node scaling problem was not yet solved, even as they described prototype testing and a path for Tesla workloads. They explained how simulation could supplement real driving data with rare situations, and how large offline models could label data for smaller in-car networks. Musk also outlined plans for a humanoid robot and made optimistic predictions about driverless safety; Tesla’s enabled Full Self-Driving features still require active driver supervision.
- 1Dojo’s scaling problem was not yet solvedAn engineer describes prototype tests and the challenge of keeping computation close to memory.
- 2Large training models need not run inside the carEngineers explain how offline models can generate labels for smaller networks deployed with latency constraints.
- 3Simulation fills gaps in rare driving scenariosThe team describes creating variations of unusual events and evaluating against real-world video.
- 4The safety trade-off question remains open in the answerMusk says computation could make conflicts rarer but does not give a detailed policy for unavoidable trade-offs.
- 5Musk’s driverless future was a predictionTesla’s currently enabled Full Self-Driving features require active driver supervision and do not make a vehicle autonomous.
Takeaways
- A Dojo engineer said multi-node scaling remained unsolved and emphasized memory locality, model parallelism and data parallelism.
- The team described simulation as a targeted supplement to real video, particularly for rare and dangerous cases.
- Larger offline models can help label data even when their latency makes them unsuitable for in-car use.
- Tesla Bot was presented as a future general-purpose robot for repetitive or hazardous tasks, not a demonstrated product.
- Musk’s 2021 driverless-safety claims were predictions; Tesla says enabled Full Self-Driving features require active driver supervision.
Read the edited text by chapter
Dojo scaling and simulation
The team discusses open sourcing, training hardware, cooling and simulation ambitions.
At Tesla AI Day in August 2021, Elon Musk and engineers take audience questions on training hardware, driving software, simulation and the proposed Tesla Bot.
Asked about open sourcing, Musk says building the system is expensive and questions how a fully open project would be funded. He says other carmakers could potentially license it.
A researcher asks whether the Dojo compiler can scale across many nodes. An engineer says the problem is not solved yet, but the team has run networks on prototype hardware and modeled performance.
The engineer discusses keeping computation near local memory, combining model and data parallelism, and fitting parameters into the available SRAM. He sees a path for Tesla’s intended workloads, not a finished scaling result.
Musk predicts Dojo will be operational the following year and says the initial job is video training. His practical test is whether engineers prefer it to the existing GPU cluster.
He explains that reducing training time from days to hours could allow more iterations and speed improvement. This is an intended benefit, not a measured result shown in the question session.
A simulation engineer says the team models vehicle dynamics and hopes to integrate more vehicle components. He envisions a broader simulator and eventual support for the humanoid robot.
Asked about combining language and vision, Musk emphasizes vision networks for driving while acknowledging that audio can matter, for example for emergency vehicles or communication with passengers.
An audience member asks how a system trained largely in the United States will transfer abroad. Musk says Tesla uses video from many countries but is focusing development on the United States before extending it elsewhere.
An engineer says visual features such as roads and curbs are broadly shared, while Musk emphasizes avoiding unexpected objects even if the system does not recognize their category.
Asked about multi-view geometric accuracy, the team cites errors on the order of a few centimeters in its example and says accuracy varies with distance. The figure is a team estimate in this discussion, not a general guarantee.
Musk says factory automation varies across the production process, with some tasks still largely manual. He mentions large cast body sections as a way to simplify manufacturing.
He says Tesla Bot was not prompted by a single factory need but by the availability of components for a general-purpose humanoid robot. The proposed first use is repetitive, dangerous or boring work.
A question about Dojo power and cooling prompts an engineer to describe modular tiles with liquid cooling on both sides. Musk says Tesla applied power-electronics and cooling experience from vehicles.
Training, planning and synthetic scenes
Engineers explain offline models, end-to-end plans and targeted simulation of rare cases.
Asked whether larger training models would fit in cars, the team distinguishes training and automatic labeling from in-car inference. Some large models can improve labels without being deployed in the vehicle.
Musk describes Dojo as specialized for neural-network training and says it must prove itself against GPU clusters. His claim that specialization will improve performance is a design expectation.
Engineers discuss joining perception, planning and control in an end-to-end system. They expect to build incrementally, combining learned components with structured search or physical models.
Musk says a vehicle could use neural networks for visual perception and conventional software for planning, though he expects learned control may ultimately be better. He predicts higher-than-human safety, without presenting a comparative study here.
Asked whether Tesla applies machine learning throughout manufacturing, Musk cautions that it is difficult and advises using simpler methods when they suffice.
A simulation engineer describes conventional real-time rendering, ray tracing and experiments with neural rendering. The aim is to make simulated scenes useful without a model learning simulator-specific artifacts.
Musk says simulation can create variants of rare, dangerous situations, such as an imminent collision or a wrong-way driver, without requiring real crashes for every example.
The team says simulated clips are a targeted supplement to real-world data and that it evaluates against real video. Engineers discuss monitoring the gap between simulated and real scenes.
Robot uses and driving-system design
Questions cover the humanoid robot, model size, trust and safety trade-offs.
Asked about companionship, Musk says Tesla Bot’s immediate aim is practical physical work, with other uses potentially following later. This was a proposal for a future robot, not a demonstrated capability.
An engineer says larger models generally help when adequate data is available, but in-car latency limits deployment. Larger offline models can instead produce improved automatic labels for smaller deployed networks.
The team describes predictions about partially hidden roads and road users. These are examples of the system’s intended behavior; predicting a hidden object reliably remains a safety-critical challenge.
Asked about hand design, Musk says the initial humanoid concept uses five fingers to work with tools and carry objects, though the design may evolve.
A questioner says flickering object classifications can reduce trust in the driving display. The team says multi-camera models are improving, while acknowledging the work was not finished.
Asked about the economics of a robot doing low-paid repetitive work, Musk offers no detailed short-term cost or business model.
A question asks how a car should weigh occupant and pedestrian safety. Musk argues that fast computation could make such conflicts rarer, but does not provide a detailed policy for unavoidable trade-offs.
Uncertainty, testing and future predictions
The team discusses hardware, adversarial inputs, shadow mode and AI risks; Musk makes autonomy forecasts.
Musk predicts the then-current Hardware 3 will eventually deliver much safer driving than a person and mentions a future Hardware 4. These were 2021 predictions, not demonstrated comparative safety outcomes.
Asked about adversarial inputs, he gives a stop-sign T-shirt as an example that could trigger a mistaken response and says training can address observed cases.
Engineers say planning considers multiple possible actions by other road users and must balance caution with practical progress. They describe slowing when uncertainty is high.
Musk describes running candidate software in shadow mode to compare its hypothetical actions with those of human drivers. The team uses differences to identify cases for further analysis.
Asked which part of the stack matters most, the team says improvements across multiple components matter and points to expanding the use of surrounding video. They also say additional engineering talent is needed.
On continuous learning, an engineer says the deployed model is trained and tested before release rather than learning extensively on the spot. Musk imagines adjustable driving preferences as a future interface.
Asked about AI alignment, Musk says powerful AI warrants concern, while describing Tesla’s immediate goal as narrower systems for driving and useful robot tasks.
A final question asks about cameras and future hardware. Musk predicts current cameras and compute can support full self-driving and says later hardware could improve performance. These were forecasts made in 2021.
Musk imagines a future in which self-driving vehicles are as ordinary as automatic elevators. [Tesla’s current guidance](https://www.tesla.com/support/fsd) says enabled Full Self-Driving features require active driver supervision and do not make the vehicle autonomous.
More from this channel
3:35:00EN
41:04EN
8:15EN
53:25EN
7:20EN
3:39:05EN