Inside RIDER: The Rugged Black Box Behind MIT's Large-Scale Driving Study

The MIT autonomous vehicle technology study collects large amounts of naturalistic driving data, and the box that makes this possible is called RIDER. A hardware engineer, Dan, and a software engineer, Michael, who is behind the data pipeline and offloading, introduce the device. Three cameras in the car run back through wires to RIDER, which sits in the trunk.

The design had to work month after month, across multiple vehicles and weather conditions. It collects three camera streams, an IMU, GPS and all of the raw CAN messages from the vehicle, and these must be recorded reliably, synchronized and post-processed once offloaded.

Inside is a single-board computer running a custom version of Linux written for this application, connected to a solid-state drive. A sensor integration board carries a real-time clock with battery backup, a CAN transceiver, a GPS receiver and an IMU. A CAN-controlled power board watches the car's bus to decide whether the system should be on, and steps the vehicle's 12 volts down to 5 volts. A 4G link reports health data such as free drive capacity, temperature and power usage.

The team set three requirements: record from at least three cameras, record vehicle telemetry, and store everything onboard long enough that people can drive for months before the data is offloaded. With hundreds of thousands of miles planned, the storage challenge is large. They estimate that every 100,000 miles would be about 100 petabytes of uncompressed video, and it must also be offloaded onto thousands of machines for deep learning and computer vision processing.

Their answer was the Logitech C920 webcam, which records up to 1080p at 30 frames per second and compresses video to H.264 on the camera. That removes the processing burden from the single-board computer, so a slim, lightweight one can run all the sensors. The cameras were put into custom cases that accept CS-type lenses, which allows a zoom lens and a fisheye lens inside the car for a wider range of fields of view.

To make sure the cameras would survive a hot parked car in summer, the team put them in a scientific toaster oven. They cycled them between 58 and 75 degrees Celsius, about the maximum a car would reach, and then pushed them to 127 degrees Celsius to see what prolonged heat would do. The cameras kept working perfectly afterward.

Because participants are simply driving their own cars, the device cannot rely on them to start and stop recording. RIDER watches CAN activity: when CAN is active the car is running and the system turns on, and when it goes quiet the system ends recording. That also allows triggers on specific CAN messages, such as someone unlocking the door or putting the car in drive.

The cars carry a price tag roughly a thousand times that of the box, with some costing over a hundred thousand dollars, so the wiring must not damage the vehicle. The most common failure has been camera cables coming unplugged. In that case the system restarts the subsystem several times and, if it still fails, stops recording entirely and will not start up until the cable is fixed. The team accepts some lost data because the requirement is that all video streams are always recorded perfectly and in sync.

Synchronization was the top priority from the start. The real-time clock keeps timestamping accurate to about two parts per million, which means timestamps drift by around seven milliseconds over a ninety-minute drive. After offload, the first step in the pipeline is to verify the timestamps and use them to align everything: video at 30 frames per second with GPS, and a separate scheme for faster sources such as the IMU and CAN messages.

That alignment is what lets researchers study how people interact with semi-autonomous vehicles, combining the face camera, body camera and forward view with GPS, IMU and vehicle telemetry. The heavy compression work happens on the cameras, leaving enough CPU cycles on RIDER for collection without skips or drifts in the sensor streams.

The data is not offloaded remotely. It lives on RIDER until participants come back, at which point the team swaps the hard drive, connects it to a local computer and copies the data to a server. They check it for consistency and fix problems in the raw data before synchronization.

Following a pixel's path: the camera records the raw image, compresses it to H.264, and sends it over USB to the single-board computer, which writes it to a video file on the solid-state drive. It stays there for about six months for the study's longer-term subjects and about one month for the shorter 50-subject group. After offload, initial cleaning algorithms remove corrupt data and fix subject information in each trip's configuration files, and the data is then synchronized at 30 frames per second.

Only then can it feed detection algorithms or manual annotation. The team calls this the hard work behind the magic of deep learning and computer vision: synchronization and cleaning of messy data produce a clean set of multiple perfectly aligned streams for analysis and for annotating data to improve neural network models. So far RIDER has collected data across over 30 vehicles, hundreds of thousands of miles and billions of video frames, close to 300 terabytes compressed.

A major planned improvement is moving to an NVIDIA Jetson TX2 single-board computer, which offers more room for sensors and more compute, and even the possibility of real-time systems. This matters because most driving is boring: nothing happens that helps in understanding driver behavior or training vision models on edge cases.

Based on what they have found in the data so far, the team can begin to judge which parts are interesting. Onboard algorithms could then decide in real time whether to keep or discard a stretch of video. The initial phase kept everything, since the area is largely unknown, to analyze body pose, glance allocation, activity, smartphone use, accelerations and Autopilot usage across geography, weather and time of day. Selectivity should follow as the fundamental insights emerge.

The study has collected over 320,000 miles so far and adds 500 to 1,000 miles every day. It is adding vehicles such as a Tesla Model 3 and a Cadillac CT6 with Super Cruise. The team's guiding principle is to record not only the forward roadway but rich information about the driver, including the face, glance, cognitive load, body pose and activity. They argue that autonomy needs an understanding of how human supervisors behave and how to keep them attentive and effective.