MIT sharpens underwater robot vision

MIT has presented Sonar-MASt3R, an underwater mapping method that combines optical cameras and sonar to produce real-time 3D maps in murky water. The central fact is verified by MIT News, published on June 11, 2026: Amy Phung, a graduate student in the MIT-WHOI Joint Program, presented the work at the IEEE International Conference on Robotics and Automation, with Richard Camilli of Woods Hole Oceanographic Institution as co-author. The method targets underwater vehicles and robotic arms that need to move, inspect, or recover objects when visibility is too poor for cameras alone.

The problem is practical rather than decorative. A camera captures fine visual detail, but only when the water is clear enough and sufficiently lit. Sonar measures distance by sending acoustic waves and analyzing their return, so it remains useful in water loaded with sediment, but its maps usually carry less visual detail. Sonar-MASt3R is an opti-acoustic fusion method, meaning it combines visual and acoustic data to make perception more robust. MIT says earlier approaches often required offline processing or focused on narrower reconstruction tasks, while this work aims for 3D maps that can be used in real time under turbid, low-visibility conditions.

The software builds on MASt3R, an image-matching algorithm that can quickly estimate relative pixel depth from 2D images. Its weakness is scale: it can infer that one point is closer than another, but not always whether the difference is centimeters or meters. Sonar supplies that absolute measurement. In the experiments, the researchers filled a tank with water, sediment, and objects, then mounted an underwater camera and a sonar sensor on a robotic arm. An initial sweep creates a coarse sonar map. The arm can then move closer to useful regions, and selected camera images refine the map through a keyframe approach, where reference frames are kept only when they add new information.

The robotics value is larger than a cleaner 3D picture. MIT reports that Sonar-MASt3R was tested across eight turbidity levels and generated more accurate reconstructions than other fusion approaches, resolving centimeter-scale details in cloudier conditions. In the murkiest case, the arm’s camera could no longer see the objects, but the sonar still produced a rough map that helped guide safer motion toward them. The cited applications are deliberately grounded: scientific exploration, underwater construction and maintenance, deep-sea recovery, and work around unexploded underwater mines. The next step is to move beyond controlled tank experiments into natural underwater settings, where the researchers expect some echo and reverberation problems to be less severe.