Vision for underwater inspection, from image segmentation to 3D mapping with video and sonar.
Motivation
Computer Vision tasks above water provide challenges.
Underwater, the challenges intensify. While cameras provide rich visual detail, they struggle in murky water, or in poor lighting.
Sonar, on the other hand, works in these conditions, but provides far less visual detail.
Additionally, labelled real-world underwater data is scarce and expensive to collect, which limits how well machine learning models can be trained for these tasks.
Our research addresses these challenges via image segmentation and visual odometry towards reliable 3D mapping of underwater scenes.
3D Mapping of Underwater Scenes using Video and Sonar (2027–2029)
New project! More details tba
In 2026, we received funding from the Orient Fund and the Danish Maritime Fund for the project 3D Mapping of Underwater Scenes using Video and Sonar.
We will investigate when and how combining cameras and 3D sonar improves underwater 3D mapping compared to either technology alone.
The project will make its data, evaluation methods, and reference implementations publicly available.
Potential applications include the inspection of ship hulls, offshore energy installations, harbours, bridges, and pipelines.
Open position: A research assistant position for this project will be announced.
Preliminary Related Work
Ongoing work with Olaya Álvarez-Tuñón: 3D Gaussian Splatting for underwater scenes.
Code and evaluation scripts are publicly available.
In ongoing student projects, we explore underwater scene reconstruction, e.g. by evaluating 3D Gaussian Splatting for underwater 3D reconstruction in degraded visual conditions.
Image Segmentation for Underwater Inspection
Together with Andrzej Wąsowski and Yury Brodskiy, I co-supervised Luiza Ribeiro Marnet, who defended her PhD thesis Vision-based classification for underwater safety critical applications in March 2025.
Her work showed how active learning can drastically reduce the amount of labelled data needed for pipeline segmentation in underwater inspection, and how models trained on simulated images can be adapted to real underwater footage.
As part of this work, the MarinaPipe dataset was published.
References
2026
Gaussian Splatting Underwater: A Controlled Cross-Regime Study
Labeling images for every new task or data pattern a model needs to learn is a significant time bottleneck in real-world applications. Moreover, acquiring the necessary data for training the models can be challenging. Ideally, one would train the models with simulated images and adapt them for the desired real tasks using the least possible amount of data. Active learning can be used to solve this problem with minimal effort. In this work, we train SegFormer for pipeline segmentation with synthetic images from an underwater simulated environment and fine-tune the model with real underwater pipeline images recorded in a marina.
The evaluation shows that selecting real data with active learning for fine-tuning the model gives better results than randomly selecting the images. As part of the work, we release the dataset recorded in the marina, MarinaPipe, which will be publicly available.
2024
Uncertainty Driven Active Learning for Image Segmentation in Underwater Inspection
Luiza
Ribeiro Marnet
, Yury
Brodskiy
, Stella
Grasshof
, and
1 more author
In Robotics, Computer Vision and Intelligent Systems , Feb 2024
Active learning aims to select the minimum amount of data to train a model that performs similarly to a model trained with the entire dataset. We study the potential of active learning for image segmentation in underwater infrastructure inspection tasks, where large amounts of data are typically collected. The pipeline inspection images are usually semantically repetitive but with great variations in quality. We use mutual information as the acquisition function, calculated using Monte Carlo dropout. HyperSeg is trained using active learning with an underwater pipeline inspection dataset of over 50,000 images. To allow reproducibility and assess the framework’s effectiveness, the CamVid dataset was also utilized. For the pipeline dataset, HyperSeg with active learning achieved 67.5% meanIoU using 12.5% of the data, and 61.4% with the same amount of randomly selected images. This shows that using active learning for segmentation models in underwater inspection tasks can lower the cost significantly.
