Computer Vision based Seafloor Surveying Technique

The Vision, Graphics and Computational Design (VGD) Group at HKUST led by Prof. Sai-Kit Yeung performs cutting-edge research on underwater object detection, semantic segmentation, simultaneous localization and mapping. The advanced computer vision techniques have been introduced for generating biology report, replacing with the traditional scuba diving and trawling based surveying.

The VGD group is now working with utilizing the underwater drones and object recognition for marine conservation. The group is to provide marine biologists, field ecologists, and coral conservationists with novel computer vision and 3D techniques to solve a wide range of marine related problems.

Seafloors covered by marine soft sediments, which compose more than 70% of earth's seafloor, are also inhabited by a remarkable number of animals. Unfortunately, in recent decades, marine seafloor habitat is threatened by marine pollution. Hence, seafloor survey is an essential part in estimating biodiversity and evaluating the condition of the local marine ecosystem. Conventional methods for seafloor survey mainly rely on scuba diving and trawling. However, survey by scuba diving is costly, labor-intensive, low throughput, and highly limited by water depth and turbidity while seabed trawling is highly destructive and will result in major damage to the benthic environment.

We propose a new seafloor surveying technique by using latest underwater drone and simultaneous localization, mapping, and object recognition technology. The key advantages over traditional method such as scuba diving and trawling is that we can perform high-throughput surveys while remaining non-invasive. The project will be divided into five main parts: 1) Data collection using underwater drone. 2) Marine object detection and classification algorithm development. 3) Underwater localization algorithm (SLAM) development. 4) System fusion, verification and optimization. 5) Final report on the seafloor condition.

Our underwater drone:

Our customized underwater drones equipped with multiple cameras for wide-angle video capture will be deployed at designated locations. The drones will be remotely controlled to move along a transect a fixed distance above the seafloor and capture video of the seafloor along the way.

One of our goal is to perform underwater surveying, including detecting the underwater organism with regular shapes (e.g., sea urchin, sea cucumber, starfish and so on) and segmenting the corals with irregular boundaries.

Underwater object detection video demo:

In our system, it is essential for robots to be able to be aware of their location. We are going to further improve our SLAM system with multi-sensors fusion to enhance system’s robustness and increase localization accuracy. We will combine the marine species classification algorithm and the underwater localization algorithm. After integration of algorithms, the system will be able to not only identify marine specifies, but also localize them. The proposed system can facilitate robust data analysis and automatic underwater surveying.

Semantic SLAM video demo:

We hope our system can inspire the underwater research community to further develop and increase awareness of the general public on marine conservation.

We are hiring! If you are an ocean lover and want to develop new technology to solve different marine related problems, please email saikit@ust.hk to join our research program.

[1] Ziqiang Zheng, Huajian Huang, Apple, Pui-Yi Chui, Yue Him Wong, Sai-Kit Yeung. Advanced Computer Vision Techniques for Efficient Seafloor Surveying. International Conference on Biodiversity, Ecology and Conservation of Marine Ecosystems (BECoME 2022)