Artificial intelligence (AI) Allows Peering into the Deep: High-Resolution 3D Tracking of Coral Reef Fish
A study by the Leibniz Centre for Tropical Marine Research (ZMT) is employing new methods in coral reef research. Under the leadership of fish ecologist Dr Julian Lilkendey, an international research team utilised innovative AI technologies to analyse the movements of reef fish in the Red Sea with high precision.
The recently published study in the journal "Ecology and Evolution", which also involved researchers from the Laboratory of Computer Science, Robotics and Microelectronics (LIRMM) at the University of Montpellier, France, and Auckland University of Technology (AUT) in New Zealand, combines stereo-video technology with AI-driven 3D tracking. The method provided detailed insights into the movement patterns and energy expenditure of two surgeonfish species in their natural habitat in the Red Sea.
The researchers initially observed that the Brown surgeonfish (Acanthurus nigrofuscus) showed a preference for algae growing on dead corals during foraging, whilst the Yellowtail tang (Zebrasoma xanthurum) utilised a broader food spectrum and also grazed on algae found on sedimentary rock, coral rubble, and sand.
The notion of "peering into the deep" is evident on multiple levels: Spatially, the researchers captured the three-dimensional movement of fish during foraging on the coral reef using calibrated stereo-video systems, which goes far beyond conventional two-dimensional observations, according to lead author Julian Lilkendey. The depth of insight was further enhanced through AI algorithms, which allowed for precise measurements of energy expenditure.
Targeted training of the AI model for species recognition
Initially, the pre-trained programme YOLOv5 (You Only Look Once version 5) was employed, a neural network for real-time object detection. For the study, YOLOv5 was fine-tuned with additional background images from the Red Sea to better recognise fish in the video recordings. Subsequently, the neural network classified the detected fish by species.
A particular challenge was the targeted training of this AI model for species recognition: "Because there were few specific training images for the two surgeonfish species and the region, we relied on media from the citizen science website 'iNaturalist'", explains Lilkendey. "This allows us to use a variety of publicly accessible photos".
For the subsequent three-dimensional data acquisition, the scientists employed the so-called DeepSORT algorithm (Simple Online and Realtime Tracking with a Deep Association Metric). "This algorithm enables robust multi-object tracking by following the detected fish across consecutive video frames", Lilkendey explains. "DeepSORT can track the movements of individual fish even when they temporarily disappear from view or are obscured by other objects. By integrating the 3D information from the stereo image pairs, the algorithm generates precise three-dimensional movement patterns of the fish".
By coupling this with an approach to model their energy expenditure, the research team gained new insights into the ecology of the surgeonfish species. "The Brown surgeonfish demonstrated specialised feeding behaviour, preferring certain algae growing on specific substrates, in contrast to the generalised feeding behaviour of the Yellowtail tang," reports Lilkendey. "Despite their low biomass, both species significantly contribute to reef grazing, using the energy obtained from food with similar efficiency in their movements."
Study results underline the role of surgeonfish in maintaining the ecological balance in coral reefs
These findings underscore the importance of niche partitioning and the role of surgeonfishes in maintaining the ecological balance in coral reefs. "Changes in feeding behaviour and energy budgets of surgeonfishes can influence algal growth and coral larval recruitment, thereby affecting the health and biodiversity of the entire reef ecosystem," Lilkendey continues.
Through this advanced analysis, the researchers have been able to peer more deeply into the functioning of marine ecosystems, laying the foundation for a better understanding of how energy is absorbed, transformed, and distributed within the reef.
Dr Lilkendey stresses: "With high spatial and temporal resolution, we were able to analyse the three-dimensional movements of many fish in a coral reef simultaneously. Our innovative methodological approach allows us to peer deeper into the complexity of fish behaviour and resulting energy flows."
The research methodology also opens up new possibilities for deriving 'Energy Seascapes' – detailed representations of the energy expenditure of animals in marine ecosystems. Such mappings are crucial for developing effective health indicators and novel protection measures for reefs.
Publication: Lilkendey, J., Barrelet, C., Zhang, J., Meares, M., Larbi, H., Subsol, G., Chaumont, M., Sabetian, A. (2024) Herbivorous fish feeding dynamics and energy expenditure on a coral reef: Insights from stereo-video and AI-driven 3D tracking. Ecology & Evolution. https://doi.org/10.1002/ece3.11070
Videos showing the feeding behaviour of Brown surgeonfish and Yellowtail tang as well as AI-based tracking
By clicking on this preview image, you consent to content being downloaded from YouTube or Google (USA). As a result, YouTube or Google (USA) receives the information that you have accessed our site and the data technically required in this context. We have no influence on further data processing by Google. Further information can be found in our privacy policy.Watch on YouTube
Contacts