Development and application of a robot-assisted computer vision system to map Great Lakes bottom habitats and biology
Autonomous Underwater Vehicle
Lake Huron near Alpena
Deploying an Underwater Camera
Lake Michigan
Underwater Camera in Use
Diver's Point of View
Pivot Point for Circular Transect
Lake Michigan
Lake bottom environments are critical zones of interface between geology and biological processes that support health ecosystems and human well-being. Over the past thirty years, Great Lake food webs have become dominated by bottom dwelling invasive species and nuisance algae, that are poorly mapped and understood. USGS is developing a suite of new technologies to map habitat, invasive mussels and fish, and nuisance algae in high resolution to meet the needs of lake managers.
Overview
Great Lakes bottom environments are critical zones of interface between geology, the physical processes that control morphology and bottom composition, and biological processes that support healthy ecosystems and human well-being. Many challenging natural resource management issues are tightly coupled with the lake bottom, and for many of these we do not have sufficient information to support effective management responses. For instance, round goby and quagga mussels are highly influential invasive species that are common to the bottom environments of all the Great Lakes except Lake Superior to 130 m depth, but their distributions and abundances are poorly documented. Further, well-resolved maps of habitat types have been identified as a major data gap that impedes progress in habitat assessments such as those being undertaken in the Great Lakes Water Quality Agreement Annexes 2 and 7. Finally, excessive growth of the nuisance algae Cladophora is tightly associated with the lake bottom environment, and improved methods are needed for understanding its distribution, biomass, and impacts.
New technologies are needed that can efficiently map and quantify bottom features of the Great Lakes to serve the needs of management agencies in the United States and Canada. The current project proposes to combine three technologies—underwater robots, computer vision, and stereoscopic imaging—to develop methods to remotely characterize bathymetry, surface geology, round goby abundances, and Cladophora biomass.
Goals and objectives
The goal of the proposed work is to demonstrate the utility of underwater remote sensing by developing a robot-deployed computer vision system capable of automatically quantifying lake bottom features.
This goal will be accomplished with multiple partners by pursuing the following objectives:
- Implement a field sampling program to collect spatially extensive data across (a) a range of bottom habitat types, (b) a gradient of Cladophora productivity, and (c) a gradient of round goby biomass density.
- Develop computer models to automatically classify bottom types, with special emphasis on substrates, Cladophora, and other submerged aquatic vegetation (SAV).
- Develop computer models to predict Cladophora biomass from its volumetric occupancy of the water column.
- Develop algorithms to identify round goby and estimate their biomass from pixel areas occupied.
- Train partners in field data collection, image handling, and the application of algorithms to automate image processing and interpretation.
- Publish the results of the research in peer-reviewed scientific literature.
Environment and Climage Change Canada Webinar Recording
Anticipated products
The following products and deliverables will result from this work in 2019.
- High-resolution maps of bathymetry, bottom type, and Cladophora biomass for sampled areas in the form of geospatial datasets.
- Open-source software for:
- Bottom type classification
- Cladophora biomass estimation
- Round goby biomass estimation
- Written tutorial for using the software
- Training of field and lab personnel in the use of computer vision methods
- Image libraries of labeled images for use in future model refinement
- Interim report in March 2018 reporting on what was achieved up to that point.
- Scientific publications
Useful Links
Images, Videos, and Presentations from the study. Choose the Gallery media type if you're just looking for our still images.
Digital 3D reconstruction of lake bottom using 10 photo frames taken by the dive camera system. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Digital 3D reconstruction of lake bottom using 10 photo frames taken by the dive camera system. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Digital 3D reconstruction of lake bottom using 50 photo frames taken by the dive camera system. This example shows an arc since the sampling transect is a circle. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Digital 3D reconstruction of lake bottom using 50 photo frames taken by the dive camera system. This example shows an arc since the sampling transect is a circle. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Close up of crayfish next to center pivot. When crayfish swims away, then a diver can be seen next to the center pivot attaching the line for the dive camera transects.
Close up of crayfish next to center pivot. When crayfish swims away, then a diver can be seen next to the center pivot attaching the line for the dive camera transects.
Photos taken by the dive camera system of the lake bottom covering sections of sandy and rocky substrate with lots of dreissenid mussels dead and alive.
Photos taken by the dive camera system of the lake bottom covering sections of sandy and rocky substrate with lots of dreissenid mussels dead and alive.
Cladophora sampling site with brick marker and quadrat. Divers put cladophora samples from inside quadrat into bags to take back to the lab.
Cladophora sampling site with brick marker and quadrat. Divers put cladophora samples from inside quadrat into bags to take back to the lab.
Quadrat for sampling cladophora with gobies swimming around. There is another quadrat in the distance for a different sampling site.
Quadrat for sampling cladophora with gobies swimming around. There is another quadrat in the distance for a different sampling site.
Diver completing a transect with the dive camera while another diver stands on the center pivot to keep it upright. The transect is a circle and is being controlled with the line attached to the pivot.
Diver completing a transect with the dive camera while another diver stands on the center pivot to keep it upright. The transect is a circle and is being controlled with the line attached to the pivot.
Diver point of view from above the dive camera system. This is a transect over rocky substrate with some cladophora on the bottom of Lake Ontario.
Diver point of view from above the dive camera system. This is a transect over rocky substrate with some cladophora on the bottom of Lake Ontario.
Diver travelling along the surface of the lake along with the dive camera system. The crew on the boat are there to meet the divers along with pulling the center pivot back onto the boat with the help of the davit.
Diver travelling along the surface of the lake along with the dive camera system. The crew on the boat are there to meet the divers along with pulling the center pivot back onto the boat with the help of the davit.
Smallmouth bass swimming near lake bottom. The camera pans up a couple times to show the center pivot where the diver is holding on.
Smallmouth bass swimming near lake bottom. The camera pans up a couple times to show the center pivot where the diver is holding on.
The camera glances at the bottom of the boat above and then pans down to the lake bottom. The lake bottom is covered in tons of round gobies swimming around in and out of the cladophora. Again the camera pans up to show a diver and the dive camera completing a transect in the distance.
The camera glances at the bottom of the boat above and then pans down to the lake bottom. The lake bottom is covered in tons of round gobies swimming around in and out of the cladophora. Again the camera pans up to show a diver and the dive camera completing a transect in the distance.
Smallmouth bass swimming in front of the dive camera system during a transect. Camera system is attached to the center pivot by a line in order to follow around in a circle.
Smallmouth bass swimming in front of the dive camera system during a transect. Camera system is attached to the center pivot by a line in order to follow around in a circle.
Descending to the lake bottom to get ready for transects with the camera system and cladophora sampling. Organizing quadrats and sampling bags near the center pivot before sampling.
Descending to the lake bottom to get ready for transects with the camera system and cladophora sampling. Organizing quadrats and sampling bags near the center pivot before sampling.
Images, Videos, and Presentations from the study. Choose the Gallery media type if you're just looking for our still images.
Digital 3D reconstruction of lake bottom using 10 photo frames taken by the dive camera system. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Digital 3D reconstruction of lake bottom using 10 photo frames taken by the dive camera system. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Digital 3D reconstruction of lake bottom using 50 photo frames taken by the dive camera system. This example shows an arc since the sampling transect is a circle. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Digital 3D reconstruction of lake bottom using 50 photo frames taken by the dive camera system. This example shows an arc since the sampling transect is a circle. This reconstruction allows us to gain more insight about the lake bottom since elevation has been added to the photographs.
Close up of crayfish next to center pivot. When crayfish swims away, then a diver can be seen next to the center pivot attaching the line for the dive camera transects.
Close up of crayfish next to center pivot. When crayfish swims away, then a diver can be seen next to the center pivot attaching the line for the dive camera transects.
Photos taken by the dive camera system of the lake bottom covering sections of sandy and rocky substrate with lots of dreissenid mussels dead and alive.
Photos taken by the dive camera system of the lake bottom covering sections of sandy and rocky substrate with lots of dreissenid mussels dead and alive.
Cladophora sampling site with brick marker and quadrat. Divers put cladophora samples from inside quadrat into bags to take back to the lab.
Cladophora sampling site with brick marker and quadrat. Divers put cladophora samples from inside quadrat into bags to take back to the lab.
Quadrat for sampling cladophora with gobies swimming around. There is another quadrat in the distance for a different sampling site.
Quadrat for sampling cladophora with gobies swimming around. There is another quadrat in the distance for a different sampling site.
Diver completing a transect with the dive camera while another diver stands on the center pivot to keep it upright. The transect is a circle and is being controlled with the line attached to the pivot.
Diver completing a transect with the dive camera while another diver stands on the center pivot to keep it upright. The transect is a circle and is being controlled with the line attached to the pivot.
Diver point of view from above the dive camera system. This is a transect over rocky substrate with some cladophora on the bottom of Lake Ontario.
Diver point of view from above the dive camera system. This is a transect over rocky substrate with some cladophora on the bottom of Lake Ontario.
Diver travelling along the surface of the lake along with the dive camera system. The crew on the boat are there to meet the divers along with pulling the center pivot back onto the boat with the help of the davit.
Diver travelling along the surface of the lake along with the dive camera system. The crew on the boat are there to meet the divers along with pulling the center pivot back onto the boat with the help of the davit.
Smallmouth bass swimming near lake bottom. The camera pans up a couple times to show the center pivot where the diver is holding on.
Smallmouth bass swimming near lake bottom. The camera pans up a couple times to show the center pivot where the diver is holding on.
The camera glances at the bottom of the boat above and then pans down to the lake bottom. The lake bottom is covered in tons of round gobies swimming around in and out of the cladophora. Again the camera pans up to show a diver and the dive camera completing a transect in the distance.
The camera glances at the bottom of the boat above and then pans down to the lake bottom. The lake bottom is covered in tons of round gobies swimming around in and out of the cladophora. Again the camera pans up to show a diver and the dive camera completing a transect in the distance.
Smallmouth bass swimming in front of the dive camera system during a transect. Camera system is attached to the center pivot by a line in order to follow around in a circle.
Smallmouth bass swimming in front of the dive camera system during a transect. Camera system is attached to the center pivot by a line in order to follow around in a circle.
Descending to the lake bottom to get ready for transects with the camera system and cladophora sampling. Organizing quadrats and sampling bags near the center pivot before sampling.
Descending to the lake bottom to get ready for transects with the camera system and cladophora sampling. Organizing quadrats and sampling bags near the center pivot before sampling.