North American Amphibian Monitoring Program
The North American Amphibian Monitoring Program (NAAMP) was a collaborative citizen science effort between the US Geological Survey (USGS) and 26 partners (state agencies, universities, and nonprofit organizations) for monitoring calling amphibian populations over much of the eastern and central United States. Initiated in 1997, in response to needs set forth by the Declining Amphibian Populations Task Force (DAPTF) in 1994, NAAMP was designed to provide scientifically and statistically defensible, long-term distribution and trend data for calling frog and toad populations at both the state and regional level.
The national program closed at the conclusion of the 2015 season.
NAAMP Protocol
Route Creation
Routes were generated in a stratified random block design at USGS Patuxent Wildlife Research Center. Routes were then distributed to Regional Coordinators. These roadside routes were then groundtruthed to determine suitability (not too dangerous, not too noisy to hear) and stop placement. There were 10 stops per route. Two methods of stop placement were permitted: equidistant stops or stratified by habitat. In equidistant stop placement, each stop was exactly 0.5 miles apart, regardless of surrounding habitat. When stratified by habitat, the stops were at least 0.5 miles apart and were located at wetland habitats. The wetland habitat was appropriate potential habitat (pond, vernal pool, roadside ditch, etc) but the presence or absence of amphibians was not used as a stop selection factor. Some alteration of the route could occur during groundtruthing. Stop locations and any route alterations were shared with NAAMP to keep route maps accurate and up to date (although, this did not always occur). Once a route was groundtruthed and the 10 stops determined the route and stops was not changed, unless exceptional circumstances occurred, see Stop Inaccessibility, Stop Relocation, and Stop Retirement section of this document. In addition, some regions may have had nonrandom routes that were created by other methods, but not included in USGS analyses.
Seasonal Sampling Periods
Each state established three or four sampling periods to cover the calling phenology of its local species. States used three sampling periods, unless an additional period to target wood frogs was desired. The sampling periods were created to target the peak vocalization times for early-, mid- and late-season breeding amphibians and to assist observers in understanding when to collect data. A state may have subdivided into regions and established different sampling dates within these regions. Sampling periods could not overlap, but could be separated by an interval or begin and end on adjoining dates. States were permitted to adjust the sampling dates each year to account for an early or late arrival of the calling season.
Nightly Sampling Conditions
Surveys were meant to begin 30 minutes after sunset or later. No matter what time a route was started, it was meant to be completed by 1 a.m. Appropriate sampling conditions were based upon wind, sky, and air temperature conditions. For most regions the wind code was meant to be at level 3 or less, but the wind prone Great Plains region was permitted to sample at level 4 or less. Surveys should not have been conducted during heavy rainfall, but light rainfall was acceptable (sound of the rain may impair hearing ability).
The air temperature criteria were the minimum allowable temperatures, varying for each sampling period.
Data Collection
Stops were conducted in numerical order, in one night by one observer. We encourage, but do not require, that one observer conduct all surveys of a route in a given year. Because some observers had assistants who wished to collect data and occasionally a state coordinator had to assign multiple individuals to the same route to meet high volunteer demand, multiple observers were instructed to each fill out their own datasheet, separately and independently and to submit their datasheets under their name, not another observer’s name. Each observer was responsible for passing the FrogQuiz in order for their data to be released. All datasheets were returned to the State Coordinator for Data Quality Checking and archival purposes. This “one observer per datasheet” rule allowed each survey conducted to be of equal effort, without outside influence.
Observers recorded the time, sky code, and wind code, at the beginning and end of each survey to verify that the sampling conditions were met on the evening of the survey. At each stop air temperature was recorded to verify that sampling conditions were met on the sampling night; at least eight of the 10 stops must have met temperature guidelines. For Gulf Coast states that recorded air temperature only at the beginning and end of a survey, both temperature readings must have met these guidelines. Gulf Coast and Great Plains states required documentation of the last rainfall event, since routes should have been conducted within 3 days of rainfall, based on anuran calling behaviors in those regions.
At each stop the observer listened for 5 minutes, and recorded the amphibian calling index for each species heard. The 5 minute listening period had no initial waiting period. Starting in 2006, the observer also recorded the number of cars that passed during the listening period and whether the moon or moonlight was visible. Car counting was often conducted by an assistant. The observer indicated whether background noise impaired his/her ability to hear (most surveys used yes/no checkbox; some adopted the noise index developed by Massachusetts). If there was a major noise disturbance, lasting one minute or longer, the observer broke the listening period to avoid sampling during the excessive noise. If such a time out was taken, this was noted on the datasheet. After the major disturbance ended, the observer resumed listening for the time remaining.
Stop Inaccessibility, Stop Relocation, and Stop Retirement
- Stop Inaccessibility: Temporary stop inaccessibility may have occurred for some transient reason (i.e. traffic accident blocked road access).
- If only one stop was missed, the route was considered complete. The observer wrote on the datasheet which stop was missed and noted why in the comments section. When entering the data into the database, they marked the checkbox indicating which stop was missed.
- If more than one stop was missed, the route should have been re-ran on another night.
- Stop Relocation: Stop relocation was when a stop needed to be shifted to a new location, after the groundtruthing phase occurred. During groundtruthing the permanent stop locations are set (see groundtruthing guidelines). Stop relocations were a rare event.
- Stop relocation should only have occurred for safety reasons (i.e. route was safe before-or appeared to be, but perhaps a homeowner fired a gun in the air as warning to observer).
- Stops should NOT have been relocated because of habitat loss or lack of calling amphibians at the site.
- To relocate (for safety reasons) a stop, the State Coordinator should have used their best judgment on when it was necessary and where to relocate. It was preferable to move the stop a short distance away, not impacting the 0.5 mile minimum distance rule, when possible. If possible, then stops were relocated by creating a new stop at the end or beginning of the route and renumbering all the stops. A written record of when, why, and how a stop was relocated, was kept in most cases.
- Stop Retirement: Once the route was groundtruthed and listening stations established, these locations were permanent and locations were changed unless a safety issue arose. If habitat destruction occurred at a listening station, and a local extinction of amphibians occurred, this was important information. To document habitat destruction, the location should have been surveyed for three seasons beyond the destruction date. After three seasons of non-activity, the listening station was retired, and null data were assumed for this site in analyses from that point forward. A listening station could not be retired merely because the wetlands were uninhabited by anurans. These retired stops should have been revisited periodically to verify that no suitable habitat existed, but five minutes of listening was no longer required.
Data Review Process
State Coordinators performed data checks each year to ensure all entered data followed the same review procedures. Some checks and balances were incorporated into the database design (pop-up warning boxes, etc.), while others were procedures only Coordinators could do. These procedures were adopted at the Nashville NAAMP Coordinators meeting.
- All data entered same way: All datasheets were entered "as they appeared" and then "checked" for any errors. This pattern was obvious if the volunteer entered their own data, as the State Coordinator could not "check" the data before they were entered. This pattern was followed, even for datasheets that the State Coordinator entered. This way all data went through the same data review process. Also, by entering the data "as it was" and then making any necessary corrections after the fact, the database had a record of the correction and when and why it occurred.
- The only exceptions were "simple obvious data entry errors" such as the observer wrote 70 degrees and then marked Celsius (when meant Fahrenheit). If any such corrections were made to data, then these changes were marked on the datasheet. The change was initialed on the datasheet and the reason noted.
- An example of an error that should not have been changed during data entry was if the observer recorded a species that the Coordinator knew was highly unlikely within that region and/or season (you will handle this during step three - documenting other changes).
- Manual check of data: After data were entered (by Volunteer or Coordinator), there was a manual check - comparing the electronic entry to the physical datasheet. This helped catch any data entry errors. If a data entry error was found, the correction was made. To indicate data had been through the manual check, the database had a checkbox to mark when the Coordinator had completed the review for each run of each route.
- Documenting other changes: Data for potential species misidentifications (false positive or false negative errors) were not changed until Coordinators conferred with the volunteers. If the volunteer agreed that they made an error, then the species entry was changed using the edit button. If the volunteer did not agree, then the species entry was not changed, but was flagged as suspect data and was never released. In either case it was documented by the database as to who made the change (or marked as questionable) and why.
- Reasons for changing data were designated as: observer error or data entry error.
- Reasons for questionable data were documented as: questionable identification, observer uncertainty, outside known distribution, or outside phenology.
- Datasheet archiving: State/provincial programs maintained the original datasheets.
Observer Training
Volunteers received training from the State Coordinators. Training covered how to collect calling survey data following the NAAMP unified protocols and identification of calling amphibians of their state or region. USGS provided State Coordinators with a PowerPoint presentation describing the unified protocol and how to collect data. States provided observers with local distribution and phenology information, to help observers learn what species were expected in their area and when they were likely to vocalize. States also provided observers with a training tape, audio disc, or mp3 files to help them learn the vocalizations of the frogs and toads of the state. USGS also created an on-line training resource, the Frog Call Quiz and Frog Call Lookup, where observers can practice and assess their frog call identification skills. Starting in 2006, observers had to take the assessment portion of the Frog Call Quiz (NAAMP Quiz) for their state or region and meet the detection index requirement each year. See the Frog Call Quiz section below for more information.
Frog Call Quiz
USGS created an on-line resource, the Frog Call Quiz for observer training and assessment. The Frog Call Quiz included:
- Frog Call Look-Up - a reference section where users could select species by common or scientific name to hear example frog calls and a description of the call. This section also included state species lists.
- Public Quiz - a practice section where users could select a state and then receive a 10 question quiz. Each quiz session was unique, as the sound file library included several hundred sound files for each state. Each sound file had one or more calling amphibians and users were asked to identify all species on the sound file. The public quiz provided immediate feedback and an opportunity to replay the sound files.
- NAAMP Quiz - an assessment section available for NAAMP participants only. Login required using the participant’s Route Number and Observer Number. As with the Public Quiz, each quiz session was unique and each question had one or more calling amphibians. The number of questions varied, depending on the number of species in the state or region. Sound files are randomly selected to provide a variety of species. Rare species were treated the same as more common species and could appear more frequently on the quiz than what was expected on survey nights. For the NAAMP Quiz, State Coordinators decided whether to have a state-wide quiz or regional quizzes within their state.
Starting in 2006, observers needed to take the assessment portion of the Frog Call Quiz (NAAMP Quiz) for their state or region and meet the detection index requirement on a biennial basis. Observers could retake the NAAMP Quiz as many times as needed to achieve this requirement. Observers were permitted to use any reference materials that would be used while collecting data (the quiz was "open book"). The minimum detection index was 65. The detection index was calculated as ((user's correct responses) - (misidentifications)) / (total possible correct identifications). Since misidentifications were subtracted from a user's correct responses, wild guesses may lower the detection index. It was possible to have a negative value.
Ideally, observers took the quiz as part of the beginning of their field season, to ensure the observers were ready to collect data. Since some southern states began surveying in January, the Frog Quiz considered November and December as belonging to the next year's field season. Thus an observer could take the 2006 NAAMP Quiz as early as November 1, 2005 and as late as October 31, 2006.
Data were not used for population trend analyses or publicly available unless the observer had met quiz requirements. Observers were expected to annually meet the detection index requirements. For observers who met the requirements in the previous year but were unable to take the quiz in the current year, their data were still used in analyses for both years. Data were not used for population trend analyses or publicly available in the following circumstances: observer never took the quiz, did not meet minimum detection index requirement for the current year, or went more than 2 years without taking the quiz after having met index requirement previously.
Index and Code Definitions
* A regional program may choose whether ambient noise is documented in yes/no format or by using the Massachusetts noise index.
The North American Amphibian Monitoring Program (NAAMP) was a collaborative citizen science effort between the US Geological Survey (USGS) and 26 partners (state agencies, universities, and nonprofit organizations) for monitoring calling amphibian populations over much of the eastern and central United States. Initiated in 1997, in response to needs set forth by the Declining Amphibian Populations Task Force (DAPTF) in 1994, NAAMP was designed to provide scientifically and statistically defensible, long-term distribution and trend data for calling frog and toad populations at both the state and regional level.
The national program closed at the conclusion of the 2015 season.
NAAMP Protocol
Route Creation
Routes were generated in a stratified random block design at USGS Patuxent Wildlife Research Center. Routes were then distributed to Regional Coordinators. These roadside routes were then groundtruthed to determine suitability (not too dangerous, not too noisy to hear) and stop placement. There were 10 stops per route. Two methods of stop placement were permitted: equidistant stops or stratified by habitat. In equidistant stop placement, each stop was exactly 0.5 miles apart, regardless of surrounding habitat. When stratified by habitat, the stops were at least 0.5 miles apart and were located at wetland habitats. The wetland habitat was appropriate potential habitat (pond, vernal pool, roadside ditch, etc) but the presence or absence of amphibians was not used as a stop selection factor. Some alteration of the route could occur during groundtruthing. Stop locations and any route alterations were shared with NAAMP to keep route maps accurate and up to date (although, this did not always occur). Once a route was groundtruthed and the 10 stops determined the route and stops was not changed, unless exceptional circumstances occurred, see Stop Inaccessibility, Stop Relocation, and Stop Retirement section of this document. In addition, some regions may have had nonrandom routes that were created by other methods, but not included in USGS analyses.
Seasonal Sampling Periods
Each state established three or four sampling periods to cover the calling phenology of its local species. States used three sampling periods, unless an additional period to target wood frogs was desired. The sampling periods were created to target the peak vocalization times for early-, mid- and late-season breeding amphibians and to assist observers in understanding when to collect data. A state may have subdivided into regions and established different sampling dates within these regions. Sampling periods could not overlap, but could be separated by an interval or begin and end on adjoining dates. States were permitted to adjust the sampling dates each year to account for an early or late arrival of the calling season.
Nightly Sampling Conditions
Surveys were meant to begin 30 minutes after sunset or later. No matter what time a route was started, it was meant to be completed by 1 a.m. Appropriate sampling conditions were based upon wind, sky, and air temperature conditions. For most regions the wind code was meant to be at level 3 or less, but the wind prone Great Plains region was permitted to sample at level 4 or less. Surveys should not have been conducted during heavy rainfall, but light rainfall was acceptable (sound of the rain may impair hearing ability).
The air temperature criteria were the minimum allowable temperatures, varying for each sampling period.
Data Collection
Stops were conducted in numerical order, in one night by one observer. We encourage, but do not require, that one observer conduct all surveys of a route in a given year. Because some observers had assistants who wished to collect data and occasionally a state coordinator had to assign multiple individuals to the same route to meet high volunteer demand, multiple observers were instructed to each fill out their own datasheet, separately and independently and to submit their datasheets under their name, not another observer’s name. Each observer was responsible for passing the FrogQuiz in order for their data to be released. All datasheets were returned to the State Coordinator for Data Quality Checking and archival purposes. This “one observer per datasheet” rule allowed each survey conducted to be of equal effort, without outside influence.
Observers recorded the time, sky code, and wind code, at the beginning and end of each survey to verify that the sampling conditions were met on the evening of the survey. At each stop air temperature was recorded to verify that sampling conditions were met on the sampling night; at least eight of the 10 stops must have met temperature guidelines. For Gulf Coast states that recorded air temperature only at the beginning and end of a survey, both temperature readings must have met these guidelines. Gulf Coast and Great Plains states required documentation of the last rainfall event, since routes should have been conducted within 3 days of rainfall, based on anuran calling behaviors in those regions.
At each stop the observer listened for 5 minutes, and recorded the amphibian calling index for each species heard. The 5 minute listening period had no initial waiting period. Starting in 2006, the observer also recorded the number of cars that passed during the listening period and whether the moon or moonlight was visible. Car counting was often conducted by an assistant. The observer indicated whether background noise impaired his/her ability to hear (most surveys used yes/no checkbox; some adopted the noise index developed by Massachusetts). If there was a major noise disturbance, lasting one minute or longer, the observer broke the listening period to avoid sampling during the excessive noise. If such a time out was taken, this was noted on the datasheet. After the major disturbance ended, the observer resumed listening for the time remaining.
Stop Inaccessibility, Stop Relocation, and Stop Retirement
- Stop Inaccessibility: Temporary stop inaccessibility may have occurred for some transient reason (i.e. traffic accident blocked road access).
- If only one stop was missed, the route was considered complete. The observer wrote on the datasheet which stop was missed and noted why in the comments section. When entering the data into the database, they marked the checkbox indicating which stop was missed.
- If more than one stop was missed, the route should have been re-ran on another night.
- Stop Relocation: Stop relocation was when a stop needed to be shifted to a new location, after the groundtruthing phase occurred. During groundtruthing the permanent stop locations are set (see groundtruthing guidelines). Stop relocations were a rare event.
- Stop relocation should only have occurred for safety reasons (i.e. route was safe before-or appeared to be, but perhaps a homeowner fired a gun in the air as warning to observer).
- Stops should NOT have been relocated because of habitat loss or lack of calling amphibians at the site.
- To relocate (for safety reasons) a stop, the State Coordinator should have used their best judgment on when it was necessary and where to relocate. It was preferable to move the stop a short distance away, not impacting the 0.5 mile minimum distance rule, when possible. If possible, then stops were relocated by creating a new stop at the end or beginning of the route and renumbering all the stops. A written record of when, why, and how a stop was relocated, was kept in most cases.
- Stop Retirement: Once the route was groundtruthed and listening stations established, these locations were permanent and locations were changed unless a safety issue arose. If habitat destruction occurred at a listening station, and a local extinction of amphibians occurred, this was important information. To document habitat destruction, the location should have been surveyed for three seasons beyond the destruction date. After three seasons of non-activity, the listening station was retired, and null data were assumed for this site in analyses from that point forward. A listening station could not be retired merely because the wetlands were uninhabited by anurans. These retired stops should have been revisited periodically to verify that no suitable habitat existed, but five minutes of listening was no longer required.
Data Review Process
State Coordinators performed data checks each year to ensure all entered data followed the same review procedures. Some checks and balances were incorporated into the database design (pop-up warning boxes, etc.), while others were procedures only Coordinators could do. These procedures were adopted at the Nashville NAAMP Coordinators meeting.
- All data entered same way: All datasheets were entered "as they appeared" and then "checked" for any errors. This pattern was obvious if the volunteer entered their own data, as the State Coordinator could not "check" the data before they were entered. This pattern was followed, even for datasheets that the State Coordinator entered. This way all data went through the same data review process. Also, by entering the data "as it was" and then making any necessary corrections after the fact, the database had a record of the correction and when and why it occurred.
- The only exceptions were "simple obvious data entry errors" such as the observer wrote 70 degrees and then marked Celsius (when meant Fahrenheit). If any such corrections were made to data, then these changes were marked on the datasheet. The change was initialed on the datasheet and the reason noted.
- An example of an error that should not have been changed during data entry was if the observer recorded a species that the Coordinator knew was highly unlikely within that region and/or season (you will handle this during step three - documenting other changes).
- Manual check of data: After data were entered (by Volunteer or Coordinator), there was a manual check - comparing the electronic entry to the physical datasheet. This helped catch any data entry errors. If a data entry error was found, the correction was made. To indicate data had been through the manual check, the database had a checkbox to mark when the Coordinator had completed the review for each run of each route.
- Documenting other changes: Data for potential species misidentifications (false positive or false negative errors) were not changed until Coordinators conferred with the volunteers. If the volunteer agreed that they made an error, then the species entry was changed using the edit button. If the volunteer did not agree, then the species entry was not changed, but was flagged as suspect data and was never released. In either case it was documented by the database as to who made the change (or marked as questionable) and why.
- Reasons for changing data were designated as: observer error or data entry error.
- Reasons for questionable data were documented as: questionable identification, observer uncertainty, outside known distribution, or outside phenology.
- Datasheet archiving: State/provincial programs maintained the original datasheets.
Observer Training
Volunteers received training from the State Coordinators. Training covered how to collect calling survey data following the NAAMP unified protocols and identification of calling amphibians of their state or region. USGS provided State Coordinators with a PowerPoint presentation describing the unified protocol and how to collect data. States provided observers with local distribution and phenology information, to help observers learn what species were expected in their area and when they were likely to vocalize. States also provided observers with a training tape, audio disc, or mp3 files to help them learn the vocalizations of the frogs and toads of the state. USGS also created an on-line training resource, the Frog Call Quiz and Frog Call Lookup, where observers can practice and assess their frog call identification skills. Starting in 2006, observers had to take the assessment portion of the Frog Call Quiz (NAAMP Quiz) for their state or region and meet the detection index requirement each year. See the Frog Call Quiz section below for more information.
Frog Call Quiz
USGS created an on-line resource, the Frog Call Quiz for observer training and assessment. The Frog Call Quiz included:
- Frog Call Look-Up - a reference section where users could select species by common or scientific name to hear example frog calls and a description of the call. This section also included state species lists.
- Public Quiz - a practice section where users could select a state and then receive a 10 question quiz. Each quiz session was unique, as the sound file library included several hundred sound files for each state. Each sound file had one or more calling amphibians and users were asked to identify all species on the sound file. The public quiz provided immediate feedback and an opportunity to replay the sound files.
- NAAMP Quiz - an assessment section available for NAAMP participants only. Login required using the participant’s Route Number and Observer Number. As with the Public Quiz, each quiz session was unique and each question had one or more calling amphibians. The number of questions varied, depending on the number of species in the state or region. Sound files are randomly selected to provide a variety of species. Rare species were treated the same as more common species and could appear more frequently on the quiz than what was expected on survey nights. For the NAAMP Quiz, State Coordinators decided whether to have a state-wide quiz or regional quizzes within their state.
Starting in 2006, observers needed to take the assessment portion of the Frog Call Quiz (NAAMP Quiz) for their state or region and meet the detection index requirement on a biennial basis. Observers could retake the NAAMP Quiz as many times as needed to achieve this requirement. Observers were permitted to use any reference materials that would be used while collecting data (the quiz was "open book"). The minimum detection index was 65. The detection index was calculated as ((user's correct responses) - (misidentifications)) / (total possible correct identifications). Since misidentifications were subtracted from a user's correct responses, wild guesses may lower the detection index. It was possible to have a negative value.
Ideally, observers took the quiz as part of the beginning of their field season, to ensure the observers were ready to collect data. Since some southern states began surveying in January, the Frog Quiz considered November and December as belonging to the next year's field season. Thus an observer could take the 2006 NAAMP Quiz as early as November 1, 2005 and as late as October 31, 2006.
Data were not used for population trend analyses or publicly available unless the observer had met quiz requirements. Observers were expected to annually meet the detection index requirements. For observers who met the requirements in the previous year but were unable to take the quiz in the current year, their data were still used in analyses for both years. Data were not used for population trend analyses or publicly available in the following circumstances: observer never took the quiz, did not meet minimum detection index requirement for the current year, or went more than 2 years without taking the quiz after having met index requirement previously.
Index and Code Definitions
* A regional program may choose whether ambient noise is documented in yes/no format or by using the Massachusetts noise index.