North American Bat Data Integration

The purpose of this project was to integrate the Bat Banding Program data (1932-1972) and the U.S. and Canada diagnostic data for white-nose syndrome with the USGS Bat Population Data (BPD) Project and provide the bat research community with secure, role-based access to these previously unavailable datasets. The objectives of this project were to: 1) integrate WNS diagnostic data into the BPD (http://my.usgs.gov/bpd - content no longer available); 2) incorporate the historical bat banding data produced by the Bat Banding Program into the BPD; and, 3) develop the application programming interfaces (APIs) and data services required to share these datasets with DOI and USGS enterprise data resources, BISON and Sciencebase.



Principal Investigator : Laura E Ellison

Cooperator/Partner : Anthony L Everette, Anne E Ballmann, Suzanne C Peurach, Jeremy T. Coleman



Given the differing states of the bat banding and WNS datasets, we developed two technical approaches to migrating data and a third for developing REST services for improved data access. First, the original bat banding data collected from 1932 to 1972 were documented and maintained in 90 drawers of 3x5-inch index cards containing banding information for approximately 1.5 million individual bats. The file drawers were shipped to the Fort Collins Science Center and are currently being scanned into digital format. Data managers transcribe the digital records into the BPD “Observational Data Workbook” which facilitates batch data uploads to the BPD. To date, approximately 250,000 individual capture and recaptures have been entered into the BPD. 



Our technical approach for the second dataset, the WNS diagnostic data, was simpler due to the source data’s digital format. We conducted a data mapping exercise to document the existing WNS diagnostic data and validate those source data types against the existing BPD data model. Scripts were then used to transfer the existing data to the BPD. We developed a “Bat Population Data Project License Agreement” with the USGS Technology Transfer office that protects sensitive bat data (e.g., location, disease, and species data). 



Lastly, we developed a series of REST services that provide structured query options for users to conduct custom searches for BPD data based on their BPD data project access permissions. For example, DOI users can conduct a custom search for their BISON eligible data and export it as a single CSV that can be quickly incorporated into the BISON data workbook template for processing.



There are several benefits of this project. First, by integrating the WNS diagnostic data into the BPD, those data are now available to the WNS Working Groups, other USGS scientists, conservation managers, and the greater research community within a secure, role-based data management environment. Second, when completed, the project will provide 1.5 million indexed and discoverable bat species occurrence records will be available to bat research community as well as the general public. Third, by creating secure BPD data services WNS and bat banding data are available to BPD users as well as web and data applications used by bat researchers and policy makers. These benefits directly relate to the CDI Science Support Framework by providing applications and web services that make data and information assets accessible.



Deliverables:

• 250,000 records from the historical U.S. Government Bat Banding Program Data (1932-1972). We will continue entering and importing the remainder of the files in FY15-16. (https://my.usgs.gov/bpd/dataProject/show/54 - content no longer available)
• White-nose Syndrome (WNS) diagnostic laboratory data. These data have been migrated in their entirety to BPD. However, due to the sensitive nature of this data it is not publicly available and can only be accessed by the WNS Data Management Team only. The BPD or WNS Teams can provide a live tour of the data to interested CDI parties.  
• Bat Population Data Project License Agreement. This agreement is not available online, but by request. We plan to provide this document through the BPD by the end of FY15.
• BPD Observational Data Workbook and Upload (https://my.usgs.gov/bpd/uploadXL/downloadObsTemplate - content no longer available)
• BPD Rest Services
• The USGS Bat Population Data (BPD) Project web page. (https://www.fort.usgs.gov/science-tasks/2217 - content no longer available)
• The USGS Bat Population Database product web page (https://www.fort.usgs.gov/publication/23233 - content no longer available)
• USGS Highlight (June 2015): The USGS Bat Population Data Project Partners with WNS Laboratories to Improve Disease Tracking and Data Management.
• Ellison, L.E., A.L. Everette, A.E. Ballmann, S.C. Peurach, and J. Coleman. 2014. North American Bat Data Integration. [abs.]. In: Federation of Earth Science Information Partners meeting, Frisco, Colorado, 15 July 2014. Federation of Earth Science Partners. P.1. (https://www.fort.usgs.gov/products/23762 - content no longer available) 
• Map of WNS spread (https://www.sciencebase.gov/flexviewer/wns/ - content no longer available)

Note:   this description is from the FY14 CDI Annual Report