502.9 - Fundamental Science Practices: Preservation Requirements for Digital Scientific Data
This chapter provides requirements and procedures to ensure the preservation of all U.S. Geological Survey (USGS) digital scientific data and associated metadata.
01/13/2017
OPR: Office of Science Quality and Integrity
Instruction: This is a new Survey Manual (SM) chapter that replaces Instructional Memorandum (IM) OSQI-2015-04, dated February 19, 2015.
1. Purpose and Scope. This chapter provides requirements and procedures to ensure the preservation of all U.S. Geological Survey (USGS) digital scientific data and associated metadata. Preservation requirements for other scientific information products, nondigital data, paper records containing data or descriptions of data, or physical samples are addressed in other SM chapters.
2. References.
A. Open Data Policy—Managing Data as an Asset (Office of Management) memorandum, May 9, 2013)
B. Increasing Access to the Results of Federally Funded Scientific Research (Office of Science and Technology Policy (OSTP) memorandum, 22 February 2013)
C. Coordinating Geographic Data Acquisition and Access: The National Spatial Data Infrastructure (Executive Order 12906)
D. 305 DM 3 – Integrity of Scientific and Scholarly Activities
E. 378 DM 1 – Data Resource Management
F. SM 502.1 – Fundamental Science Practices: Foundation Policy
G. SM 502.2 – Fundamental Science Practices: Planning and Conducting Data Collection and Research
H. SM 502.4 – Fundamental Science Practices: Review, Approval, and Release of Information Products
I. SM 502.5 – Fundamental Science Practices: Safeguarding Unpublished USGS Data, Information, and Associated Scientific Materials
J. SM 502.6 – Fundamental Science Practices: Scientific Data Management
K. SM 502.7 – Fundamental Science Practices: Metadata for USGS Scientific Information Products including Data
L. SM 502.8 – Fundamental Science Practices: Review and Approval of Scientific Data for Release
M. SM 431.1 – Records Management Program
N. SM 600.5 – Information Technology Systems Security - General Requirements
O. SM 601.1 – USGS Web Standards
P. SM 1100.1 – Information Product Planning
Q. SM 1100.3 – USGS Publication Series
R. SM 1100.4 – Use of Outside Publications, Including Abstracts
S. USGS Records Disposition Schedules
T. USGS Data Management Web site
U. USGS Public Access Plan
3. Policy. It is the policy of the USGS to preserve scientific data and associated metadata funded by the Bureau’s information and research programs. All scientific data collected or created as a result of USGS funding must be preserved as follows:
A. Data Management Plans (DMPs) must clarify responsible parties for maintaining authoritative copies of USGS funded data. When scientific data are produced in cooperation with non-USGS entities, ownership and preservation of those data must be clearly documented in the DMP.
B. USGS digital scientific data and associated metadata must be discoverable, available, and useable (refer to the USGS Public Access Plan). In cases where access must be restricted because of security, privacy, confidentiality, or other constraints, data preservation activities, including associated metadata, are still required. Additionally, other Federal Government Open Data and Open Access requirements from the OMB and the OSTP for preservation of data collected through federally funded research must be met.
C. Data must be preserved in accordance with the USGS records disposition requirements (http://www.usgs.gov/usgs-manual/schedule/) and the Federal Records Act 36 CFR 1220.14. When creating permanent Federal records, consider long-term preservation needs and select formats that will facilitate storage or transfer to National Archives and Records Administration (NARA).
D. The USGS will retain an authoritative or original copy of all digital data and associated metadata for which it is responsible.
E. USGS digital data and associated metadata must be stored in digital repositories approved by the USGS. Non-USGS repositories may also be used for release of USGS data as long as the Bureau maintains the authoritative copy. Additional information on acceptable digital repositories for USGS scientific data is available at https://www.usgs.gov/office-of-science-quality-and-integrity/acceptable-digital-repositories-usgs-scientific.
F. Refer to IM OSQI 2016-01 for software preservation requirements.
4. Preservation Elements. Each element listed below represents a component of required digital scientific data preservation. Detailed guidance and specific implementation instructions are available on the USGS Data Management Web site: http://www.usgs.gov/datamanagement/preserve.php.
A. Storage and Geographic Location. Storage and geographic location involves storage systems, locations, and the planning for multiple copies of data. At a minimum, two complete copies of data, metadata, and documentation must be maintained. These two copies should be geographically separate; for example, a backup tape/disk can be stored in a location that is separate from the Science Center that is the primary storage location of the data and associated metadata. Backup copies of data should be tested and verified periodically to ensure ability to recover in the event of primary data loss.
B. Data Viability and Integrity. Data viability and integrity are ensured through procedures to maintain the accessibility, utility, accuracy, and consistency of data over their entire lifecycle. This includes activities to prevent, detect, and recover from unexpected or unintentional changes to data as well as activities that maintain the viability of data through technological changes over time. Examples include verifying file integrity at fixed intervals, maintaining procedures to replace or repair corrupted data, and upgrading operating systems.
C. Information Security. Information security includes procedures to prevent human-caused corruption or deletion of the data and unauthorized access to the data. Role-based authorization applies with regard to read, write, move, and delete actions (refer to SM 600.5).
D. Metadata. Metadata ensures proper documentation of USGS data to enable contextual understanding andlong-term usability (refer to SM 502.7). Complete metadata must be preserved and directly associated with the scientific data they describe. Aspects of metadata such as online linkages and contact information should be reviewed periodically to ensure their accuracy.
E. File Formats. Data file formats include file types, data structures, and naming conventions to aid other aspects of how data are packaged and managed. The USGS must comply with National Archives and Records Administration (NARA) formats for records deemed to be permanent at the time of transfer to NARA (refer to http://www.archives.gov/records-mgmt/policy/transfer-guidance-tables.html).
F. Hardware Obsolescence. Technology changes and hardware is upgraded and all digital data should be reviewed at the time of migration to new media to preserve the viability of the data and ensure successful transition to the new media. Data migration includes transferring data between storage types, formats, and computer systems.
5. Responsibilities.
A. Associate Directors and Regional Directors. Associate Directors (ADs) and Regional Directors (RDs) provide oversight and support for the data preservation activities in their mission and regional areas. They collaborate with each other to address issues or take corrective action with regard to data preservation activities.
B. Office of Science Quality and Integrity, Core Science Systems, and Office of Enterprise Information. The Office of Science Quality and Integrity (OSQI), Core Science Systems (CSS), and Office of Enterprise Information (OEI) are responsible for jointly developing USGS data management policy and collaborating on the development of related guidance and procedures, including those needed for trusted digital data repositories. The OSQI coordinates with the ADs and RDs as needed to address and resolve issues regarding the execution of this policy and related data-management review and approval processes. The OSQI maintains and communicates this and other FSP related policy documents. The CSS maintains comprehensive guidance and procedures related to data management, including preservation (refer to the USGS Data Management Web site). The OEI is responsible for the policy development, coordination, and overall management of the USGS Records Management Program that informs data preservation policy.
C. Science Center Directors, Managers, and Scientists. Science Center Directors, managers (project and program), and scientists involved in any phase of the USGS science data lifecycle (refer to SM 502.6) are responsible for complying with the preservation requirements in this policy Science Center Directors (or their designees) collaborate with their respective ADs and/or RDs, as they deem necessary, to assign or ensure the assigning of data managers to oversee their centers’ and offices’ data preservation activities. At the start of the project, as part of the data management plan, USGS scientists must identify an appropriate digital data repository that complies with the requirements in this chapter and records disposition requirements.
D. Data Managers. Designated data managers are responsible for coordinating and enacting preservation activities for USGS scientific data. Data managers will ensure that the preservation activities for which they are assigned are met. They also collaborate with the data producers and the USGS Records Officer to ensure the appropriate records management requirements for their data are met.
E. Records Officer. The Records Officer in the OEI ensures bureau-wide policies, standards, and procedures including records schedules are in place to provide guidance on creating accurate and complete records, maintaining these records throughout the science data lifecycle, and transferring permanent scientific data in accordance with applicable USGS and NARA records management requirements.
/s/ Jose R. Aragon 01/13/2017
__________________________________________ _______________
Jose R. Aragon Date
Associate Director for Administration