Building Search Queries

Basic Search Queries

• Items
• Folders

Search USGS Data Releases in ScienceBase

• USGS mission area 
• USGS science topic keyword 
• USGS cost center name 
• Persistent identifiers (ORCIDs and DOIs) 
• Additional options 

Advanced Search Queries

• Filters
• Search Results Control
• Spatial Query
• Extent Query
• Vocabulary Search

Note: the instructions on this page focus on query URLs. The same query logic can be applied through programmatic workflows using the ScienceBase Python and R libraries. For helpful examples written in Python, see this Jupyter Notebook.

Basic Search Queries

Items

All pages in ScienceBase have an item ID. The ID is the alphanumeric string at the end of a page's URL: 
https://www.sciencebase.gov/catalog/item/568e9959e4b0e7a44bc58c00

Search for items containing a word using the keyword q search:
https://www.sciencebase.gov/catalog/items?q=bison

To view the search results in JSON or CSV format, add "&format=json" or "&format=csv" to the URL:
https://www.sciencebase.gov/catalog/items?q=bison&format=json
https://www.sciencebase.gov/catalog/items?q=bison&format=csv

The default number of search results returned in JSON or CSV format is 20. To return additional results, use the parameter "max". The upper limit for this parameter is 1000. 

https://www.sciencebase.gov/catalog/items?q=bison&format=json&max=500
https://www.sciencebase.gov/catalog/items?q=bison&format=csv&max=500

An "offset" parameter can be used when there are more than 1000 results. Please note that specified "max" and "offset" integers in search parameters must be in increments of 5. Searches with result sets larger than 150,000 will require subsetting - please contact the ScienceBase team for advanced search guidance.

Folders

Items in ScienceBase are organized into a hierarchy defined by the terms "parent" and "child" item. A folder in ScienceBase is considered the "parent" item and nested items are "child" items. For multiple levels in the item hierarchy, the terms "ancestors" and "descendants" are used.

Find the folder ID (itemID) at the end of the URL, for example:
https://www.sciencebase.gov/catalog/item/55b7a980e4b09a3b01b5fa6f

Perform a filter search by adding the parameter "&filter=".

• Search for all items in the folder using the "parentId" filter:
  https://www.sciencebase.gov/catalog/items?q=&filter=parentId=55b7a980e4b09a3b01b5fa6f
• A "parentId" query returns items nested one level under the parent. To find all the descendants of an item, use the "ancestors" filter:
  https://www.sciencebase.gov/catalog/items?q=&filter=ancestors=55b7a980e4b09a3b01b5fa6f

More than one filter can be added to a query URL.

• Search for all items in two folders:
  https://www.sciencebase.gov/catalog/items?q=&filter=parentId=55b7a980e4b09a3b01b5fa6f&filter=parentId=5ab27be8e4b081f61ab45ebb&conjunction=parentId=OR

Note: ScienceBase communities are folders and can be queried in the same way. 

Search USGS Data Releases in ScienceBase

USGS data releases are formally published products that meet all the criteria of USGS data policies and have been approved for dissemination. To limit search results to data release products, add "filter=systemType=Data Release" to any query URL. The following lists of queries demonstrate how to filter this collection by categories including USGS organizational groups and science topic keywords.

Note: writing query URLs is an advanced method. For a more user-friendly, interactive way to explore data release metrics, see the ScienceBase data release dashboard.
 

USGS mission area 

Example query URL: https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data+Release&filter=party={name:Core Science Systems,type:USGS Mission Area} 

USGS science topic keyword 

Example query URL: https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data+Release&filter=tags={type:USGS Scientific Topic Keyword,name:Water Quality}

USGS cost center name 

Example query URLs:  

• https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data+Release&filter=party={type:SDC Data Owner,name:Science Analytics and Synthesis}
• https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data+Release&filter=party={type:SDC Data Owner,name:Science Analytics and Synthesis}&format=csv&max=1000

Persistent identifiers (ORCIDs and DOIs) 

Query by ORCID (https://orcid.org/). Returns items where the associated person is listed in the contacts section (e.g., as an author or point of contact). 

Note: this query will only detect a person’s name in the contacts section if the name is matched to its ScienceBase directory record and if the directory record contains the person’s ORCID.
https://www.sciencebase.gov/catalog/items?q=&filter=party={orcId:0000-0001-8054-5139}

Query by ORCID and limit search results to data release products: https://www.sciencebase.gov/catalog/items?q=&filter=party={orcId:0000-0003-1329-5285}&filter=systemType=Data+Release  

Query by DOI: https://www.sciencebase.gov/catalog/items?filter=itemIdentifier={type:DOI,key:doi:10.5066/P99K85SW} 

Additional options for data release queries 

• Data releases in progress (not yet published):  
  https://www.sciencebase.gov/catalog/items?q=&filter=browseCategory=Data Release - In Progress 
  Note: in-progress data releases have restricted permissions. Users will need to log in to view in-progress data releases for which they have read access. 
    
• Data releases published within a date range (customize URL by date range): 
  https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data Release&filter=dateRange={dateType:Publication,start:2018-10-01,end:2019-10-01} 
    
• Data releases that have been revised after publication (customize URL by date range): 
  https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data Release&filter=dateRange={dateType:revision,start:2018-10-01,end:2019-10-01} 
    
• Note: to filter the search results of any query by science center, append the following to the URL and customize by name: &filter=party={type:SDC Data Owner,name:Science Analytics and Synthesis} 
  https://www.sciencebase.gov/catalog/items?q=&filter=systemType=Data Release&filter=dateRange={dateType:Publication,start:2018-10-01,end:2019-10-01}&filter=party={type:SDC Data Owner,name:Science Analytics and Synthesis}
   
• Query for information about data release views and downloads: 
   
  https://www.sciencebase.gov/catalog/item/metrics/52a7f9efe4b0de1a6d2dd10a?&fromDate=2017-01-01&toDate=2020-02-07&timescale=week&format=json
   
  https://www.sciencebase.gov/catalog/item/metrics/52a7f9efe4b0de1a6d2dd10a?&fromDate=2017-01-01&toDate=2020-02-07&timescale=week&format=json&category=files

Advanced Search

The following information is intended for advanced ScienceBase users.
 

Search with Filters

Allowed Filters
ancestors
ancestorsExcludingLinks
browseCategory
browseTag
contactNameLike
dateRange
extentQuery
facets.facetName
files.name
files.originalMetadata
ids
itemIdentifier
itemIdentifierKeys
itemIdentifierSchemes
itemIdentifierTypes
linkParentId
parentId
parentIdExcludingLinks
parentIdIncludingLinks
party
​​​​​​partyWithName
projectStatus
spatialQuery
spatialQueryOnExtents
spatialQueryOnXFields
systemType
tags
tagslq
webLinks

Examples

browseType
filter=browseType=ArcGIS Service Definition

browseCategory
filter=browseCategory=Data

dateRange
Format is dateRange={choice:<choice>,start:<yyyy-mm-dd>,end:<yyyy-mm-dd>,dateType:<date type>}
filter=dateRange={choice:year,dateType:creation}
filter=dateRange={start:2014-03-04,end:2020-03-04,dateType:creation}

facets
filter=facets.facetName=Shapefile

itemIdentifier
Format is itemIdentifier = {type:<type>,scheme:<scheme>,key:<key>}
filter=itemIdentifier={type:DOI,key:doi:10.5066/P913N38G}

party
filter=party=Core Science Systems
filter=party={name:Core Science Systems,type:USGS Mission Area}
filter=party={orcId:0000-0001-8054-5139}

partyWithName
filter=partyWithName=17387_Geosciences and Environmental Change Science Center

tags
filter=tags=water
Format is tags={name:<name>,type:<type>,scheme:<scheme>}
filter=tags={'name':'water'}

Filter Conjunction
filter=tags=water&filter=tags=birds&conjunction=tags=OR

Note: "should" instead of "filter" performs an "OR" search
should=tags=water&should=tags=birds

Multiple Filters
filter0=facets.facetName=NGGDPP Item&filter1=browseCategory=Physical Item
 

Search Results Control

fields -- specifies which fields in the sbJSON to return, e.g., title, summary, distributionLinks, webLinks, previewImage.
fieldset -- a wrapper for a specific set of fields. Resolves to a "field=" parameter.
loadInstances -- loads the result items from mongoDB vs. just responding with the elasticsearch response. Some fields, such as distributionLinks, are only available from mongoDB, so the loadInstances parameter would be required to fetch those.
max -- maximum number of items to return per request. Note: ScienceBase enforces an upper limit of 1000. If "max" is not specified, the default is 20.
offset -- paging parameter. Returns items beginning at offset.
order -- can be ascending (asc) or descending (desc).
sort -- sorts by title, dateCreated, lastUpdated, or firstContact.
​​​​​

Spatial Query

This query type is a query filter in the following format:
https://www.sciencebase.gov/catalog/items/?q=&filter=spatialQuery=<quer…;

Spatial search supports only the following types of shapes in any format:

• point and multipoint
• linestring
• polygon and multipolygon
• envelope

The query supports either WKT or GeoJSON input spatial data in one of four different forms:

• Raw WKT, e.g., POINT(-105 43)
• JSON with WKT and relation, e.g., {wkt:POINT(-105 43),relation:disjoint}
• Raw GeoJSON, e.g., {type:point, coordinates:[-105.078056,40.559167]}
• JSON with GeoJSON and relation, e.g., {shape:{type:point,coordinates:[-105.078056,40.559167]},relation:disjoint}

The relation field must be one of:

• intersects
• disjoint
• within

# Polygon extent query
filter=spatialQuery={wkt:"POLYGON((-102 37,-102 41,-109 41,-109 37,-102 37),(-104 38,-106 38,-105 39,-104 38))",relation:intersects,fields:Extents}
 

Extent Query

This query type is a query filter in the following format:
https://www.sciencebase.gov/catalog/items/?q=&filter=extentQuery=<query…;

There are two forms the contents of the query can take:

• Extent ID, e.g., 36
• JSON with extent ID and relation type, e.g., {​​​​​​extent:36,relation:within}

The relation field must be one of:

• intersects
• disjoint
• within

# Find items whose footprint intersects Colorado (extent 36) with a buffer of .125 degrees.
filter=extentQuery={extent:36,relation:within,buffer:'.125'}
 

Vocabulary Search

The ScienceBase Vocab application has two query builder forms for creating vocab search queries: 
https://www.sciencebase.gov/vocab/vocabularies/queryForm
https://www.sciencebase.gov/vocab/terms/queryForm

View all vocabularies in ScienceBase Vocab:
https://www.sciencebase.gov/vocab/categories/get?nodeType=vocabulary&max=100&offset=0&format=json

Get IDs back for all vocabularies in a folder:
https://www.sciencebase.gov/vocab/categories/get?parentId=58b45852e4b0f974afcf03b4&nodeType=vocabulary&max=100&offset=0&format=json

Get terms back from a vocabulary, by vocab ID:
https://www.sciencebase.gov/vocab/categories/get?vocabKey=58b45852e4b0f974afcf03b4&nodeType=term&max=100&offset=0&format=json