Evidence on the effectiveness of small unmanned aircraft systems (sUAS) as a survey tool for North American terrestrial, vertebrate animals: a systematic map protocol

Abstract

Background

Small unmanned aircraft systems (sUAS) are replacing or supplementing manned aircraft and ground-based surveys in many animal monitoring situations due to better coverage at finer spatial and temporal resolutions, access, cost, bias, impacts, safety, efficiency, and logistical benefits. Various sUAS models and sensors are available with varying features and usefulness depending on survey goals. However, justification for selection of sUAS and sensors are not typically offered in published literature and existing reviews do not adequately cover past and current sUAS applications for animal monitoring nor their associated sUAS model and sensor technologies, taxonomic and geographic scope, flight conditions and considerations, spatial distributions of sUAS applications, and reported technical difficulties. We outline a systematic map protocol to collect and consolidate evidence pertaining to sUAS monitoring of animals. Our systematic map will provide a useful synthesis of current applications of sUAS-animal related studies and identify major knowledge clusters (well-represented subtopics that are amenable to full synthesis by a systematic review) and gaps (unreported or underrepresented topics that warrant additional primary research) that may influence future research directions and sUAS applications.

Methods

Our systematic map will investigate the current state of knowledge using an accurate, comprehensive, and repeatable search. We will find relevant peer-reviewed and grey literature as well as dissertations and theses using online publication databases, Google Scholar, and by request through a professional network of collaborators and publicly available websites. We will use a tiered approach to article exclusion with eligible studies being those that monitor (i.e., identify, count, estimate, etc.) terrestrial vertebrate animals. Extracted data concerning sUAS, sensors, animals, methodology, and results will be recorded in Microsoft Access. We will query and catalogue evidence in the final database to produce tables, figures, and geographic maps to accompany a full narrative review that answers our primary and secondary questions.

Background

Small unmanned aircraft systems [sUAS, also commonly referred to as unmanned aerial vehicle (UAV), unmanned vehicle system (UVS), unmanned aircraft (UA), remotely piloted aircraft (RPA), or drones], are defined as powered aircraft controlled either remote or by automatic programming, weighing less than 24.9 kg, and without an onboard human pilot [1, 2]. In many situations, sUASs are replacing or supplementing manned aircraft and ground based surveys as they have demonstrated promise for developing automated and standardized animal assessments from digital aerial surveys [3,4,5,6]. Primary benefits of using sUASs for animal surveys include their ability to cover expansive areas at fine spatial and temporal resolutions, reduce surveyor bias and labor costs, access inconvenient locations, minimize environmental impacts including problematic animal behavior, and increased personnel safety and logistical operations compared to manned aircraft [4, 5, 7,8,9,10,11,12,13,14,15].

To date, sUAS have been used for a wide variety of animal assessments including wildlife and domestic animals. Monitoring efforts include counts for abundance or richness, behavioral studies, location, movement tracking, and habitat quality assessments for a diverse array of taxa [4] including birds (e.g., [5, 15,16,17,18]), mammals (e.g., [19,20,21,22]), and reptiles (e.g., [23]) in both marine and terrestrial systems. Multiple commercial sUAS models and sensors are available and vary in their usefulness by survey goals and situations [10, 16]. For example, sUAS models (e.g., DJI Matrice 600 Pro) may be chosen based on platform type (e.g., fixed-wing or multirotor), ease of use, payload capacity to carry the desired sensor, or cost [4], and sensors may be chosen based on what type of remotely sensed data are desired and survey conditions (e.g., optical sensors for determining animal behavior; see [24]; or thermal sensors for detecting cryptic nocturnal animals; see [25]). ‘Off-the-shelf’ sUAS packages further support widespread use of sUAS and exploration of new applications [16]. However, recent rapid increases in sUAS use, improvements to associated model, sensor, and computer vision technologies, and a common interest to incorporate sUAS in myriad situations have stemmed from a relatively sparse foundation of scientific investigations, common approaches and limited shared knowledge of operating guidelines to effectively incorporate this new technology. For example, justifications for selecting particular sUAS models and sensors are typically not offered in studies, and standard reporting mechanisms have only recently emerged [26]. Further confounding the use of sUAS for animals counts are a lack of bias corrected estimates (i.e., estimates that account for sampling errors such as not detecting the presence of an individual or double counting individuals), despite calls for such studies [27]. Bias-corrected estimates are important to determining best practices for using sUAS as an effective and reliable animal monitoring tool as they allow accurate estimates of animal populations [27, 28].

Systematic maps aim to answer broader questions than systematic reviews by collating, describing, and cataloging evidence related to the topic of interest [29] while following reporting standards for systematic evidence synthesis (see Additional file 1 for ROSES checklist). A preliminary literature search returned no existing systematic maps pertaining to this issue, and yet, a systematic map would set the necessary foundation for an explosion of scientific rigor supporting sUAS applications in animal monitoring. Most published reviews focus on future sUAS use [10], overall accomplishments and challenges [5, 6], animal behavioral responses [30], or general research summaries [4, 17]. A systematic map, however, would at minimum, allow evaluation and summarization of past and current sUAS applications for animal monitoring among sUAS model and sensor technologies, taxonomic and geographic scope, flight conditions and operational considerations, spatial distributions of sUAS applications, and reported technical benefits and pitfalls. From the generated body of evidence, standard reporting mechanisms, selection criteria, and use applications could be revealed to complement recent efforts (e.g., [26]) and investigate potential bias associated with sUAS animal surveys.

Our objective is to develop a systematic map to consolidate evidence in the aforementioned areas affecting and pertaining to use of sUASs to monitor animals in terrestrial environments. Given the rapid expansion of sUAS technology, we envision this systematic map informing future researchers, practitioners and other end users planning to use sUAS in terrestrial ecosystems to monitor animals, especially wild animals, of the best practices based on the current state of knowledge. While much literature has been published in marine systems, this systematic map aims to focus on vertebrates in terrestrial systems only, and primarily those of North America, due to interest of collaborators, stakeholders, and funders whose overall goal involves understanding and mitigating terrestrial animal-vehicle collisions [31, 32]. The systematic map will also elucidate major knowledge clusters (well-represented subtopics that are amenable to full synthesis by a systematic review) and gaps (unreported or underrepresented topics that warrant additional primary research) among sUAS-animal related studies, as well as summarizing current applications, as previously mentioned, and a repeatable framework for future sUAS assessments.

This protocol was developed in collaboration with Mississippi State University Department of Wildlife, Fisheries and Aquaculture (MSU-WFA), the United States Department of Agriculture Animal and Plant Health Inspection Service, Wildlife Services, National Wildlife Research Center (USDA-APHIS-WS-NWRC), and the Federal Aviation Administration (FAA). These three entities are all considered stakeholders. Authors are members of these entities with the exception of the FAA, whose senior members reviewed this systematic map protocol. The main authors discussed and refined the scope and objective of this systematic map during initial project planning meetings. The findings of this systematic map will be of direct interest to each of these stakeholders in addition to numerous other stakeholders in academia, research, industry, and other government agencies.

Objective of the review

The objective of this review is to determine the current state of knowledge regarding how sUAS have been used to monitor terrestrial, vertebrate animals. While sUAS have been used for automated and standardized animal assessments using digital aerial surveys, the extent to which they have been used has not been fully summarized, including study specifics such as effectiveness and accuracy. This review will provide a quantitative, repeatable process to develop objective selection criteria and rank sUAS models and sensors for monitoring and counting animals in terrestrial environments. Our specific goal is to provide a comprehensive, catalogued state of knowledge surrounding sUAS models and sensors used to monitor terrestrial animals.

Primary question

What evidence exists on the effectiveness of sUAS as a survey tool for terrestrial, vertebrate animals?

Secondary questions

• 1. What sUAS models and sensors are most used?

• 2. What are the common statistical approaches and field methodologies of sUAS applications for monitoring animals?

• 3. Do sUAS applications for monitoring animals differ among geographic ranges, habitat types (i.e., land covers), species or species groups, survey models, or survey environments?

• 4. What factors affect accuracy (i.e., sampling bias) in sUAS imagery (e.g., animal size, behavior, land/water cover, weather or light conditions, etc.)?

• 5. What are the common constraints of sUAS for monitoring animals (e.g., sUAS models and sensors, government restrictions, sensor calibration, expense, battery life, etc.)?

• 6. What are the suggested statistical approaches and characteristics of sampling designs that can lead to a consistent set of best practices to avoid, reduce, and correct sampling bias from sUAS aerial imagery while still achieving project goals?

Components of the primary question

Population (P): all terrestrial vertebrate wildlife species and domestic animals (inclusive of wildlife that also commonly occur in aquatic systems but are typically referred to as terrestrial such as shorebirds, waterfowl, turtles, or crocodilians).

Index test (I): the technology of interest (sUAS).

Target condition (T): presence or abundance of population.

Methods

Searching for articles

Search terms and languages

All searches will be performed in English (i.e., search terms will only be in English), and only studies published in or translated to English will be included in this systematic map due to the languages understood by the systematic map team. Some search terms are specific to North America due to stakeholder driven requirements, but the geographic scope of studies returned or included in this systematic map will not be limited.

When applicable, search terms were truncated and a wildcard (*) added at the end of the root word to include all alternate forms of root words, in order to account for alternative spelling or hyphenation (e.g., estimat* to account for estimate, estimation, estimating, estimates). Dollar signs ($) were added when applicable to designate the addition of one extra character. Quotation marks were placed around some multiple word terms to allow for the search of exact phrases. All search terms and phrases were combined using the Boolean operators AND or OR. Additional terms for the three major groups will be added if found and deemed necessary from literature and reference searches. All searches will be conducted on the article title, abstract, and keywords with the exception of Google Scholar that only allows for whole text or title only searches.

Search string

We conducted a test search on October 20th 2020, in multiple databases, as part of a scoping exercise to help build search terms and ensure the correct use of operators to yield the best performance in returning results. An example search string that produced efficient results with  >  99.99% accuracy among two searchers (1502 in Web of Science, 2041 in Scopus, 172 in Wildlife and Ecology Studies Worldwide, and 144 in Proquest Dissertations and Theses) is presented below:

[(UAS$ OR UAV$ OR UVS$ OR RPA$ OR “unmanned aerial system$” OR “unmanned aerial vehicle$” OR “unmanned vehicle system$” OR “unmanned aircraft” OR “remotely piloted aircraft” OR “unoccupied aerial vehicle$” OR “unoccupied aerial system$” OR “drone$”)

AND

(animal* OR avian OR bird* OR mammal* OR reptil* OR wildlife OR carnivor* OR cattle OR deer OR furbearer* OR livestock OR mesocarnivore* OR shorebird* OR ungulate* OR waterbird* OR waterfowl* OR alligator* OR blackbird* OR cow* OR coyote* OR dove* OR eagle* OR geese OR goose OR gull* OR hawk* OR hog* OR owl* OR passeri* OR pig* OR quail* OR raptor* OR snake* OR starling* OR “terrestrial vertebrate$” OR turkey* OR turtle* OR vulture*)

AND

(abundance* OR assess* OR count* OR estimat* OR monitor* OR population*)]

Publication databases

Relative to online publication databases, we will perform searches through Mississippi State University Libraries on the Web of Science search platform, Scopus, Wildlife and Ecology Studies Worldwide, and Proquest Dissertations and Theses. Databases included in the Web of Science search platform are the Core Collection, SciELO Citation Index, and Zoological Record. Databases were chosen based on the comprehensive coverage of published literature.

Internet searches

An internet search will be conducted using Google Scholar, and the first 100 or 10% results (whichever is fewer) sorted by relevance will be examined.

Supplementary searches

A search for non-peer reviewed literature will be conducted by request through a professional network of collaborators (e.g., United States Department of Agriculture, Federal Aviation Administration, United States Department of Defense, The Wildlife Society) and publicly available websites including Research Gate (http://www.researchgate.net), LinkedIn (http://www.linkedin.com), Academia (http://www.academia.edu) and Twitter (http://twitter.com) as well as the Ecological Society of America mailing list (https://www.esa.org/membership/ecolog/) (Additional file 2). Government reports, white papers, gray literature, and information from conference proceedings not returned from our primary searches will be included and undergo all screening. We will also scan reference lists of all articles included after full text screening for our aforementioned search terms, review articles, and “cited by lists” for the top-20 most cited articles, to search for relevant, but missed articles which will then be added to the search results.

Search settings

To help control for bias injected into searches by learning algorithms of internet browsers, browser history and cookies will be disabled when conducting all searches. The search team will use “InPrivate” or “incognito” mode and will not access any electronic accounts. All searches will be conducted by two search team members to ensure a consistent number of returned articles.

Comprehensiveness of the search

To evaluate the comprehensiveness of the search strategy, we compiled a list of 41 benchmark articles (Additional file 3) from and including two recent reviews; we included articles listed under the “wildlife research and management” section of one review [33] and under the “UAS” section of another review [34]. Articles deemed outside the realm of our search (e.g., marine, non-sUAS, etc.) were excluded from our list of benchmark articles. We will compare the article records returned from our search to this list, and if any are found to be missing, the search string will be amended.

Article screening and study eligibility criteria

Screening process

We will screen article records using a hierarchical approach in the order of title, abstract, and full text. Information for all article records returned from our search will first be imported to Rayyan to resolve duplicates and screen for relevance (i.e., those using UAS to monitor terrestrial animals) [35]. Any article records that cannot be deemed relevant or irrelevant will be included in the next stage of screening (i.e., records that cannot be determined as relevant at the title stage will move forward to the abstract stage of screening). A subset of 100 or 10% of article records (whichever is fewer) at each stage will be assessed by 2 reviewers, and we will calculate Cohen’s kappa coefficient to determine agreement between reviewers, with Κ  ≥  0.6 indicating consistency. If inconsistency occurs, reviewers will discuss, clarify, and modify inclusion criteria as necessary and a new subset of article records will be reviewed, and reviewer agreement assessed. Any article records moving to the full text stage will be accessed using licenses through Mississippi State University, Google Scholar, U.S. Department of Agriculture, or personal inquiries with corresponding authors of articles. We will track article record exclusion stages and record exclusion reasonings in an Excel table and include this table in the systematic map as additional material.

Eligibility criteria

Eligible population: all terrestrial vertebrate animals including humans, wildlife, and domestic animals will be deemed as eligible subjects.

Eligible index test: any studies that use sUAS technology to monitor (i.e., identify, count, estimate, etc.) eligible populations will be included.

Eligible target condition: any study reporting presence or abundance of animal population will be included. Studies observing strictly behavior or deterrence, or opinion, comment, or discussion type manuscripts will be excluded from the systematic map.

Eligible study designs: any studies designed to count or monitor terrestrial vertebrate animals will be deemed as eligible studies. We will only evaluate studies published in or translated to English. We will not apply any date restrictions.

Study validity assessment

No formal study validity assessment will be conducted, but study design elements will be coded and reported.

Data coding strategy

Data extracted from studies will included a variety of aspects from categories including bibliographic information, study characteristics, index information, population information, and target information. See Additional file 4: Table S1 for a complete overview of our proposed data coding strategy. These data will initially be recorded in Microsoft Access to reduce redundancies in the database. The extracted information will be used to summarize whether sUAS can be used to monitor and count animals. If data from an article are confusing or not easily comprehensible, we will contact corresponding authors for clarification. Two reviewers will extract data from a subset of 100 or 10% of articles (whichever is fewer) to determine data accuracy of extracted information, again using kappa coefficients of Κ  ≥  0.6 to indicate consistency, and discussing, clarifying, and modifying data extraction criteria if necessary.

Study mapping and presentation

In the resulting systematic map, we will describe the process and include a summarized narrative and numbers of articles for each stage of the inclusion process and extracted data. We will publish the final database with all extracted data, both as an Excel spreadsheet and an Access database. Different queries from the final database will be used to produce tables, figures, and geographic maps, which will be accompanied and fully explained using a narrative review to answer our primary and secondary questions. Any additional subtopic or questions identified through the course of the systematic mapping process will be described in detail in the resulting systematic map. We will also create heat maps of study frequencies to identify major knowledge clusters and gaps which will both be discussed for future research. Finally, we will make recommendations for priorities in future research on using sUAS to monitor or count terrestrial vertebrate animals.