What’s that frog? Putting the public to the test with frog call identification

Just how easy is it to tell a Striped Marsh Frog from a Spotted Marsh Frog, just by listening to their calls? Recently, we asked citizen scientists to try their hand at FrogID validating to see just how easy it is to determine “what’s that frog?”

Globally, citizen science is leading to the rapid collection of invaluable biodiversity records across a wide range of plants, animals, and fungi. The Australian Museum’s flagship citizen science project, FrogID is no exception, having amassed over 1.1 million records of frogs in just seven years! But while this biodiversity data collection is incredible, it can be challenging to review and identify such large quantities of biodiversity data using expert validation. So, we put people to the test to see just how challenging it is for members of the public to identify frogs from just their calls.

In August 2022 to February 2023, we asked the public to try their hand at identifying frog species in FrogID recordings from the Greater Sydney area using the Australian Museum’s DigiVol platform. We provided a guide to the likely species present and their calls, but didn’t provide all the tools or the intense training provided to the FrogID team before they identify calls submitted to the FrogID project. We then compared the public responses with those from our FrogID team, allowing us to evaluate how accurate the public identifications were.

We found that public frog identifications were correct in only 57% of the recordings (ie. every frog species calling was correctly identified with none missing). However, frog identification accuracy varied considerably according to a variety of factors, particularly when it came to the number of frog species calling at once. FrogID recordings are often of more than one species calling in a multi-species chorus – in fact, FrogID citizen scientists have recorded up to 13 species calling in a single recording! But, with so many different frogs calling at the same time, these multi-species choruses can be difficult to identify, even for the FrogID team. Not surprisingly, the public were most accurately able to identify recordings in which only one species was calling, but as more species were added to the mix, we saw a decrease in the number of completely correct identifications.

We also found that some frog species were inherently easier to identify than others, often depending on how common, loud, or distinctive the calls of the species were. Common and distinct frog species such as Peron’s Tree Frogs (Litoria peronii) and Striped Marsh Frogs (Limnodynastes peronii) were identified with much higher accuracy than more uncommon and generally quiet frogs, such as the Stony Creek Frogs (Litoria lesueuri and Litoria wilcoxii). While some of these quieter species were simply missed by the public ears, we noticed that some frogs were commonly mistaken for other similar-sounding species. For example, the Common Eastern Froglet (Crinia signifera) was often mistaken with various species of Brood Frogs (Pseudophryne spp.) and was also often “heard” amongst background insect noise.

Overall, our test highlighted just how challenging it is to identify frog species from their calls, particularly when there’s more than one frog species calling, and without rigorous training. As crowdsourcing species identification can be great for processing large quantities of citizen science data and enabling the public to be engaged in more aspects of citizen science, we continue to look at opportunities to more rapidly identify frog calls submitted to the FrogID project, including public validation and machine learning. But for now, thankfully, the FrogID team continues to listen to and identify each and every recording submitted, ensuring the frogs submitted to FrogID are identified as accurately as possible.

Grace Gillard, Research Assistant, Herpetology, Australian Museum Research Institute

Dr Jodi Rowley, Curator, Amphibian and Reptile Conservation Biology, Australian Museum Research Institute; and, UNSW Sydney

More information:

Gillard, GL, Flemons, PKJ, Roslan, N, Woods, A and Rowley, JJL. 2024. What’s That Frog? Evaluating a Crowdsourced Approach to Species Identification From Audio Recordings. Citizen Science: Theory and Practice, 9(1): 19, pp. 1–14. DOI: https://doi.org/10.5334/cstp.688

Acknowledgements

We would like to offer our sincere thanks to the 233 citizen scientists who took part in the FrogID Audio Analysis trials. We would also like to mention that the results of our project in no way reflect either the capacity or willingness of citizen scientists to contribute to species identification – rather, they highlight some of the key challenges with species identification which need to be addressed before we can expand citizen scientist involvement in FrogID validation.