Evaluating Niche Models with Presence-Only Data is Uninformative for Many Applications

Abstract

Species distribution models and environmental niche models are used across evolution, ecology, conservation, and epidemiology to make critical decisions and study biological phenomena, often in cases where experimental approaches are intractable. Choices regarding optimal models, methods, and data are typically made based on discrimination accuracy: a model's ability to predict subsets of species occurrence data that were withheld during model construction. However, empirical applications of these models often involve making biological inferences based on continuous estimates of relative habitat suitability. Using a simulation approach, we demonstrate that discrimination accuracy is a poor indicator of a model's ability to estimate habitat suitability or species responses to environmental gradients. These results suggest that many empirical studies and decisions are based on model selection criteria that are unrelated to models' usefulness for their intended purpose. We argue that empirical modeling studies need to place significantly more emphasis on biological insight, and that the current approach of maximizing discrimination accuracy at the expense of other considerations is detrimental to both the empirical and methodological literature in this active field. Finally, we argue that future development of the field must include an increased emphasis on simulation, as methodological studies based on ability to predict withheld occurrence data may be largely uninformative about best practices for many modeling applications and will unduly penalize more biologically informative modeling approaches.