Evaluating Weights of Evidence method for habitat suitability modeling: a comparison with Maximum Entropy for a case of few presence records

Daniel Fernandes Mamede Teixeira Lopes, Andre Carvalho Silveira and Britaldo Silveira Soares Filho

Spatial modeling is widely applied to map habitat suitability and species distribution thus enabling exploratory analysis of a species’ geographical distribution. Suitability models predict the likelihood of species occurrence based on the niche theory translated into a set of environmental variables that indicate the suitability for presence or absence of a species (Guisan & Zimmermann 2000; Hirzel & Le Lay 2008). These tools are applied to map the ecological distribution of species based on a few presence records only and without records of absence (Pearson et al. 2007). Here we report the application of two modeling methods that produce suitability maps for Cotinga maculata (Cotingidae). The comparison of results indicate that the Maximum Entropy together with Weights of Evidence method presents satisfactory performance for cases with only few records of presence, based on the ROC analysis and Similarity comparison. The Weights of Evidence method available in Dinamica-EGO (Soares-Filho et al. 2013) performed on par with Maxent, thus offering an additional means to map habitat suitability.

Habitat suitability, Cotinga maculata, Maximum Entropy, Weights of Evidence, Maxent, Dinamica EGO, ROC analysis.

Spatially explicit models are computational representations of environmental systems (Wu & David 2002), such as the ecological niche of species (Guisan & Zimmermann 2000; Hirzel & Le Lay 2008). Based on the relationship between different environmental variables and records of species occurrence, it is possible to establish a spatial model for habitat suitability. In theory, such models enable identifying species’ potential spatial distribution (Franklin 2011), even in areas without sampling.

Cotinga maculata is a species of the Order of Passariformes, Family Cotingidae. It is endemic to small remnants of the Brazilian Atlantic Forest between south of Bahia and Rio de Janeiro states. The species occurs in lowland rainforest, up to 200 meters, primary vegetation or in advanced regrowth stage. The species visit small forest patches searching for fruits that compose its staple food. Considered rare by experts, this species is difficult to observe due to long immobile and quiet perching periods. The few occurrence records available concentrate in conservation units in the south Bahia state and north of Espírito Santo state (MMA, 2008). In this study, we used 18 records from Conservation International Brazil database. ==== Methods ==== === Maximum entropy === The maximum entropy method infers from incomplete knowledge a probability distribution function that includes all the constraints of a given dataset. It aims to maintain the maximum entropy of the data. The method assumes that constraints are obtained by overlaying selected spatial variables with species occurrence points (organized in a raster grid). The entropy represents a measure of “inner amount of choice”, and thus it is stochastically maximized to encompass the larger number of constraints. As a result, the method avoids any unknown assumption (Philips et al. 2006). The final product is a map that indicates the suitability for a species occurrence. === Weights of evidence === The Weights of Evidence method consists of a Bayesian approach that calculates the influence of explanatory variables on the spatial prediction of a response variable (Bonham-Carter 1994, Soares-Filho et al. 2004). Here this approach employs categorical and binary explanatory variables to assess how attractive or repulsive these variables are to a species occurrence (response variable). Continuous variables must be categorized and each variable category is evaluated in terms of its association/disassociation to the species occurrence. Calculation of Weights of Evidence is performed using the Dinamica EGO platform.

The explanatory variables selected initially were elevation, annual precipitation, maximum, minimum, and mean annual temperature, all obtained from WorldClim database (Hijmans et al. 2005). Raster grids of these variables were resampled to 1000×1000 meters. All variables and its intervals were evaluated for statistical significance. Same variables were used on both software: Maxent (for Maximum Entropy method) and Dinamica EGO (for Weights of Evidence method). === Suitability maps, similarity and disagreement === Figure 1 shows the suitability maps obtained from both methods. Areas with higher suitability match on both maps. The coastal area in the northeast of study area is the main region with high values of suitability. However, a substantial difference between the methods is the fact that the Maximum Entropy treats directly continuous variables, whereas the Weights of Evidence method categorizes all the continuous variables and treats each category as a binary secondary variable. Thus, the map produced by Weights of Evidence presents shades of gray that correspond to the categories created previously.

Figure 01: Suitability maps: (a) Weights of Evidence, (b) Maximum Entropy. Both the gradients were normalized to 0:100 range.

One way to explore the concordance between different methods of building a suitability surface is generate congruence and divergence maps. Thereby it is possible observe spatially areas predicted suitable by both methods, areas predicted suitable exclusively by one method, and also the concordance by ranges of suitability. Furthermore, both maps can also be evaluated by the Dinamica EGO reciprocal similarity functor, as ilustrated by figure 2. This functor calculates a two-way fuzzy similarity index between two maps (Calc Reciprocal Similarity Map).

Figure 02: Congruence and divergence maps comparing Maximum Entropy and Weights of Evidence methods + similarity index.

=== ROC performance evaluation === The Receiver Operating Characteristic (ROC) is a method to evaluate image similarity considering a prefixed binary pattern. ROC ponders true positive rate and false positive rate through incremental binary classifications (Mas et. al, 2013a). Despite the method has been applied to many study fields, ROC is commonly used in GIS to evaluate predictions provided by modeling versus observed data. Thus this work uses ROC metrics to evaluate the performance of each method individually, as well as to compare predictions between the both methods.

Figure 03: ROC curve and respective metrics.

The main ROC metrics used to evaluate the results were the area under curve (AUC) and the partial area under curve (pAUC). Figure 03 presents the standard ROC chart contrasting true positive rate and false positive rate. The red diagonal curve represents a low-skilled prediction, ie, a hypothetical model that predicts how much hits as much false alarms. The suitability maps are interpreted on ROC as predictions to be compared with the fixed diagonal. Each suitability map evaluated generates a new curve for the same chart. Any superposition of the prediction in analysis over the fixed diagonal is interpreted as a performance gain. The final gain offered by the prediction analyzed (relative to the suitability map) is summarized by the AUC measure. The same reading can be applied for a restricted range of hit rate or error rate, this partial measure is called pAUC, as illustrated in the figure 03. ==== Results and Discussions ==== The suitability maps generated by maximum entropy and weights of evidence were compared by sampling due to allow a feasible analyses in terms of computational effort. The comparison process more costly took around 15 hours to be concluded on a computer with 64 GB of memory RAM. There were executed 469 bootstraps, each one generating a curve based in binary classifications incremented by 10% (ie, 10 points to compose the ROC curve). The methods were compared considering all the area under curve (AUC), and also considering partial area under curve (pAUC).

The maximum entropy method has reached AUC = 0.92, while the weights of evidence method has reached AUC = 0.81. The comparison between the methods through multiple sampling has generated a p-value = 0,030. The comparison restricted to high hit indices, conform suggested by Pearson (2007), resulted in a p-value = 0,045. To the partial area under curve comparison were used 50 bootstraps in order of computational limitations. The p-value of 0,030 obtained by comparison between both methods points a statistical correlation between both projections. This fact indicates that weights of evidence method has enough skill for habitat suitability modeling, even in cases of small size samples. Being the maximum entropy a method considered high skilled for these cases.

Figure 04: ROC curve and p-value for AUC comparison between maximum entropy and weights of evidence.

Figure 05: ROC curve and p-value for pAUC comparison between maximum entropy and weights of evidence.

Besides direct comparison, the suitability maps of both methods were normalized to 0:100 range and then compared by ROC. In this case the p-value was 0.117. Meanwhile this result was obtained by a less exhaustive analysis in reason of computational limitations: 50 bootstraps and 10% of increment.

Results show that performance of weights of evidence is enough close to maximum entropy one. Once the maximum entropy is recognized as appropriated approach to model habitat suitability in low sampling cases (Pearson et al., 2007), weights of evidence emerges as an alternative for such studies. Considering the availability of weights of evidence method in the spatially explicit environment Dinamica EGO, it turns up an alternative to the commercial software Maxent, the main framework used to apply maximum entropy method. Still it is important to note that both modeling methods could be improved by more sophisticated configurations (heuristic searching, knowledge-driven adjustments, etc.). In this work both methods were compared assuming only calibration direct by sampling. ==== References ==== - Bonham-Carter, G. 1994. Geographic information systems for geoscientists: modelling with GIS. Pergamon, Oxford, UK. - Franklin, J. Mapping Species Distribution: Spatial Inference and Prediction. 2011. Cambridge. Cambridge, UK. - Guisan, A., Zimmermann, N. E. Predictive habitat distribution models in ecology. Ecological Modelling. 2000 (135): 147-186 p. - Hijmans, R. J., Cameron, S. E., Parra, J. L., Jones, P. G. and Jarvis, A. (2005), Very high resolution interpolated climate surfaces for global land areas. Int. J. Climatol., 25: 1965–1978. - Hirzel, A. H., Le Lay, G. Habitat Suitability modelling and niche theory. Journal of Applied Ecology 2008 (45): 1372-1381. - Jianguo Wu, John L David, A spatially explicit hierarchical approach to modeling complex ecological systems: theory and applications, Ecological Modelling, Volume 153, Issues 1–2, 15 July 2002, Pages 7-26, ISSN 0304-3800. - Mas, J.F.; Soares-Filho, B. S.; Pontius Jr, R. G.; Gutiérrez, M. F.; Rodrigues, H. O. A Suite of Tools for ROC Analysis of Spatial Models. ISPRS International Journal of Geo-Information. 2013a; 2(3):869-887. - Mas, J.F. Tools for ROC analysis of spatial models Installation instructions and application examples. Centro de Investigaciones en Geografía Ambiental Universidad Nacional Autónoma de México (UNAM), 2013b. - Ministério do Meio Ambiente. Livro Vermelho da Fauna Brasileira Ameaçada de Extinção. V(2) Brasília. 2008. - Pearson, R. G., Raxworthy, C. J., Nakamura, M., Townsend P. A. Predicting species distributions from small numbers of occurrence records: a test case using cryptic geckos in Madagascar. Journal of Biogeography. Blackwell Publishing Ltd, 2007. - Soares-Filho B.S., Alencar A., Nepstad D. , Cerqueira G., Vera Diaz M., Rivero S., Solorzano L., Voll E. Simulating the response of land-cover changes to road paving and governance along a major Amazon highway: the Santarém-Cuiabá corridor. Global Change Biology 10, 2004. - Soares-Filho, B.S.; Rodrigues, H.O.; Follador, M. A hybrid analytical-heuristic method for calibrating land-use change models. Environ. Model. Soft. 2013, 43, 80–87. ==== Resources ==== The links below are relative to models and available datasets used in this article: Available inputs, outputs and models