Evaluating Weights of Evidence method for habitat suitability modeling: a comparison with Maximum Entropy for a case of few presence records

Daniel Fernandes Mamede Teixeira Lopes, Andre Carvalho Silveira and Britaldo Silveira Soares Filho

Abstract

Spatial modeling is widely applied to map habitat suitability and species distribution thus enabling exploratory analysis of a species’ geographical distribution. Suitability models predict the likelihood of species occurrence based on the niche theory translated into a set of environmental variables that indicate the suitability for presence or absence of a species (Guisan & Zimmermann 2000; Hirzel & Le Lay 2008). These tools are applied to map the ecological distribution of species based on a few presence records only and without records of absence (Pearson et al. 2007). Here we report the application of two modeling methods that produce suitability maps for Cotinga maculata (Cotingidae). The comparison of results indicate that the Maximum Entropy together with the Weights of Evidence method presents satisfactory performance for cases with only few records of presence, based on the ROC analysis and Similarity Fuzzy comparison. The Weights of Evidence method available in Dinamica-EGO (Soares-Filho et al. 2013) performed on par with Maxent, thus offering an additional means to map habitat suitability.

Keywords

Habitat suitability, Cotinga maculata, Maximum Entropy, Weights of Evidence, Maxent, Dinamica EGO, ROC analysis.

Introduction

Spatially explicit models are computational representations of environmental systems (Wu & David 2002), such as the ecological niche of species (Guisan & Zimmermann 2000; Hirzel & Le Lay 2008). Based on the relationship between different environmental variables and records of species occurrence, it is possible to establish a spatial model for habitat suitability. In theory, such models enable identifying species’ potential spatial distribution (Franklin 2011), even in areas without sampling.

Cotinga maculata is a species of the Order of Passariformes, Family Cotingidae. It is endemic to small remnants of the Brazilian Atlantic Forest between south of Bahia and Rio de Janeiro states. The species occurs in lowland rainforest, up to 200 meters, primary vegetation or in advanced regrowth stage. The species visit small forest patches searching for fruits that compose its staple food. Considered rare by experts, this species is difficult to observe due to long immobile and quiet perching periods. The few occurrence records available concentrate in conservation units in the south Bahia state and north of Espírito Santo state (MMA, 2008). In this study, we used 18 records from Conservation International Brazil database.

Methods

Maximum entropy

The maximum entropy method infers from incomplete knowledge a probability distribution function that includes all the constraints of a given dataset. It aims to maintain the maximum entropy of the data. The method assumes that constraints are obtained by overlaying selected spatial variables with species occurrence points (organized in a raster grid). The entropy represents a measure of “inner amount of choice”, and thus it is stochastically maximized to encompass the larger number of constraints. As a result, the method avoids any unknown assumption (Philips et al. 2006). The final product is a map that indicates the suitability for a species occurrence.

Weights of evidence

The Weights of Evidence method consists of a Bayesian approach that calculates the influence of explanatory variables on the spatial prediction of a response variable (Bonham-Carter 1994, Soares-Filho et al. 2004). This approach employs categorical and binary explanatory variables to assess how attractive or repulsive these variables are to a species occurrence (response variable). Continuous variables must be categorized and each variable category is evaluated in terms of its association/disassociation to the species occurrence. Calculation of Weights of Evidence is performed using the Dinamica EGO platform.

The explanatory variables selected initially were elevation, annual precipitation, maximum, minimum, and mean annual temperature, all obtained from WorldClim database (Hijmans et al. 2005). Raster grids of these variables were resampled to 1000×1000 meters. All variables and its intervals were evaluated for statistical significance. Same variables were used as input for both software: Maxent (for Maximum Entropy method) and Dinamica EGO (for Weights of Evidence method).

Suitability maps, similarity and disagreement

Figure 1 shows the suitability maps obtained from both methods. Areas with higher suitability match on both maps. The coastal area in the northeast of study area is the main region with high values of suitability. However, a substantial difference between the methods is the fact that the Maximum Entropy treats directly continuous variables, whereas the Weights of Evidence method categorizes continuous variables and treats each category as a binary secondary variable. Thus, the map produced by Weights of Evidence presents shades of gray that correspond to the ranges created in the categorization process.

Figure 01: Suitability maps: (a) Weights of Evidence, (b) Maximum Entropy. Both the gradients were normalized to 0:100 range.

Similarity and disagreement maps consist of one way to explore the matching between different methods for calculating a suitability map. Thereby it is possible to observe areas predicted by both methods, areas predicted exclusively by one method, and agreement of ranges of suitability. Furthermore, both maps can also be evaluated by using reciprocal similarity metric, as illustrated in figure 2. This method calculates a two-way fuzzy similarity index between a pair of maps (Calc Reciprocal Similarity Map).

Figure 02: Similarity and disagreement maps comparing Maximum Entropy and Weights of Evidence methods + similarity index.

ROC performance evaluation

The Receiver Operating Characteristic (ROC) evaluates map similarity considering a reference binary pattern. ROC compare the amount of true positive and false positive cells through an incremental binary classification (Mas et al. 2013a). ROC is commonly used in GIS to evaluate spatial predictions versus observed data. In this work, wee used ROC metrics to evaluate the performance of each method individually, as well as to compare predictions between the two methods.

Figure 03: ROC curve and respective metrics.

The main ROC metrics used to evaluate the results are the area under curve (AUC) and the partial area under curve (pAUC). Figure 03 presents the standard ROC graph of true positive and false positive. The red diagonal curve represents a hypothetical model that predicts the same number of hits and false alarms. The suitability maps are interpreted on the ROC as prediction curves compared with the fixed diagonal. Each suitability evaluation generates a new curve on the graph. Curves over the fixed diagonal represent models that perform better than a random model. The final gain (relative to the suitability map) is summarized by the AUC measure. An equivalent metric can be applied to measure hit rate or error rate, this partial measure is called pAUC, as illustrated in the figure 03.

Results and Discussions

The suitability maps generated by Maximum Entropy and Weights of Evidence were compared using sampling. The comparison procedure took about 15 hours to be concluded on a computer with 64 GB of memory RAM and 32 processors. The procedure executed 469 bootstraps, each one generating a curve based in binary classifications incremented by 10% (i.e., 10 points to compose the ROC curve). The methods were compared considering all the area under curve (AUC), and considering partial area under curve (pAUC).

The Maximum Entropy method AUC amounted to 0.92, while the Weights of Evidence method reached 0.81. The comparison between the methods through multiple sampling has generated a p-value of 0.030. The comparison constrained to high hit indices, as suggested by Pearson (2007), resulted in a p-value = 0.045. The comparison of partial area under curve used 50 bootstraps due to computational time required. The p-value of 0.030 obtained from comparing the two methods indicates a statistical correlation between both predictions. This result shows that Weights of Evidence method performs well for modeling habitat suitability, even in cases of small size samples. Such performance is compatible with the one of the Maximum Entropy, a method considered highly suitable for these cases.

Figure 04: ROC curve and p-value for AUC comparison between Maximum Entropy and Weights of Evidence.

Figure 05: ROC curve and p-value for pAUC comparison between Maximum Entropy and Weights of Evidence.

In addition, the suitability maps of both methods were normalized to 0:100 range and then compared using ROC. In this case, the p-value was 0.117. This result was obtained using 50 bootstraps and 10% of increment.

In sum, our results show that Weights of Evidence performed on par with the Maximum Entropy. It is important to note that both modeling methods could be further improved by fine-tuning (heuristic searching, knowledge-driven adjustments in Maxent), more finer ranges, or applying genetic algorithm in the case of Weights of Evidence.