Evaluation of ForestPA for VHR RS image classification using spectral and
superpixel-guided morphological profiles
Alim Samat a,b, Sicong Liu c, Claudio Persello d, Erzhu Li e, Zelang Miaofand Jilili Abuduwailia,b aState Key Laboratory of Desert and Oasis Ecology, Xinjiang Institute of Ecology and Geography, CAS, Urumqi, China;bResearch Center for Ecology and Environment of Central Asia, CAS, Urumqi, China;cCollege of Surveying and Geoinformatics, Tongji University, Shanghai, China;dDepartment of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands;eDepartment of Geographical Information Science, Jiangsu Normal University, Xuzhou, China; fSchool of Geosciences & Info-Physics, Central South University, China
ABSTRACT
In very high resolution (VHR) remote sensing (RS) classification tasks, conventional pixel-based contextual information extraction methods such as morphological profiles (MPs), extended MPs (EMPs) and MPs with partial reconstruction (MPPR) with limited numbers, sizes and shapes of structural elements (SEs) cannot perfectly match all sizes and shapes of the objects in an image. To overcome such limitation, we introduce novel spatial feature extractors, namely, the superpixel-guided morphological profiles (SPMPs), where the super-pixels are used as SEs in opening by reconstruction and closing by reconstruction operations. Moreover, to avoid possible side effects from unusual maximum and minimum values within superpixels, the mean pixel value of superpixels is adopted (SPMPsM). Additionally, new decision forest based on penalizing the attributes in previous trees, the ForestPA is intro-duced and evaluated through a comparative investigation on three VHR multi-/hyperspectral RS image classification tasks. Support vector machine and benchmark ensemble classifiers, including bagging, AdaBoost, MultiBoost, ExtraTrees, Random Forest and Rotation Forest, are adopted. The experimental results confirm the effectiveness and superior performances of the proposed SPMPs and SPMPsM relative to those of the MPs and MPPR. Moreover, ForestPA outperforms only bagging and is not suitable for learning from large numbers of samples with high dimensionality from the computational efficiency and classification accuracy perspective. ARTICLE HISTORY Received 17 April 2018 Revised 10 December 2018 Accepted 10 December 2018 KEYWORDS ForestPA; MPs; MPPR; superpixel; superpixel-guided morphological profiles; VHR images; image classification
Introduction
In recent years, airborne and spaceborne multi-/hyperspectral remote sensors have advanced in terms of spectral and spatial resolution, which makes the analysis of small spatial structures possible with unprecedented spatial details. Hyperspectral sensors can provide detailed spectral information with hundreds of spectral wavelengths and increase the ability to accurately discriminate the materials of interest. However, the high dimensionality of hyper-spectral images may lead to the Hughes phenom-enon, which is related to the curse of dimensionality in classification tasks (Camps-Valls, Tuia, Bruzzone, & Benediktsson,2014). Additionally, while high reso-lution (HR) and very high resoreso-lution (VHR) data solve the problem of “seeing” structural objects and elements, they do not help in focusing on the extrac-tion procedure (Gamba, Dell’Acqua, Stasolla, Trianni, & Lisini, 2011). Two major challenges, namely, the necessity of spectral dimensionality reduction and the need for specific spectral-spatial classifiers, have been identified by the HR and VHR multi-/hyperspectral
remote sensing (RS) image processing community (Fauvel, Tarabalka, Benediktsson, Chanussot, & Tilton,2013; Plaza et al., 2009).
Among the efforts of addressing the above-mentioned challenges, the value of adding contextual information for revealing relationships and depen-dencies among image objects has become one of the most important challenges for the successful analysis of HR and VHR RS images. Principally motivated by the ability of texture features to provide a quantitative description of image properties, including smooth-ness, roughsmooth-ness, symmetry and regularity, many tex-ture extraction methods, such as statistical (gray-level co-occurrence matrix, GLCM), geometrical and structural approaches; Markov random field (MRF)-and conditional r(MRF)-andom field (CRF)-model-based approaches; and signal processing (Gabor filter) approaches, have been examined for urban land-cover mapping (Kasetkasem, Arora, & Varshney,
2005; Ma et al., 2017; Rajadell, García-Sevilla, & Pla,
2013; Zhang, Wang, Gong, & Shi, 2003). However, statistical textures (e.g. GLCM) are typically
CONTACTAlim Samat alim_smt@ms.xjb.ac.cn State Key Laboratory of Desert and Oasis Ecology, Xinjiang Institute of Ecology and Geography, CAS, Urumqi, China
https://doi.org/10.1080/22797254.2019.1565418
© 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
computed in a moving window with a specified size along certain direction, thereby imposing a crisp and not common for each pixel in the image. Additionally, pixel-based graph models, including MRF and CRF, often suffer from “salt and pepper” noise and cannot capture contextual information about objects. Instead of pixel-by-pixel classification, object-based image analysis (OBIA) and geographic OBIA (GEOBIA) techniques have been widely used to divide images into homogeneous segments and assign semantic labels according to the properties of image segments in HR and VHR image classification tasks. However, due to the complexity and heteroge-neity of HR and VHR images, the segmentation pro-cess is challenging because this propro-cess typically relies on parameters that are highly dependent on the image at hand and the specific tasks (Blaschke et al.,
2014; Costa, Foody, & Boyd, 2017; Gu et al., 2017; Ma, Cheng, Li, Liu, & Ma, 2015). For instance, objects and geographic objects often have their own optimal segmentation scales, even within the same class (Zhao, Du, Wang, & Emery, 2017). Accordingly, multiresolution segmentation (MRS) methods have been proposed to segment HR and VHR images at multiple scales. Specifically, the MRS methods aim to partition HR and VHR images into image objects by minimizing the heterogeneity within objects and maximizing the differences across objects. Although MRS-based approaches can pro-duce multi-scale segments by employing various parameters, these approaches still require prior knowledge about the inherent scales of each geo-graphic class, which depend strongly on the spatial and spectral features but also the semantic class.
To add a new family member of contextual informa-tion extraction, mathematical morphology-based approaches such as morphological profiles (MPs), extended morphological profiles (EMPs), attribute pro-files (APs) and morphological propro-files with partial reconstruction (MPPR) have demonstrated the benefits of using geometrical information from HR and VHR images in many urban applications (Benediktsson, Palmason, & Sveinsson, 2005; Dalla Mura, Villa, Benediktsson, Chanussot, & Bruzzone, 2011; Liao et al., 2017). However, being connected filters, these approaches have limitations such as the following: (1) structural elements (SEs) with user-specified shape and size are highly constrained when modeling concepts of the characteristics of size, shape and homogeneity infor-mation; (2) attribute filters (AFs) based on geodesic reconciling still suffer from the problem of leakage, which is also referred to as an over-reconstruction problem; and (3) most importantly, a sequence of SEs with predefined sizes and shapes cannot perfectly match all sizes and shapes of objects in a certain image, speci-fically at a single SE for the entire image at each opera-tion case. To this end, we introduce superpixel guide
morphological profiles (SPMPs). All superpixels are used as SEs in opening by reconstruction (OBR) and closing by reconstruction (CBR) operations. To avoid possible side effects from unusual maximum and mini-mum values within superpixels, the mean pixel value of superpixels, namely, the SPMPsM, is further adopted.
For any specific purpose of image classification, the processing framework should be designed to operate with a robust classifier and suitable input data with high-quality features that carry statistical discrimination descriptions of corresponding classes. Numerous studies have been conducted by the HR/ VHR multi-/hyperspectral image processing commu-nity to develop accurate spectral-spatial classifiers. In most cases, spatial contextual information has been incorporated into advanced machine learning (ML) methods, including supervised and semi-supervised, parametric and nonparametric classifiers (Chan & Paelinckx, 2008; Fauvel, Benediktsson, Chanussot, & Sveinsson,2008; Kavzoglu, Colkesen, & Yomralioglu,
2015Li, Bioucas-Dias, & Plaza,2013; Samat, Du, Liu, Li, & Cheng,2014; Samat et al.,2016). Among those, decision tree (DT)-based ensemble classifiers such as bagging, boosting, random subspace, random forest (RaF), rotation forest (RoF) and decision-tree-based gradient boosting (DTGB) have attracted increasing interest due to their higher accuracy and robustness to noise than that of single classifiers (Belgiu & Drăguţ, 2016; Du, Samat, Waske, Liu, & Li, 2015; Godinho, Guiomar, & Gil, 2016; Xia, Du, He, & Chanussot,2014).
Recently, a new decision forest algorithm, which is called ForestPA and is constructed by penalizing attri-butes that are used in previous trees, was proposed by Adnan & Islam (2017). According to their experiments on 20 well-known UCI ML datasets, ForestPA is effective in generating highly accurate, more balanced and more evenly suitable decision forests than bagging, random subspace, RaF or ExtraTrees in terms of classification accuracy. ForestPA is found to be analogous to other contending algorithms in terms of complexity. However, the performance of ForestPA in the classification of HR and VHR RS images is unclear, specifically compared with that of bagging, random subspace, RaF, ExtraTrees and RoF. Thus, it is of interest to comparatively investi-gate the performance of ForestPA in the classification of HR and VHR multi-/hyperspectral images over urban areas, specifically using MPs, EMPs, MPPR and the proposed SPMPs and SPMPSM features.
The main contributions of this paper include: (1) SPMPs and SPMPSM are proposed and comprehen-sively applied for spatial features extraction in VHR RS images; (2) some effective model parameters are recom-mended based on different experimental scenarios; (3) ForestPA is introduced and comparatively investigated in the classification of HR and VHR multi-/hyperspectral images over complex urban areas.
Methods
ForestPA
ForestPA is constructed by penalizing attributes that are used in previous trees in a decision forest (Adnan & Islam,2017). More specifically, considering that an attribute tested at a lower level can influence more logic rules than an attribute tested at a higher level, ForestPA imposes weights in a systematic way such that an attribute that is tested at a lower level receives a lower weight/higher penalty than an attribute tested at a higher level. Additionally, ForestPA randomly selects the weight of an attribute from the weight range that is allocated for the attribute’s level. The weight range is defined as follows:
WRλ ¼ ½0:00; eλ 1 ; if λ ¼ 1 ½eðλ1Þ1þ ρ; eλ1 ; if λ > 1 ( (1)
where λ represent the attribute’s level and λ¼ 1 means the root node, respectively, and ρ is used to ensure that the weight ranges for various levels are non-overlapping; it is recommended to setρ to 0.001 (Adnan & Islam,2017).
During the construction process, ForestPA imposes weights only on those attributes that appear in the latest tree; the weights of the attributes that do not appear in the latest tree (and thus the weights of the attributes that are obtained from any previous tree) are automatically preserved. By retaining previous weights, ForestPA avoids switching among similar trees. However, this strategy may also have the negative impact of removing the attributes that receive relatively small weights in any subsequent trees. To address this issue, ForestPA adopts the gradual weight increment value of an attribute, which is calculated as follows (Adnan & Islam,2017):
σi¼
1 ωi
ðη þ 1Þ λ (2)
whereωiis the weight of the ith attribute Aifrom the previous tree andη is the tree height, which is equal to the highest level of the tree. Accordingly, ForestPA can be built by following four main steps as described inTable 1.
Superpixel-guided morphological profiles
Mathematical morphological operators act on the values of the pixels and consider the pixels’ neighbor-hoods, which are determined by SE with predefined size and shape, based on the two basic operators of dilation and erosion. The erosion of an image by SE at any location (x, y) is defined as the minimum value of all the pixels in its SE-defined neighborhood. In contrast, dilation returns the maximum value of the image in the window that is outlined by SE. In gray-scale morphological reconstruction, morphological OBR of gray-scale images can be obtained by eroding the input image (f) and using it as a marker (g), while CBR can be obtained by complementing the image, obtaining the opening by reconstruction and comple-menting the result of the procedure (Benediktsson et al., 2005; Dalla Mura et al., 2011; Liao et al.,
2017, Rafael & Richard,2010).
As stated earlier, SEs with user-specified shape and size are highly constrained when modeling objects with different sizes, shapes and homogeneity information. Additionally, a sequence of SEs with predefined sizes and shapes cannot perfectly match all the sizes and shapes of objects in an image, specifically with a single SE for the entire image for each operation case. Moreover, according to the definitions of OBR and CBR, if SEs are replaced with perceptually meaningful atomic regions, namely, superpixels, then we can use many SEs with various shapes and sizes to better match the sizes and shapes of objects in an image. More importantly, the operation provides sufficiently many SEs for the entire image for each operation case. In
Table 1.Algorithmic steps of ForestPA.
Inputs: Labeled training set X, number of treesT, set of weights W, set of weight increment values S. Training:
for (each Ai2 X) do
wi¼ 1, σi¼ 0,W W [ wf g,i S S [ σf g.i
end
for (t = 1 to T) do
Step 1: bootstrap-sampling-based generation of new setXt;
Step 2: build a conventionalDTtusingXt;
Step 3: update weights and gradual weight increment values of the attributes in the latest tree for (each Aj2 DTt)do
calculatewjandσjusing Equation (1) and Equation (2), respectively;
update weightsW W [ wj
and weight increment valuesS S [ σj
. end
Step 4: update weights of the applicable attributes that are not present in the latest tree for (each Aj: Aj‚DTtand wj<; 1) do
wj¼ wjþ σj,W W [ wj ; end ForestPA ForestPA[ DTf tg end Output: ForestPA
other words, we can obtain spatial features of similar discrimination capability with lower dimensionality.
Again, according to the definitions of OBR and CBR, the superpixel-guided OBR (SPOBR) can be obtained by eroding the input image using selected superpixels J0 ¼ Jf 0i; :::; J0Sg, where S represents the number of superpixels, and the result can be used as a marker in geodesic reconstruction by a dilation phase:
OSPRðf Þ ¼ RDf½ðf ð9J0i 2 J0ÞÞ (3) Similarly, we can define
CSPRðf Þ ¼ REf½ðf ð9J0i2 J0ÞÞ (4) for the superpixel-guided CBR (SPCBR), which is obtained by complementing the image, obtaining the SPOBR using 9J0i2 J0 as SEs, and complement-ing the result:
CRSPðf Þ ¼ RDCf ðfC ð9J0i 2 J0ÞÞ
(5) Finally, the SPMPs of an image f can be defined as
SPMPsðf Þ ¼ O SPR ðf Þ; f ; CSPR ðf Þ (6) To avoid the possible side effects from unusual mini-mum or maximini-mum pixel values within superpixels, the SPMPsM can be obtained by adding mean pixel values:
SPMPsMðf Þ ¼ SPMPsðf Þ; SP½ meanðf Þ (7)
Although the use of MPs can help in creating an image feature set that carries more discriminative information, redundancy is still evident in the feature set, particularly for hyperspectral images. Therefore, feature extraction can be used to find the most important features first. Then, morphological opera-tors can be applied. After PCA has been applied to the original feature set, the extended SPMPs and SPMPsM can be obtained by applying the basic prin-ciples of SPMPs and SPMPsM that are described above, for the first few (typically three) features.
Datasets and experimental configuration
Datasets
A ROSIS Pavia University hyperspectral image was acquired with a ROSIS optical sensor, which provides 115 bands with a spectral range of 0.43–0.86 μm. The geometric resolution is 1.3 m. The image, which is shown in Figure 1(a), was captured over the Engineering School, University of Pavia, Pavia, Italy, and has pixel dimensions of 610 × 340 with 103 spectral channels (several original bands were noisy and were discarded immediately after the data acqui-sition). The validation data refer to nine land-cover classes (as shown inFigure 1andTable 2).
The second hyperspectral image was acquired at a spatial resolution of 2.5 m by the NSF-funded Center for Airborne Laser Mapping over the University of Houston campus and the neighboring urban area on 23 June 2012. The image has 349 × 1905 pixels with 144 spectral bands in the spectral range between 380 and 1050 nm. The 15 classes of interest selected by the Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society (GRSS) are reported for both the training and validation sets (Debes et al.,
2014). In this work, a subset image with pixel dimensions of 340 × 1350 is obtained by removing
Figure 1.ROSIS Pavia University dataset: (a) color composite of the scene; (b) training set; (c) test set. Table 2.Sample details for the ROSIS Pavia University
hyper-spectral image.
No. Class Test Training Total
1 Asphalt 6631 548 7179 2 Meadows 18,649 540 392 19,189 3 Gravel 2099 2491 4 Trees 3064 524 3588 5 Metal 1345 265 1610 6 Bare soil 5029 532 5561 7 Bitumen 1330 375 1705 8 Bricks 3682 514 4196 9 Shadows 947 231 1178
the blank areas and the cloud-covered area in the right side of the image, as shown in Figure 2.
The last set of test data was obtained from a multispectral VHR image (Figure 3) that was col-lected over the city of Zurich (Switzerland) by the QuickBird satellite in August 2002, which is freely available at https://sites.google.com/site/michelevolpire
search/data/zurichdataset. Originally, the image had
a pixel size of 1295 × 1364, was composed by 4 channels (NIR-R-G-B) and was pansharpened to a PAN resolution of approximately 0.62 cm GSD. A total of 7 urban and peri-urban classes were manu-ally annotated: roads, buildings, trees, grass, bare soil, railways and swimming pools. The cumulative num-ber of class samples is highly unbalanced, which reflects real-world situations (see Table 4). In the experiments, we evaluate the generalization accuracy in a leave-one-out setting, that is, by training on a small portion of samples (smaller than 250) and evaluating the classifier on the remaining left-out samples.
Experimental configuration
To generate MPs and MPPR, we applied a disk-shaped SE with n = 10 openings and closings by conventional and partial reconstructions, which range from one to ten with a step-size increment of one as recommended by Benediktsson et al. (2005), Dalla Mura et al. (2011) and Liao et al. (2017). Therefore, we obtained datasets with dimensions equal to 70 (i.e. = 10 + 3 × 10 × 2), 67 (i.e.7 + 3 × 10 × 2) and 84 (i.e.4 + 4 × 10 × 2) for ROSIS, GRSS-DFC2013 and Zurich QuickBird datasets, respectively. Specifically, only the first three PCA-transformed features were used for extracting MPs and MPPR for ROSIS and GRSS-DFC2013, and the raw four spectral bands were used for Zurich QuickBird.
To generate superpixels, we apply the simple linear iterative clustering algorithm, which adapts k-means clustering to generate superpixels in an over-segmented manner (Achanta et al.,2012). Notably, the algorithm carries the advanced properties of striking
Figure 2.GRSS-DFC2013 dataset: (a) color composite of the scene; (b) training set; (c) test set.
Figure 3.Color composite (a) of Zurich QuickBird with the corresponding ground truth (b).
Table 3.Sample details for the GRSS-DFC2013 hyperspectral image.
No. Class Training Test No. Class Training Test
1 Healthy grass 198 1053 9 Road 193 1053
2 Stressed grass 190 1064 10 Highway 191 1036
3 Synthetic grass 192 505 11 Railway 181 1050
4 Trees 188 1056 12 Parking lot 1 192 1041
5 Soil 186 1056 13 Parking lot 2 184 285
6 Water 182 143 14 Tennis court 181 247
7 Residential 196 1072 15 Running track 187 473
simplicity, high speed and memory efficiency, free availability and state-of-the-art processing. Moreover, the number of superpixels can range from 100 to 20,000 with step sizes of 10, 20, 30, 40, 50, 100, 200, 250, 300, 500, 750 and 1000 for evaluating the effectiveness and configuration of optimal sets. In all cases, SPMPs are generated at the same dimensionality as MPs and MPPR for fair comparison. All the datasets used in experiment are normalized into [−1,1].
As classifiers, we considered bagging (Breiman,
1996), AdaBoost (Rätsch, Onoda, & Müller,2001) and MultiBoostAB (Webb, 2000), ensembles of C4.5, ExtraTrees (Geurts, Ernst, & Wehenkel, 2006; Samat et al., 2018), RaF (Breiman, 2001), RoF (Rodriguez, Kuncheva, & Alonso, 2006), ForestPA (Adnan & Islam, 2017) and the SVM (Cortes & Vapnik,1995). The parameters of the SVM are tuned with a cross-validation technique. The numbers of DTs in bagging, RaF, RoF, ExtraTrees and ForestPA and the numbers of iterations in AdaBoost and MultiBoostAB are set to 100 by default. The overall accuracy (OA), kappa statistic and CPU running time are used to evaluate the classi-fication performances of these methods using EMP, EMPPR, ESPMP and ESPMPsM features. Finally, ForestPA is evaluated in terms of classification accu-racy, computational cost and robustness to the numbers of DTs in the ensemble.
Result analysis and discussion
Evaluation of SPMPs
To evaluate the performance of contextual information extraction capability of SPMPs using visual interpreta-tion, we first present the OBR, CBR, OBR and CBR with
partial reconstruction (OBPR and CBPR, respectively) and superpixel-guided OBR and CBR (SPOBR and SPCBR, respectively) at various scales from the second component of ROSIS Pavia University image in the lower-left corner inFigure 4.
According to the graphs in the left part ofFigure 4, the OBR and OBPPR images become brighter as the size of SE increases, while the boundaries between the different land-cover types become increasingly thin, and some parts ultimately merge with other land-cover classes. Specifically, many large objects that should remain are filtered, while many small objects that should disappear remain at a high scale after OBPR and CBPR at a specific scale of the area attribute; see the graphs in line 2. In contrast, the boundaries between the different land-cover types remain exactly as in the ori-ginal; only the areas within the corresponding bound-aries are filtered by the proposed SPOBR. Moreover, number of superpixels (scale parameter) affects only the brightness or darkness of the areas within the corre-sponding boundaries. Similar results can be found from the graphs that are presented in the right part of
Figure 4.
To further evaluate the effectiveness of the proposed feature extraction methods,Figure 5presents the over-all accuracy (OA) values that are obtained using various sets of SEs (from 1 to 10 with a step size of 1) in MPs and MPPR and various numbers of superpixels (from 1000 to 2000 with a step size of 100 for the ROSIS Pavia University and GRSS-DFC2013 images and from 1000 to 10,000 with a step size of 1000 for the Zurich QuickBird image) in SPMPs and SPMPsM to extract spatial features.
According to the graphs in Figure 5, the proposed SPMPs and SPMPsM are effective. Moreover, SPMPsM is superior to SPMPs in terms of classification accuracy. Specifically, the best improvements are achieved by AdaBoost using MPs for the ROSIS Pavia University data, by ExtraTrees using SPMPsM for GRSS-DFC 2013, and by AdaBoost using SPMPsM for the Zurich QuickBird data.
The number of SEs, shape and step-size are the main elements that control the multiscale spatial
Figure 4.Examples of OBR, OBPR and SPOBR at various scales, computed from the second principal component of the ROSIS Pavia University data in the lower-left corner.
Table 4.Sample details for Zurich QuickBird dataset.
No. Label Total Train Test
1 Roads 5070 170 4900 2 Buildings 311,833 233 311,600 3 Trees 251,222 222 251,000 4 Grass 120,532 232 120,300 5 Bare soil 16,043 143 15,900 6 Rails 79,153 153 79,000 7 Pools 72,429 229 72,200
feature extraction capabilities of conventional MPs and MPPR. Similarly, and according to the definition of the proposed SPMPs, the number of superpixels S and the scale step-size are the two critical para-meters for SPMPs and SPMPsM. Figure 6 presents the results for the configuration of these parameters using RaF with 200 DTs for the considered datasets. SPMPs are generated at the same dimensionality with MPs and MPPR for fair comparison. In this sense, there are 15 ranges with a step size of 100 but only 2 ranges with a step size of 750 from 100 to 15,000 (numbers of superpixels) for the ROSIS Pavia University and GRSS-DFC2013 images, which is why the lines in Figure 6 are composed of different numbers of dots.
Based on the results shown in Figure 6, both the number of superpixels and the scale step-size can affect the performance of SPMPs and SPMPsM in terms of classification accuracy. The use of too many or too few superpixels cannot result in positive improvements in OA values. For instance, first positive and then negative
improvements in OA values are observed inFigure 6(a– c, e) because at a specific size of a given image, the use of too few superpixels results in large individual superpixels that cover multiple different ground objects, while the use of too many superpixels results in small individual super-pixels that cannot provide valuable contextual informa-tion. These results can also explain the result that SPMPs can outperform SPMPsM in scenarios with too few superpixels, in which the mean pixel values from a superpixel cover multiple ground objects, which can corrupt the spatial discrimination capability of SPMPs
(seeFigure 6(a)). In contrast, SPMPsM can outperform
SPMPs in cases of small individual superpixels, which cannot provide valuable contextual information, whereas both spectral and spatial discrimination capabilities can be improved by considering mean pixel values (seeFigure 6(c–f)). The effects of scale step size are more complex and mixed with effects from the number of superpixels since a larger scale step size ultimately leads to a larger number of superpixels. Additionally, the effects of the scale step size are different on various test images with
Figure 6.OA values versus number of superpixels and scale step in SPMPs and SPMPsM of ROSIS Pavia University (a, d), GRSS-DFC2013 (b, e) and Zurich QuickBird (c, f) images.
Figure 5.OA values from the considered classifiers using various features from ROSIS Pavia University (a), GRSS-DFC2013 (b) and Zurich QuickBird (c) images.
various object and image sizes. According to the results that are shown inFigure 6, optimal ranges for the number of superpixels are 100–1000, 600–1600 and 1000–10,000 for the ROSIS Pavia University, GRSS-DFC2013 and Zurich QuickBird (c, f) images, respectively. Accordingly, the optimal ranges for the scale step size are 10–100, 10–100 and 10–1000 for the ROSIS Pavia University, GRSS-DFC2013 and Zurich QuickBird (c, f) images, respectively.
Evaluation of ForestPA
In this part, we first evaluate the classification perfor-mance of ForestPA as a function of its critical para-meter: the number of DTs in the ensemble. Benchmark and widely accepted EL classifiers including bagging, AdaBoost, MultiBoostAB, ExtraTrees, RaF and RoF are considered with the recommended parameters, except that the numbers of iterations for AdaBoost and MultiBoostAB, and numbers of DTs in other ensembles are set to the same value as the number of DTs in ForestPA for fair evaluation. Figure 7 presents the results of all considered approaches using different fea-tures from all three test images, where the y-axis repre-sents the OA values and the x-axis shows the number of DTs in the ensemble or the number of iterations.
According to the graphs inFigure 7, while the highest OA values are achieved by either RoF (black curves with five-pointed stars), AdaBoost (red curves with rectangles) or MultiBoostAB (pink curves with upward-pointing triangles) in most cases, better only than bagging (blue curves with diamonds) OA values reached by ForestPA (cyan curves with downward-pointing triangles), parti-cularly using MP, MPPR, SPMP and SPMPsM features with high dimensionality. ForestPA reached better OA values than only RaF using SPMP features from the ROSIS Pavia University image, using the first seven prin-ciple components from the GRSS-DFC2013 image, and using raw spectral features from the Zurich QuickBird
image; see the OA curves shown in cyan for ForestPA and shown in yellow for RaF inFigure 7(d, f, k). Additionally, increasing the tree size beyond 100 does not yield detect-able improvements in OA values for ForestPA using any of the considered features, which is similar to the findings for RaF in many works (Belgiu & Drăguţ,2016; Du et al.,
2015; Pal,2005). The performance of boosting ensembles such as AdaBoost and MultiBoostAB could be limited by overfitting due to the focus on challenging-to-classify but fewer sample criteria, particularly for small numbers of training samples with low discrimination capabilities, as shown inFigure 7(k).
In addition to classification accuracy, computational efficiency is considered a key factor when evaluating classifier performance. According to the algorithmic description of ForestPA in the“ForestPA” section, the nested loop operations slow classifier operation. However, its speed with respect to other benchmark EL classifiers is unclear, particularly in the classification of VHR RS images using features of different dimensional-ity. Thus, Figure 8 shows the CPU running times (in seconds) for bagging, AdaBoost, MultiBoostAB, ExtraTrees, RaF, RoF and ForestPA in the training and classification phases. A total of 100 C4.5 DTs for bagging, RaF, RoF and ForestPA, 100 ERDTs for ExtraTrees and 100 iterations for AdaBoost and MultiBoostAB with C4.5 DT are used for both computational efficiency and unbiased evaluation.
According to a comparison of the charts inFigure 8, ForestPA is the slowest of the methods in the training phase, particularly with high-dimensionality data, as expected. However, interestingly, ForestPA is slower than RaF but faster than others in the classification phase, while the worst efficiency is shown by RoF. In contrast, ExtraTrees shows the best computational effi-ciency, followed by RaF in the training phase. Additionally, the best efficiency is shown by RaF, while AdaBoost, bagging, MultiBoostAB and ExtraTrees per-form similar in the classification phase.
Figure 7.OA values versus number of trees or iterations in the considered classifiers using principle components (a, f), raw (k), MP (b, g, l), MPPR (c, h, m), SPMP (d, i, n) and SPMPsM (e, j, o) features of the ROSIS Pavia University (a–e), GRSS-DFC2013 (f–j) and Zurich QuickBird datasets.
Our results first confirm that ForestPA is better than bagging, which is accordance with the findings by Adnan & Islam (2017). However, their finding of ForestPA is better than RaF and ExtraTrees in terms of classification accuracy is arguable, especially in pro-cessing of high dimensional data case. This could be explained by the fact that attribute penalizing strategy may not correctly weight the carried logic rules when the attributes are high dimensional and may be highly correlated (such as in our case). Consequently, the negative impact of removing the attributes that received relatively small weights in any subsequent trees could be severed and limit the gradual weight strategy that ForestPA adopted. Another finding is that ForestPA has slow model training efficiency due to the nested loop operations, but compatible in classification phase compared with considered classifies. Summing-up,
ForestPA may not accommodate large numbers of sam-ples with high dimensionality from the viewpoint of computationally efficient model training, and may not be suitable for handling data with a high dimensionality and high inter-band correlation from the high accuracy classification point of view.
Classification maps
To further compare the adopted approaches for urban land-cover mapping using PCA-transformed features or raw spectral bands and MP, MPPR, SPMP and SPMPsM features, Figure 9 shows the best land-cover maps with OA values that corre-spond to the numbers that are highlighted in bold-face and underlined in Tables 5–7.
Figure 8.CPU times in seconds of training and classification for the considered approaches with 100 trees/iterations using various features from the ROSIS Pavia University (a, d), GRSS-DFC2013 (b, e) and Zurich QuickBird datasets (c, f).
Figure 9.Classification maps with OA values that correspond to the highlighted OA values inTables 5–7for the ROSIS Pavia University, GRSS-DFC2013 and Zurich QuickBird datasets, respectively.
Table 5. Classification accuracy values (UA, AA, OA and kappa) for the considered methods on the ROSIS Pavia University data (F1: PC10, F2: PC10 + MPs, F3: PC10 + MPPR; F4: PC10 + SPMPs; F5: PC10 + SPMPsM). Classifiers SVM Bagging AdaBoost MultiBoostAB Features F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 Asphalt 83.71 95.88 82.27 92.17 89.76 82.23 91.96 70.83 94.87 89.49 82.07 93.85 81.25 97.38 93.92 82.78 93.27 79.02 95.78 94.31 Meadows 73.81 77.27 83.51 94.15 83.21 66.43 73.49 88.99 96.25 71.24 65.24 97.65 95.85 98.17 80.15 67.16 97.33 96.7 97.52 80.25 Gravel 69.46 75.46 71.46 87.09 92.66 54.22 78.37 64.36 75.42 88.52 55.03 77.61 67.37 96 95.71 55.07 77.85 70.8 83.71 94.19 Trees 98.04 99.28 96.83 99.09 98.92 98.2 98.6 95.86 98.47 95.2 98.99 99.09 97.49 99.8 95.5 99.12 98.96 97.94 99.35 96.96 Metal 99.48 99.55 99.85 100 100 99.78 99.93 99.85 99.63 98.96 99.7 99.7 99.93 100 99.33 99.48 99.7 100 99.85 99.93 Bare soil 85.92 70.77 62.16 97.89 80.85 78.94 59.97 54.5 81.94 77.65 83.42 61.4 57.17 93.16 77.63 83.42 61.58 56.39 90.71 77.73 Bitumen 90.45 99.17 91.43 100 93.98 85.41 99.77 94.51 100 97.22 85.79 99.55 90.75 100 99.25 86.17 99.55 93.61 100 99.4 Bricks 89.54 99.16 96.44 98.64 94.54 89.63 98.1 94.68 97.45 89.22 90.47 99.4 98.1 98.86 93.67 90.66 99.51 98.13 98.32 93.7 Shadows 98.2 98.1 97.99 99.26 93.45 97.89 91.55 89.33 99.47 86.17 97.25 98.31 95.99 99.58 94.09 97.36 98.31 97.25 98.94 96.2 UA 87.62 90.52 86.88 96.48 91.93 83.64 87.97 83.66 93.72 88.19 84.22 91.84 87.1 98.11 92.14 84.58 91.78 87.76 96.02 92.52 OA 81.51 84.61 83.63 95.16 87.6 76.38 80.97 82.42 93.98 80.94 76.51 92.21 87.93 97.67 86.52 77.49 92.01 88.18 96.15 86.74 K 0.77 0.8 0.78 0.94 0.84 0.7 0.76 0.77 0.92 0.76 0.71 0.9 0.84 0.97 0.83 0.72 0.89 0.84 0.95 0.83 Classifiers ForestPA ExtraTrees RaF RoF Features F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 Asphalt 81.84 93.14 84.89 94.39 91.12 82.81 95.43 87.94 94.71 93.7 83.67 94.04 89.84 95.11 93.7 84.3 93.92 87.66 92.04 90.95 Meadows 66 91.65 86.47 96.77 71.11 69.43 70.25 82.51 93.23 84.06 64.69 74.99 85.26 95.2 78.51 69.19 92.02 89.35 94.86 78.79 Gravel 52.12 94.04 61.41 71.03 96.62 59.41 80.56 72.18 81.71 87.85 54.84 74.94 72.94 68.03 81.42 61.41 77.37 72.56 78.23 93 Trees 98.73 97 96.51 99.05 90.57 99.22 99.8 99.51 99.97 99.9 98.92 99.18 98.92 99.35 99.45 98.79 99.77 99.22 99.51 97.52 Metal 99.78 99.78 99.93 99.48 99.7 99.85 99.63 100 99.93 99.93 99.85 99.7 99.93 99.93 100 99.93 99.7 100 99.93 100 Bare soil 79.62 60.49 55.06 98.33 93.54 81.89 76.36 60.57 97.26 92.82 81.27 66.24 55.78 89.6 93.34 83.87 69.54 58.82 89.36 90.69 Bitumen 85.26 99.62 95.26 100 100 88.27 99.62 92.48 100 100 85.64 99.62 94.21 100 100 89.77 99.62 91.95 100 100 Bricks 90.49 98.8 98.53 98.18 97.83 91.74 99 98.61 99.54 99.51 90.93 99.54 98.7 99.08 99.24 92.31 98.64 98.45 98.94 98.89 Shadows 99.68 98.84 98.31 98.73 95.14 99.58 98.2 98.84 99.16 99.58 98.2 98.84 98.1 98.94 98.52 99.89 98.84 98.2 98.94 98.63 UA 83.72 92.6 86.26 95.11 92.85 85.8 90.98 88.07 96.17 95.26 84.22 89.68 88.19 93.92 93.8 86.61 92.16 88.47 94.65 94.27 OA 76.24 89.99 84.02 95.84 84.12 78.74 82.42 84.09 94.95 90.97 76.31 82.83 85.06 94.21 87.82 79.28 90.71 86.78 94.05 87.61 K 0.7 0.87 0.79 0.95 0.8 0.73 0.78 0.79 0.94 0.88 0.7 0.78 0.8 0.92 0.84 0.74 0.88 0.83 0.92 0.84
Table 6. Classification accuracy values (UA, AA, OA and kappa) for the considered methods on the GRSS-DFC2013 data (F1: PC7, F2: PC7 + MPs, F3: PC7 + MPPR; F4: PC7 + SPMPs; F5:PC7 + SPMPsM). Classifiers SVM Bagging AdaBoost MultiBoostAB Features F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 Healthy grass 99.54 97.71 92.34 98.51 92.11 97.37 97.37 92 98.29 93.26 98.51 97.37 94.06 99.43 93.26 98.97 98.63 98.29 98.97 99.09 Stressed grass 93.93 98.13 92.29 96.14 82.36 91.82 96.14 92.76 95.09 95.09 97.43 97.55 99.3 98.71 98.25 97.9 98.25 98.71 98.36 98.95 Synthetic grass 99.8 99.6 99.6 100 97.23 99.41 99.6 99.21 96.83 74.26 99.8 99.8 99.6 100 97.23 99.8 99.6 99.6 100 100 Trees 99.2 98.52 95.57 95.57 81.7 98.64 87.95 97.84 94.32 88.75 98.75 91.36 97.27 97.05 91.36 98.3 98.98 97.39 97.61 97.73 Soil 97.92 99.53 99.34 97.16 97.63 96.31 97.63 95.83 94.98 95.55 98.39 100 99.91 98.58 100 99.15 99.72 99.91 97.06 97.16 Water 99.3 95.1 95.1 95.1 97.2 99.3 95.8 100 92.31 90.91 95.1 95.1 95.8 93.71 95.8 95.1 95.1 95.1 99.3 100 Residential 87.92 91.45 93.96 91.57 99.87 87.04 83.02 89.94 85.16 84.03 86.67 83.14 90.94 89.94 91.07 88.05 86.54 90.69 90.94 93.08 Commercial 65.3 84.8 69.61 80.49 65.91 58.52 83.37 89.73 78.44 69.61 64.48 86.65 89.12 82.14 80.29 59.75 79.26 85.22 80.08 80.49 Road 78.76 82.71 72.43 83.05 71.64 69.49 89.6 79.44 80.9 73.33 74.69 89.04 81.13 90.28 87.01 81.02 85.54 82.6 87.57 92.43 Highway 96.32 99.33 96.32 100 96.66 92.64 100 86.96 100 100 95.32 100 100 100 100 97.99 100 100 100 100 Railway 82.28 97.15 96.2 97.78 93.67 85.76 93.35 41.46 94.94 94.94 84.81 95.89 72.47 97.78 96.2 80.7 95.57 88.92 97.47 97.78 Parking lot 1 75.98 83.67 91.45 91.16 75.79 68.59 72.24 60.81 69.74 60.81 64.94 75.22 80.6 73.39 75.98 69.55 60.04 70.8 78.1 84.82 Parking lot 2 72.83 83.77 78.87 77.36 83.4 75.09 71.32 67.17 67.92 81.51 83.02 75.47 79.62 78.11 76.98 81.13 80 84.91 79.62 82.26 Tennis court 100 100 100 100 100 94.74 100 100 100 98.38 96.36 99.6 100 100 100 98.38 100 100 100 100 Running track 98.1 99.79 99.79 99.79 59.2 94.29 100 89.64 99.58 99.58 94.5 100 99.79 99.79 99.79 97.67 99.79 99.79 100 99.79 UA 89.81 94.08 91.52 93.58 86.29 87.27 91.16 85.52 89.9 86.67 88.85 92.41 91.97 93.26 92.21 89.56 91.8 92.8 93.67 94.91 OA 89.72 93.61 91.26 93.49 85.08 86.59 90.26 85.98 89.07 84.87 88.15 91.66 92.03 92.74 91.49 88.45 91.98 91.27 92.77 91.92 K 0.89 0.93 0.9 0.93 0.84 0.85 0.89 0.85 0.88 0.84 0.87 0.91 0.91 0.92 0.91 0.87 0.91 0.9 0.92 0.91 Classifiers ForestPA ExtraTrees RaF RoF Features F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 Healthy grass 98.63 98.97 95.77 98.97 99.09 98.97 98.63 98.29 98.97 99.09 98.86 99.2 99.09 99.09 99.09 98.29 98.63 97.94 99.09 99.54 Stressed grass 96.38 92.29 87.5 91.82 90.19 97.9 98.25 98.71 98.36 98.95 98.25 95.33 98.25 95.21 95.33 96.96 97.55 99.42 98.95 98.36 Synthetic grass 97.82 99.8 99.6 100 100 99.8 99.6 99.6 100 100 99.6 99.6 99.6 100 100 99.6 99.6 99.6 100 99.01 Trees 98.41 98.41 93.07 94.2 93.07 98.3 98.98 97.39 97.61 97.73 98.52 98.52 99.43 93.86 94.55 97.61 99.32 99.89 97.27 97.61 Soil 98.48 97.92 97.73 87.12 84.66 99.15 99.72 99.91 97.06 97.16 98.39 99.72 99.72 96.69 95.74 98.48 99.81 99.62 97.63 97.25 Water 95.1 99.3 98.6 97.9 97.9 95.1 95.1 95.1 99.3 100 95.1 95.1 95.1 97.9 99.3 95.8 97.2 98.6 97.9 98.6 Residential 86.92 84.03 87.55 89.81 89.56 88.05 86.54 90.69 90.94 93.08 87.42 84.4 89.31 91.45 92.33 88.55 83.52 88.68 90.82 93.33 Commercial 66.32 80.49 85.22 79.06 76.39 59.75 79.26 85.22 80.08 80.49 66.12 88.09 90.76 78.64 80.9 65.3 83.98 85.01 84.39 80.49 Road 79.21 86.89 86.21 84.75 85.88 81.02 85.54 82.6 87.57 92.43 78.64 86.44 82.26 86.78 85.42 83.16 86.89 79.32 88.25 87.68 Highway 96.32 100 100 99.67 100 97.99 100 100 100 100 96.99 100 99.67 100 100 94.31 100 100 100 100 Railway 84.81 94.3 41.77 94.94 92.72 80.7 95.57 88.92 97.47 97.78 83.23 94.62 88.92 94.62 97.78 75.32 95.25 76.58 96.52 97.78 Parking lot 1 68.78 57.73 70.89 65.03 74.64 69.55 60.04 70.8 78.1 84.82 65.42 56.29 67.34 63.98 72.33 80.4 78.87 72.43 71.09 84.73 Parking lot 2 75.85 69.43 70.94 72.08 78.49 81.13 80 84.91 79.62 82.26 83.02 74.34 80.38 72.45 79.62 81.89 81.13 86.79 80 84.15 Tennis court 95.55 100 100 99.19 98.79 98.38 100 100 100 100 96.76 100 100 100 100 97.57 100 100 100 100 Running track 96.62 96.83 95.56 97.46 97.04 97.67 99.79 99.79 100 99.79 96.83 97.67 98.1 98.52 98.52 96.83 100 99.58 99.79 99.58 UA 89.01 90.43 87.36 90.13 90.56 89.56 91.8 92.8 93.67 94.91 89.54 91.29 92.53 91.28 92.73 90 93.45 92.23 93.45 94.54 OA 88.48 88.98 87.82 88.42 89.02 89.4 90.49 91.9 92.93 94.55 88.95 89.86 91.63 90.11 91.44 90.54 92.8 91.49 92.46 94.06 K 0.88 0.88 0.87 0.87 0.88 0.88 0.9 0.91 0.92 0.94 0.88 0.89 0.91 0.89 0.91 0.9 0.92 0.91 0.92 0.94
Table 7. Classification accuracy values (UA, AA, OA and kappa) for the considered methods on the Zurich QuickBird data (F1: Raw, F2: Raw +MPs, F3: Raw +MPPR; F4: Raw +SPMPs; F5: Raw +SPMPsM). Classifiers SVM Bagging AdaBoost MultiBoostAB Features F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 Roads 98.41 99.12 98.84 99.55 99.12 97.35 99.29 98.22 99.45 99.45 97.92 99.39 99.16 99.47 99.51 97.41 99.43 98.98 99.49 99.51 Buildings 84.62 93.96 95.22 93.27 92.75 88.82 92.24 92.78 92.08 91.44 82.53 92.54 94.41 90.92 94.63 80.16 92.91 94.16 90.99 94.62 Trees 61.87 81.16 80.57 79.03 84.27 65.57 79.17 80.12 77.45 83.44 63.29 82.17 83.1 82.26 86.24 64.2 82.45 82.52 82.19 86.31 Grass 85.97 92.27 89.43 91.89 89.54 79.38 91.57 84.52 90.59 88.49 75.72 92.63 90.11 90.57 91.43 74.23 92.42 89.3 90.53 90.07 Bare soil 57.93 88.18 89.89 89.74 87.76 52.47 97.98 91 89.66 95.94 47.74 98.74 91.14 91.84 97.8 51.16 98.66 91.48 91.87 97.51 Rails 65.39 82.94 83.38 78.54 82.92 60.86 82.55 82.98 80.17 84.71 60.66 85.02 84.41 81.94 87.12 65.82 85.48 84.4 82.1 86.98 Pools 98.96 99.66 99.66 99.43 99.63 97.73 99.35 98.62 99.12 99.25 96.37 99.39 99.55 99.24 99.28 96.81 99.44 99.6 99.33 99.26 UA 79.02 91.04 91 90.21 90.86 77.45 91.74 89.75 89.79 91.82 74.89 92.84 91.7 90.89 93.72 75.68 92.97 91.49 90.93 93.47 OA 77.15 89.35 89.31 88.02 89.43 78.21 88.16 87.49 87.07 88.85 74.51 89.55 89.96 88.27 91.5 74.28 89.78 89.59 88.29 91.31 K 0.7 0.86 0.86 0.84 0.86 0.72 0.84 0.86 0.83 0.85 0.67 0.86 0.87 0.84 0.89 0.67 0.86 0.86 0.85 0.88 Classifiers ForestPA ExtraTrees RaF RoF Features F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 F1 F2 F3 F4 F5 Roads 97.78 98.02 97.76 99.24 98.9 98.16 98.86 98.84 99.53 99.55 97.76 99.27 98.53 99.37 99.37 98.29 98.96 98.9 99.59 99.45 Buildings 89.09 93.25 93.59 91.92 91.52 86.85 92.56 94.68 90.98 94.4 86.11 93.14 94.24 91.18 92.9 90.41 92.35 92.93 92.08 91.56 Trees 65.16 77.31 80.17 74.74 82.13 67.19 78.17 79.09 78.92 81.56 65.7 77.31 79.95 78.49 81.58 64.33 76.82 81.03 79.03 83.41 Grass 81.38 90.53 84.23 91.32 87.55 80.18 92.07 91.85 90.02 92.24 79.04 92.42 89.02 89.36 91.99 81.74 93.73 90.39 91.2 89.86 Bare soil 47.86 97.74 90.42 89.47 95.18 51.67 96.3 90.52 89.62 92.1 52.82 97.67 90.67 90.82 93.55 54.99 97.08 89.55 90.09 94.55 Rails 59.27 78.41 83.68 78.49 83.95 64.31 81.47 82.28 77.45 85.51 63.92 79.66 83.44 79.2 85.4 54.24 81.63 86.24 80.84 85.85 Pools 97.44 99.32 98.11 98.94 98.97 98.5 99.39 99.66 98.95 99.2 97.5 99.4 99.45 99.02 99.15 98.43 98.9 99.25 99.08 99.34 UA 76.85 90.65 89.71 89.16 91.17 78.12 91.26 90.99 89.35 92.08 77.55 91.27 90.76 89.63 91.99 77.49 91.35 91.18 90.27 92 OA 78.21 87.44 87.76 86.14 88.24 78.45 87.92 88.93 86.75 89.89 77.48 87.79 88.71 86.79 89.33 78.26 87.67 88.96 87.69 89.16 K 0.71 0.83 0.84 0.82 0.84 0.72 0.84 0.85 0.82 0.87 0.71 0.84 0.85 0.83 0.86 0.71 0.84 0.85 0.84 0.86
The improvements from SPMPs and SPMPsM are clear. For instance, the highest classification accuracy values (OA = 97.67%) are reached by AdaBoost using SPMPs features for the ROSIS Pavia University image, by ExtraTrees (OA = 94.55%) using SPMPsM features for the GRSS-DFC2013 image, and by AdaBoost (OA = 91.50%) using SPMPsM features for the Zurich QuickBird image. Moreover, if we compare the results from a specific classifier when using differ-ent features, all the classifiers uniformly show higher classification accuracy values when using SPMPs than MPs, MPPR and SPMPsM on the ROSIS University image, and they show higher classification accuracy values when using SPMPsM than MPs, MPPR and SPMPs on the GRSS-DFC2013 and Zurich QuickBird images, thereby confirming the results obtained in the “Evaluation of SPMPs” section. In addition, ForestPA is superior (+1% to +8% on OA values for the ROSIS Pavia University image, +2% to +5% on OA values for the GRSS-DFC2013, and approximately +1% on OA values for the Zurich QuickBird image) relative to bagging using various types of features from the con-sidered test images. ForestPA shows similar results (±1%) on the Zurich QuickBird image, but worse results (−1% to −6%) on the ROSIS Pavia University and GRSS-DFC2013 images than those of ExtraTrees and RaF in terms of classification accuracy.
Conclusions
In this paper, we have presented the implementation details, analyzed the parameter sensitivity and pre-sented a comprehensive validation of two novel spa-tial feature extractors, namely, SPMPs and SPMPsM, where the latter contains the mean values of super-pixels. Then, we introduced a recently proposed ML approach, namely, ForestPA, which is similar to RaF but is constructed by penalizing attributes used in previous trees in a decision forest. ForestPA was shown to be superior to the bagging, random sub-space, RaF and ExtraTrees algorithms. To fully inves-tigate and evaluate the performance of ForestPA, three VHR multispectral and hyperspectral RS images over urban areas were selected.
The results show that the proposed SPMPs and SPMPsM are effective for urban land-cover map-ping using VHR multi-/hyperspectral RS images. Regarding the influence of the critical parameters of SPMPs and SPMPsM, both the total number of superpixels and the scale step size are crucial, but the former has less effect than the latter. The opti-mal number of superpixels depends not only on the size of the image at hand but also on the sizes and shapes of the objects in that image. Thus, the total number of superpixels must be tuned before further implementation. In general, SPMPs are more compatible with superpixels of large size for
introducing more spatial information, with the potential shortcoming of including mixed features from multiple objects. In contrast, the SPMPsM provides better results with superpixels of smaller size, whereas both spectral and spatial discrimina-tion capabilities can be improved by considering the mean pixel values.
Comparing the algorithmic details of ForestPA with those of conventional RaF, we find that the former is more complex than the latter due to its procedure for penalizing attributes. This finding was confirmed by all the experiments, not only at the training model phase but also at the classification phase with respect to RaF. Specifically, ForestPA showed the worst computational efficiency in the training phase using high-dimensional training sets, even worse than that of RoF. In the classification accuracy evaluation, ForestPA outper-formed only bagging and peroutper-formed worse than ExtraTrees and RaF in most cases and uniformly worse than AdaBoost, RoF and MultiBoostAB on all considered test images. In summary, ForestPA may not be suitable for addressing cases in which the training set is large (e.g. big data scenarios), from both a computational efficiency and classification accuracy perspective.
In future work, we plan to focus on the self-adaptive selection of the number of superpixels and the scale step size in SPMPs and SPMPsM for mod-erate- and high-resolution multi-/hyperspectral and (probably) fully polarimetric SAR images with larger spatial coverage to further improve their classification performance. Additionally, the adoption of high-performance computing techniques, such as parallel, cluster and CPU computing, to accelerate ForestPA is worthy of investigation.
Acknowledgments
This work was partially supported by the National Natural Science Foundation of China (grant nos. 41601440, 41601354), the Youth Innovation Promotion Association Foundation of the Chinese Academy of Sciences (2018476), and the West Light Foundation of the Chinese Academy of Sciences (2016-QNXZ-B-11). In addition, we thank Prof. Paolo Gamba and Devis Tuia and his team, who freely provided the experimental datasets with the ground-truth information.
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
This work was supported by the National Natural Science Foundation of China [41601354,41601440]; West Light Foundation of the Chinese Academy of Sciences
[2016-QNXZ-B-11]; Youth Innovation Promotion Association Foundation of the Chinese Academy of Sciences [2018476].
Author contributions
Alim Samat developed the algorithms, executed all of the experiments, finished the original manuscript and the sub-sequent revisions, and provided part of the funding. Sicong Liu, Claudio Persello, Erzhu Li and Zelang Miao offered valuable suggestions and comments. Jilili Abuduwaili pro-vided part of the funding. Sicong Liu and Erzhu Li, con-tributed to revising of the manuscript.
ORCID
Alim Samat http://orcid.org/0000-0002-9091-6033 Sicong Liu http://orcid.org/0000-0003-1612-4844 Claudio Persello http://orcid.org/0000-0003-3742-5398 Erzhu Li http://orcid.org/0000-0002-5881-618X
References
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2012). SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), 2274–2282. doi:10.1109/TPAMI.2012.120
Adnan, M.N., & Islam, M.Z. (2017). Forest PA: Constructing a decision forest by penalizing attributes used in previous trees. Expert Systems with Applications, 89, 389–403. doi:10.1016/j.eswa.2017.08.002
Belgiu, M., & Drăguţ, L. (2016). Random forest in remote sensing: A review of applications and future directions. ISPRS Journal of Photogrammetry and Remote Sensing, 114, 24–31. doi:10.1016/j.isprsjprs.2016.01.011
Benediktsson, J.A., Palmason, J.A., & Sveinsson, J.R. (2005). Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 480–491. doi:10.1109/TGRS.2004.842478
Blaschke, T., Hay, G.J., Kelly, M., Lang, S., Hofmann, P., Addink, E., … Tiede, D. (2014). Geographic object-based image analysis – Towards a new paradigm. ISPRS Journal of Photogrammetry and Remote Sensing, 87, 180–191. doi:10.1016/j.isprsjprs.2013.09.014
Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140. doi:10.1007/BF00058655
Breiman, L. (2001). Random forests. Machine Learning, 45 (1), 5–32. doi:10.1023/A:1010933404324
Camps-Valls, G., Tuia, D., Bruzzone, L., & Benediktsson, J. A. (2014). Advances in hyperspectral image classifica-tion: Earth monitoring with statistical learning methods. IEEE Signal Processing Magazine, 31(1), 45–54. doi:10.1109/MSP.2013.2279179
Chan, J.C.W., & Paelinckx, D. (2008). Evaluation of ran-dom forest and Adaboost tree-based ensemble classifica-tion and spectral band selecclassifica-tion for ecotope mapping using airborne hyperspectral imagery. Remote Sensing of Environment, 112(6), 2999–3011. doi:10.1016/j. rse.2008.02.011
Cortes, C., & Vapnik, V. (1995). Support vector machine. Machine Learning, 20(3), 273–297. doi:10.1007/ BF00994018
Costa, H., Foody, G.M., & Boyd, D.S. (2017). Using mixed objects in the training of object-based image classifications.
Remote Sensing of Environment, 190, 188–197. doi:10.1016/ j.rse.2016.12.017
Dalla Mura, M., Villa, A., Benediktsson, J.A., Chanussot, J., & Bruzzone, L. (2011). Classification of hyperspectral images by using extended morphological attribute pro-files and independent component analysis. IEEE Geoscience and Remote Sensing Letters, 8(3), 542–546. doi:10.1109/LGRS.2010.2091253
Debes, C., Merentitis, A., Heremans, R., Hahn, J., Frangiadakis, N., van Kasteren, T.,… Philips, W. (2014). Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7 (6), 2405–2418. doi:10.1109/JSTARS.2014.2305441 Du, P., Samat, A., Waske, B., Liu, S., & Li, Z. (2015).
Random forest and rotation forest for fully polarized SAR image classification using polarimetric and spatial features. ISPRS Journal of Photogrammetry and Remote Sensing, 105, 38–53. doi:10.1016/j.isprsjprs.2015.03.002 Fauvel, M., Benediktsson, J.A., Chanussot, J., &
Sveinsson, J.R. (2008). Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 46(11), 3804–3814.
Fauvel, M., Tarabalka, Y., Benediktsson, J.A., Chanussot, J., & Tilton, J.C. (2013). Advances in spectral-spatial classi-fication of hyperspectral images. Proceedings of the IEEE, 101(3), 652–675. doi:10.1109/JPROC.2012.2197589 Gamba, P., Dell’Acqua, F., Stasolla, M., Trianni, G., &
Lisini, G. (2011). Limits and challenges of optical high-resolution satellite remote sensing for urban applica-tions. In X. Yang (Ed.), Urban remote sensing – Monitoring, synthesis and modelling in the urban envir-onment (pp. 35–48). New York: Wiley.
Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63(1), 3–42. doi:10.1007/s10994-006-6226-1
Godinho, S., Guiomar, N., & Gil, A. (2016). Using a stochastic gradient boosting algorithm to analyse the effectiveness of Landsat 8 data for montado land cover mapping: Application in southern Portugal. International Journal of Applied Earth Observation and Geoinformation, 49, 151–162. doi:10.1016/j. jag.2016.02.008
Gu, H., Li, H., Yan, L., Liu, Z., Blaschke, T., & Soergel, U. (2017). An object-based semantic classification method for high resolution remote sensing imagery using ontology. Remote Sensing, 9(4), 329. doi:10.3390/ rs9040329
Kasetkasem, T., Arora, M.K., & Varshney, P.K. (2005). Super-resolution land cover mapping using a Markov random field based approach. Remote Sensing of Environment, 96(3), 302–314. doi:10.1016/j. rse.2005.02.006
Kavzoglu, T., Colkesen, I., & Yomralioglu, T. (2015). Object-based classification with rotation forest ensemble learning algorithm using very-high-resolution WorldView-2 image. Remote Sensing Letters, 6(11), 834–843. doi:10.1080/2150704X.2015.1084550
Li, J., Bioucas-Dias, J.M., & Plaza, A. (2013). Spectral–Spatial classification of hyperspectral data using loopy belief pro-pagation and active learning. IEEE Transactions on Geoscience and Remote Sensing, 51(2), 844–856. doi:10.1109/TGRS.2012.2205263
Liao, W., Chanussot, J., Dalla Mura, M., Huang, X., Bellens, R., Gautama, S., & Philips, W. (2017). Taking optimal advantage of fine spatial resolution: Promoting
partial image reconstruction for the morphological ana-lysis of very-high-resolution images. IEEE Geoscience and Remote Sensing Magazine, 5(2), 8–28. doi:10.1109/ MGRS.2017.2663666
Ma, L., Cheng, L., Li, M., Liu, Y., & Ma, X. (2015). Training set size, scale, and features in geographic object-based image analysis of very high resolution unmanned aerial vehicle imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 102, 14–27. doi:10.1016/j.isprsjprs.2014.12.026 Ma, L., Li, M., Ma, X., Cheng, L., Du, P., & Liu, Y. (2017).
A review of supervised object-based land-cover image classification. ISPRS Journal of Photogrammetry and Remote Sensing, 130, 277–293. doi:10.1016/j. isprsjprs.2017.06.001
Pal, M. (2005). Random forest classifier for remote sensing classification. International Journal of Remote Sensing, 26 (1), 217–222. doi:10.1080/01431160412331269698 Plaza, A., Benediktsson, J.A., Boardman, J.W., Brazile, J.,
Bruzzone, L., Camps-Valls, G., … Trianni, G. (2009). Recent advances in techniques for hyperspectral image processing. Remote Sensing of Environment, 113, S110– S122. doi:10.1016/j.rse.2007.07.028
Rafael, C.G., & Richard, E.W. (2010). Digital image processing (3rd ed., pp. 687–698). Beijing: Publishing House of Electronic Industry.
Rajadell, O., García-Sevilla, P., & Pla, F. (2013). Spectral– Spatial pixel characterization using Gabor filters for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 10(4), 860–864. doi:10.1109/ LGRS.2012.2226426
Rätsch, G., Onoda, T., & Müller, K.R. (2001). Soft margins for AdaBoost. Machine Learning, 42(3), 287–320. doi:10.1023/ A:1007618119488
Rodriguez, J.J., Kuncheva, L.I., & Alonso, C.J. (2006). Rotation forest: A new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 28(10), 1619–1630. doi:10.1109/ TPAMI.2006.211
Samat, A., Du, P., Liu, S., Li, J., & Cheng, L. (2014). E2LMs: Ensemble extreme learning machines for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(4), 1060–1069. doi:10.1109/JSTARS.2014.2301775
Samat, A., Li, J., Liu, S., Du, P., Miao, Z., & Luo, J. (2016). Improved hyperspectral image classification by active learning using pre-designed mixed pixels. Pattern Recognition, 51, 43–58. doi:10.1016/j.patcog.2015.08.019 Samat, A., Persello, C., Liu, S., Li, E., Miao, Z., &
Abuduwaili, J. (2018). Classification of VHR multispec-tral images using extratrees and maximally stable extre-mal region-guided morphological profile. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(9), 3179–3195. doi:10.1109/ JSTARS.2018.2824354
Webb, G.I. (2000). Multiboosting: A technique for combin-ing boostcombin-ing and waggcombin-ing. Machine Learncombin-ing, 40(2), 159–196. doi:10.1023/A:1007659514849
Xia, J., Du, P., He, X., & Chanussot, J. (2014). Hyperspectral remote sensing image classification based on rotation forest. IEEE Geoscience and Remote Sensing Letters, 11(1), 239–243. doi:10.1109/LGRS.2013.2254108 Zhang, Q., Wang, J., Gong, P., & Shi, P. (2003). Study of urban spatial patterns from SPOT panchromatic imagery using textural analysis. International Journal of Remote Sensing, 24(21), 4137–4160. doi:10.1080/ 0143116031000070445
Zhao, W., Du, S., Wang, Q., & Emery, W.J. (2017). Contextually guided very-high-resolution imagery clas-sification with semantic segments. ISPRS Journal of Photogrammetry and Remote Sensing, 132, 48–60. doi:10.1016/j.isprsjprs.2017.08.011
