Using Artificial Neural Using Artificial Neural Networks toNetworks to

Using Artificial Neural Using Artificial Neural

Networks to

Networks to Predict Predict

Malignancy of Ovarian Tumors Malignancy of Ovarian Tumors

C. Lu¹, J. De Brabanter¹, S. Van Huffel¹, I. Vergote², D. Timmerman²

1Department of Electrical Engineering,

Katholieke Universiteit Leuven, Leuven, Belgium, Department of Obstetrics and Gynecology,

Overview Overview



Introduction



Data Exploration



Input Selection



Model Building



Model Evaluation



Conclusions

Introduction Introduction



Problem

 ovarian masses: a common problem in gynecology.

 develop a reliable diagnostic tool to discriminate

preoperatively between benign and malignant tumors.

 assist clinicians in choosing the appropriate treatment.

 Data

 Patient data collected at Univ. Hospitals Leuven, Belgium, 1994~1999

 425 records, 25 features.

 291 benign tumors, 134 (32%) malignant tumors.

Introduction Introduction



Methods



Data exploration:

 Data preprocessing, univariate analysis, PCA, factor analysis, discriminant analysis, logistic regression…



Modeling:

 Logistic regression (LR) models

 Artificial neural networks (ANN): MLP, RBF

 Performance measures:

Receiver operating characteristic (ROC) analysis

ROC curves

 constructed by plotting the sensitivity versus the 1- specificity, or false positive rate, for varying probability cutoff level.

 visualization of the relationship between

sensitivity and specificity of a test.

 Area under the ROC curves (AUC)

measures the probability of the classifier to correctly classify events and

nonevents.

Data exploration Data exploration



Univariate analysis:



preprocessing:



descriptive statistics, histograms…

Variable (symbol) Benign Malignant

Demographic Age (age)

Postmenopausal (meno) 45.6  15.2

31.0 % 56.9  14.6

66.0 %

Serum marker CA 125 (log) (l_ca125) 3.0  1.2 5.2  1.5

CDI High color score (colsc3,4) 19.0% 77.3 %

Morphologic Abdominal fluid (asc) Bilateral mass (bilat) Unilocular cyst (un)

Multiloc/solid cyst (mulsol) Solid (sol)

Smooth wall (smooth) Irregular wall (irreg) Papillations (pap)

32.7 % 13.3 % 45.8 % 10.7 % 8.3 % 56.8 % 33.8 % 12.5 %

67.3 % 39.0 % 5.0 % 36.2 % 37.6 % 5.7 % 73.2 % 53.2 %

Demographic, serum marker, color Doppler imaging and morphologic variables

Data exploration Data exploration



Multivariate analysis:



factor analysis



biplots

Fig. Biplot of Ovarian Tumor data.

The observations are plotted as points (0=benign,

1=malignant), the variables are plotted as vectors from the origin.

- visualization of the correlation between the variables

- visualization of the relations between the variables and clusters.

Input Selection Input Selection



Stepwise logistic regression analysis



Searching in the feature space

 fix several of the most significant variables, then vary combinations with the other predictive variables.

 different logistic regression models with different subsets of input variables were built and validated.

 subsets of variables were selected according to their predictive performance on the training set and test set.

Model building Model building



Logistic regression (LR) model



Artificial neural networks

 feed-forward neural networks, universal approximators:

- multi-layer perceptron (MLP)

- generalized regression network (GRNN)

 generalization capacity: central issue during network design and training.

Model building Model building

- - LR LR



Parameter estimation:

- maximum likelihood - iterative procedure

. . . . .

b i a s

P r o b a b i l i t y o f m a l i g n a n c y

g



) exp(

1 ) 1

(a a

g   

Fig. Architecture of LRs for Predicting Malignancy of Ovarian Tumors

 structure:



Training

Bayesian regularization combined with Levenberg- Marquardt optimization.

Model Building Model Building

- ANN - MLP - ANN - MLP

M : n u m b e r o f h i d d e n n e u r o n s

d : n u m b e r o f i n p u t v a r i a b l e s

. . . . .

b i a s

b i a s

P r o b a b i l i t y o f m a l i g n a n c y

g



g

 g

 g



M O D E L 1 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s o l i r r e g p a p M O D E L 2 : m e n o c o l s c 3 c o l s c 4 l _ c a 1 2 5 a s c s m o o t h p a p

) exp(

1 ) 1

( a a

g   







 



 









 

M j

d i

i ji

j g w x

w g

y

0 0

) 1(

) 2 (

Fig. Architecture of MLPs for Predicting Malignancy of Ovarian Tumors

 structure

MLP1: 8-3-1 MLP2: 7-3-1

Model Building Model Building

– ANN - GRNN – ANN - GRNN

Fig. Architecture of GRNNs for Predicting Malignancy of Ovarian Tumors

. . . . . o u t p u t







 _N

j j N j

j

x x t

x y

j

1 1

) (

) ( )

(





tⁱ: t a r g e t o u t p u t o f i t h t r a i n i n g d a t a















 



 ₂

2

exp 2 ) (

j i

j h

x x x



N : # t r a i n i n g d a t a x : i n p u t v e c t o r

x : i n p u t v e c t o r g



 ₂

 ₁  _N

… … …

… … …

t¹ t² t^N

 Training^:

GRNN is another term for Nadaraya- Watson kernel regression. No iterative training; the widths of RBF units h act as smoothing parameters, chosen by cross- validation.

 structure

•RMI: risk of malignancy index = score ×

 Training set : data from the first treated 265 patients

 Test set : data from the latest treated 160 patients

Model Evaluation Model Evaluation

- Holdout CV - Holdout CV

AUC estimates and standard errors from hold out CV

 stratified 7-fold CV

 for each run of 7- fold CV:

mAUC :

(_iAUC_i)/7, i =1,…7, AUC_i is the AUC on the ith validation set

expected ROC:

Averaging.

 Repeat 7-fold CV 30 times with different partitions => better

Model Evaluation Model Evaluation

- K-fold CV - K-fold CV

Box plot of meanAUC from 7-fold CVExpected ROC curves from k-fold CV

 Multiple comparison of mAUCs:

one-way ANOVA followed by Tukey multiple comparison

.

Rank ordered significant subgroups from multiple comparison on mean AUC

Models ^{RMI LR2 LR1 GRN1 GRN2 MLP2 MLP1}

mean

mAUC 0.882 0.943 0.954

SD 0.003

0.939 0.003

0.941

0.004 0.003

0.944 0.003

0.944

0.003 0.003

Note: The subsets of adjacent means that are not significantly different at 95% confidence level are indicated by drawing a line under the subsets.

Model Evaluation Model Evaluation

- K-fold CV

- K-fold CV

Conclusions Conclusions



Summary

 AUC is the advocated performance measure

 Data exploratory analysis helps to analyze the data set.

 MLPs have the potential to give more reliable prediction.



Future work

 Develop models with kernel methods, e.g. LS-SVM

 ANNs are blackbox models. A hybrid methodology, greybox models might be more promising