3D active shape modeling for cardiac MR and CT image
segmentation
Assen, Hans Christiaan van
Citation
Assen, H. C. van. (2006, May 10). 3D active shape modeling for cardiac MR
and CT image segmentation. Retrieved from https://hdl.handle.net/1887/4460
Version:
Corrected Publisher’s Version
License:
Licence agreement concerning inclusion of doctoral
thesis in the Institutional Repository of the University
of Leiden
Downloaded from:
https://hdl.handle.net/1887/4460
Note: To cite this publication please use the final published version (if
Stellingen
behorend bij het proefschrift
’3D Active Shape Modeling for Cardiac MR and CT Image Segmentation’
van Hans Christiaan van Assen 1. Adaptieve weefselklassificatie tijdens het matchen van een ”Active Shape Mo-del” (ASM) heeft als voordeel boven grijswaardentraining, dat een dergelijk ASM toepasbaar is op een breder veld van radiologische modaliteiten. (dit proefschrift) 2. Gedistribueerde computertechnologie is een vereiste voor het uitvoeren van groot-schalige beeldverwerkingsstudies en voor het bereiken van een goede fijnafstel-ling van klinische multi-parameter-afhankelijke beeldverwerkingstoepassingen. (dit proefschrift)
3. Door grijswaardentraining bij de constructie van statistische vormmodellen ach-terwege te laten, worden dergelijke modellen ook toepasbaar op beeldvlakken die een andere orientatie in de ruimte hebben dan de beeldvlakken die tijdens de modeltrainingfase zijn gebruikt. (dit proefschrift)
4. Door langs de oppervlakken van een ”Active Shape Model” (ASM) de uit een beeld-dataset afgeleide informatie met een weging te propageren, wordt een der-gelijk ASM toepasbaar op dunner bemonsterde datasets zonder significant schillende segmentatieresultaten te genereren ten opzichte van resultaten ver-kregen op dicht bemonsterde datasets. (dit proefschrift)
5. Wanneer parameters ter kwalificatie van statistische modellen, zoals generalise-rend vermogen, compactheid en specificiteit niet eenduidig zijn bij de kwalificatie van modellen die met verschillende puntcorrespondenties zijn geconstrueerd, dan zijn de kwaliteitsverschillen van de verkregen segmentatieresultaten volledig af-hankelijk van het gebruikte feature-detectie-algoritme. (dit proefschrift)
6. Als gevolg van de grote verscheidenheid in vormen en verschijningen van zowel gezonde als pathologische structuren in humane medische beelden, en vanwe-ge de verscheidenheid in kwaliteit van de medische beelden zelf, vanwe-gecombineerd met de mogelijke consequenties van de uitkomst voor de pati¨ent, is de medische beeldverwerking de moeilijkste vorm van beeldverwerking.
7. In de nabije toekomst zullen computer-ondersteund diagnosticeren en -beslissen onmisbaar zijn in de dagelijkse klinische praktijk.
8. Een model vertegenwoordigt enkel de beschikbare kennis, het vertegenwoordigt niet ”de waarheid”.
9. ”Meten is weten”, visualiseren is begrijpen.
3D Active Shape Modeling
for Cardiac MR and CT
Colophon
This thesis was typeset by the author using LATEX2
ε. The main body of the text was
set using a 9 points New Century Schoolbook font, and for the sans serif parts the Helvetica font was used; both fonts are c Adobe Systems Incorporated. Images were included formatted as Encapsulated Postscript, and represented either in grayscale, or in the CMYK color scheme. The output was converted to PDF and transferred to film for printing.
About the cover
The front and back covers were designed by Sandra Batelaan.
The covers show an abstraction of a mesh representing a cardiac model. The nodes and edges symbolize the point distribution at its surface(s). When the back and front cov-ers are seen together, from left to right a transition from chaos to order appears while the free nodes are attached to the structured mesh. This represents the formation of a model from a set of individual training shapes. The size of the nodes represents the amount of local variation, which varies from node to node.
3D Active shape modeling for cardiac MR and CT image segmentation Assen, Hans Christiaan van
Printed by Optima Grafische Communicatie, Rotterdam, The Netherlands ISBN 90-8559-163-5
c
3D Active Shape Modeling
for Cardiac MR and CT
Image Segmentation
Proefschrift ter verkrijging van
de graad van Doctor aan de Universiteit Leiden, op gezag van de Rector Magnificus Dr. D.D. Breimer,
hoogleraar in de faculteit der Wiskunde en Natuurwetenschappen, en die der Geneeskunde,
volgens besluit van het College voor Promoties te verdedigen op woensdag 10 mei 2006
klokke 15.15 uur
door
Ir. Hans Christiaan van Assen geboren te Leeuwarden
Promotiecommissie
promotor: Prof. dr. ir. J.H.C. Reiber co-promotor: Dr. ir. B.P.F. Lelieveldt referent: Prof. dr. M. Sonka
University of Iowa, Iowa, USA overige leden: Prof. dr. A. de Roos
Dr. A.F. Frangi
Universitat Pompeu Fabra, Barcelona, Spanje
Financial support for the publication of this thesis was kindly provided by: Stichting Beeldverwerking Leiden
Medis medical imaging systems B.V.
Contents
Colophon ii Contents v 1 Introduction 1 1.1 Background 2 1.1.1 Cardiac anatomy 2 1.1.2 Heart disease 21.1.3 Diagnosis: cardiac imaging and quantification 3
1.1.4 Automation in diagnostic quantification 4
1.2 Automatic segmentation 7
1.2.1 Knowledge-based solutions 7
1.2.2 Statistical shape modeling 7
1.3 Motivation of this work 10
1.4 Structure of this thesis 11
2 3D-ASM Matching for LV Segmentation in Cardiac CT 15
2.1 Introduction 16 2.2 Methodology 18 2.2.1 Model generation 18 2.2.2 Matching Algorithm 20 2.3 Experimental setup 22 2.3.1 Training data 22 2.3.2 Evaluation data 22
2.3.3 Model matching parameters 22
2.3.4 Quantitative evaluation 23
2.4 Results 24
2.5 Discussion and conclusions 24
3 Cardiac LV Segmentation Using a 3D ASM Driven by Fuzzy Inference 27
3.1 Introduction 28
3.2 Methodology 30
3.2.1 3D model generation 30
3.2.2 Model matching 30
3.2.3 Edge detection using Fuzzy Inference 31
3.3 Experimental Setup 33
3.4 Results 33
3.5 Discussion and conclusions 35
4 A 3D-ASM driven by Fuzzy Inference applied to Cardiac CT and MR 37
4.1 Introduction 38
4.2 Background 41
4.2.1 Shape Modeling 41
vi Contents
4.3 3D-ASM 42
4.3.1 Model generation 42
4.3.2 Model matching 43
4.3.3 Edge detection using Fuzzy Inference 45
4.3.4 Robust update selection 48
4.4 Experimental setup 50
4.4.1 Training and testing data 50
4.4.2 Matching parameters 50
4.4.3 Quantitative assessment indices 51
4.5 Results 53
4.5.1 Quantitative data 53
4.6 Discussion 55
4.7 Conclusion 59
5 Parametric Optimization of a Model-Based Segmentation Algorithm 61
5.1 Introduction 62 5.1.1 Shape Model 63 5.1.2 Appearance Model 63 5.1.3 Using sectorization in FCM 64 5.1.4 Matching Procedure 66 5.2 Parametric Optimization 66
5.2.1 Parameters Related to the Shape Model 66
5.2.2 Parameters Related to the Appearance Model 67
5.2.3 Fixed Settings 67
5.3 Evaluation Data Set 67
5.4 Grid Computing Approach 68
5.5 Quantitative Assessment 68
5.6 Conclusions 70
6 Assessment of an Autolandmarked Statistical Shape Model 71
6.1 Introduction 72
6.2 Construction of the Statistical Shape Models 73
6.2.1 Training Data Set 73
6.2.2 Model Building 74
6.3 PDM Parameterizations 74
6.4 Shape Model Characterization 76
6.4.1 Shape Analysis 76
6.5 Segmentation Performance Assessment 77
6.5.1 Evaluation Data Set 78
6.5.2 Segmentation Tests 79
6.6 Discussion 80
6.7 Conclusion 81
7 SPASM: 3D-ASM for Sparse and Arbitrarily Oriented MRI Data 83
Contents vii
7.2.4 Update propagation to undersampled surface regions 89
7.2.5 Feature point detection using Fuzzy Inference 92
7.3 Experimental setup 94
7.3.1 Test data and protocol 94
7.3.2 Matching experiments 97
7.4 Results 97
7.5 Discussion 101
7.5.1 Segmentation performance 101
7.5.2 Sensitivity to initial model placement 104
7.5.3 Protocol independence 105
7.5.4 Limitations 105
7.5.5 Comparison to other work 105
7.6 Conclusions 108
8 Efficient Reconstruction of Cardiac LV Surfaces Using SPASM 109
8.1 Introduction 110
8.2 Methods 111
8.2.1 Background 111
8.2.2 SPASM model construction 111
8.2.3 SPASM matching: edge detection 112
8.2.4 SPASM matching: update propagation 112
8.2.5 Experiments 113
8.3 Results 115
8.4 Discussion and Conclusions 116
9 Summary and Conclusions 119
9.1 Summary 120
9.2 Conclusions and future work 124
’Wheresoever you go, go with all your heart.’
Confucius (551–479 BC)
2 1.1 Background
1.1
Background
1.1.1 Cardiac anatomy
The heart is the organ that maintains blood circulation through the body. It contains four cavities, the left and right atria, and the left and right ventricles (see Fig.1.1). In the heart the left and right side are separated, each supporting one of two different circulations. The systemic circulation is maintained by the left side and runs through the aorta, the organ tissues, the brain, and the extremities, and the pulmonary circu-lation is maintained by the right side of the heart and runs through the pulmonary artery and the lungs. To avoid backward flow of the blood, valves between the atria and the ventricles and between the ventricles and the outflow tracts exist that open and close at the right moment in the cardiac cycle.
The cardiac cycle itself consists of an electrically organized sequence of contractions of the atria and ventricles. Hypoxic blood enters the heart from the veins at the right atrium. From there it is injected into the right ventricle through the tricuspid valve by contraction of the atrium. The filling of the right ventricle is followed by its con-traction, causing the blood to flow through the pulmonary valve, the pulmonary artery and lungs, where it is oxygenated and releases carbon dioxide. The oxygenated blood returns to the heart at the left atrium. Together with the right atrium, the left atrium contracts injecting the blood through the mitral valve into the left ventricle. Shortly after, the left ventricle contracts, pumping the blood through the aortic valve into the aorta. The first branches in the aorta, and the only branches in the ascending aorta, are the coronary arteries. These supply the myocardium itself with oxygenated blood.
1.1.2 Heart disease
The cardiac contraction relies on a delicate balance of events, where the coronary circu-lation, electrical system, valves and myocardium all function in a coordinated manner. The major causes for cardiovascular disease (CVD) are:
• atherosclerosis, depositing plaque in vessels like, e.g., the renal arteries, the carotid arteries and the coronary arteries. Plaque is composed of fatty sub-stances, cholesterol, cellular waste products, calcium and other subsub-stances, and may cause stenosis, a local decrease in vessel diameter that leads to changes in blood flow. When plaques rupture, embolies (blood clots) can drift through the vascular system, with the risk of blocking arteries and/or veins (embolism). An embolism in the coronary arteries can lead to myocardial infarction.
• hypertension (elevated blood pressure), forcing the heart to pump against higher pressures. Causes for hypertension can be, among others, reduced kidney func-tion, renal artery stenosis, and stress. Hypertension can lead to hypertrophic obstructive cardiomyopathy (HOCM) and result in congestive heart failure.
1.1 Background 3
• valvular failure/dysfunction, i.e. valves that either don’t close or open normally causing either regurgitation or limited flow. A number of causes of valvular fail-ure are inflammation of the cardiac valves, valvular displacement, and valvular stenosis.
• arrythmia, caused by failure in the electrical system and resulting in unbalance in the sequence of contractions of the chambers. Arrythmia may occur when any portion of the propagation of the trigger signal given by the sino-atrial node is interrupted or disturbed. Normally this signal is transmitted from the right atrium, travels across the atria, through the septum to specialized tissues slow-ing down its progression and passslow-ing it on to the ventricles.
In the US, cardiovascular disease (CVD) kills more people every year than cancer, chronic lower respiratory diseases, accidents, diabetes mellitus, and influenza and pneumonia combined. Because CVD is the primary cause of death, its prevalence is monitored closely. In 2002, 38% of all deaths were caused by CVD. Of over 2,400,000 deaths from all causes, nearly 60% had CVD as primary or contributing cause. Since 1984, CVD has experienced higher prevalence in women than in men. From the 2005 update on heart disease and stroke statistics, on average every 34 seconds someone in the US dies of CVD [1]. CVD does not only affect old people, but also children. For children under age 15, CVD is the number 2 cause of death. In 2003, in the Nether-lands, 47,992 of in total 142,355 deaths were caused by cardiac and (cardio-)vascular diseases.
1.1.3 Diagnosis: cardiac imaging and quantification
Cardiac Left Ventricular (LV) function and mass are important prognostic factors in risk assessment and management of heart disease [2]. LV function can be divided in global function, regional function, function related to perfusion, to infarcted tissue vi-ability and to metabolism. In order to assess cardiac function, patients can undergo several tests. Cardiac function can be monitored by ElectroCardioGrams (ECGs), and measuring (systolic and diastolic) blood pressure. In addition, patients can be diag-nosed using cardiac imaging. Cardiac imaging can be performed with multiple imag-ing modalities, e.g., (3D) cardiac Ultrasound (US), cardiovascular cine-angiography (XA), Computed Tomography (CT, or MSCT: Multi-slice CT), Single Photon Emission Computed Tomography (SPECT), Positron Emission Tomography (PET), and Mag-netic Resonance Imaging (MRI). For typical images produced by these modalities, see Figure1.2. For advantages and disadvantages of the different modalities with respect to quantification of cardiac function see Table1.1.
4 1.1 Background
Figure 1.1: The human heart. ( c Edwards Lifesciences Corporation, used with per-mission)
A recently proposed MRI technique is based on acquisition of the radially (RAD) ori-entated long-axis (LA) images, with the common intersection line coinciding with the long LV axis. This provides a clearer depiction of the basal part of the heart and can be alternatively utilized for the quantification of LV volume and mass [4]. However, more sparsely sampled regions exist in the mid-ventricular and basal endocardium and epicardium. A third alternative is the use of 2- and 4-chamber views for quantifi-cation. Because those views rely on geometric assumptions, quantification based on these views is less accurate.
1.1.4 Automation in diagnostic quantification
1.1 Background 5
Table 1.1: Imaging modalities, their properties with respect to quantification of car-diac function, and their applicability. ∗MI=Myocardial Infarction, CHD=Coronary
Heart Disease, VD=Valvular Dysfunction, EmSc=Emergency Screening, HF=Heart Failure, GF=Global Function, RF=Regional Function, MP=Myocardial Perfusion, MV=Myocardial Viability, Met=Metabolism
Modality Advantages Disadvantages Pathology∗ Analysis∗
2D US fast acquisition, high temporal resolution
limited spatial resolu-tion, relies on geometric shape assumptions MI, CHD, HOCM, EmSc, VD GF, RF 3D US (rotational) 3D volumetric data sets [5]
suboptimal image qual-ity
MI, CHD, HOCM, EmSc, VD
GF, RF
(MS)CT fast acquisition, high spatial resolution, nearly isotropic voxel size
ionized radiation, ad-ministration of contrast agent (semi-invasive, toxic), moderate tem-poral resolution, axial image acquisition
MI, CHD, HOCM, HF
GF, RF, MV
MRI non-invasive, high spa-tial resolution, high temporal resolution, intrinsically high blood-myocardium contrast, arbitrary image orien-tation
prolonged examination times, breath holding is difficult for patients, low through-plane res-olution, reproducibility of quantitative results depends on the accuracy of the positioning of the image slices [6]
MI, Ischemia, HOCM, VD, HF
GF, RF, MV, MP
XA high spatial resolution, high temporal resolu-tion
ionized radiation, ad-ministration of contrast agent (invasive, toxic), low image quality for ES, strong dependence on geometrical models
MI, CHD GF, RF
PET natural way of perform-ing functional measure-ments: perfusion, func-tion, oxygenafunc-tion, pro-tein concentration
nuclear radiation, ad-ministration of nuclear tracer (semi-invasive), low spatial resolution
MI, Ischemia [7] MV [7], MP, Met
SPECT natural way of perform-ing functional mea-surements: perfusion function, oxygenation, protein concentration; investigations over longer time interval than PET possible, because radionuclides have a longer physical half-life than with PET
nuclear radiation, nuclear tracer (semi-invasive), lower spatial resolution than PET, lower temporal resolu-tion than PET, images hard to interpret
6 1.1 Background
(a) (b) (c)
(d) (e) (f)
(g) (h) (i)
Figure 1.2: (a) 2D Cardiac Ultrasound (b) 3D Cardiac Ultrasound (c) X-Ray LV-angiography (d) Short-axis cardiac MRI view (e) Long-axis cardiac MRI view (f) Car-diac PET (g) Axial carCar-diac CT view (h) Short-axis carCar-diac CT view (with reconstruction artifacts) (i) Cardiac SPECT . (Image (b) courtesy Marco M. Voormolen MSc, Erasmus MC)
that additional capacity be built for research and development and that institutional frameworks that facilitate cardiovascular disease prevention and control be developed. Due to the increasing prevalence of cardiac and cardiovascular diseases, and if the recommendations of the IOM are supported and executed, an increasing number of diagnostic assessments and interventions will be performed in the future. Already, the amount of diagnostic assessments is increasing rapidly, and as a consequence an increasing number of imaging operations is ordered. Moreover, for the purpose of com-prehensive diagnostic analysis, multiple types of assessments are performed which in-clude: functional, anatomical, perfusion, and rest-stress imaging. This also increases the amount of acquired data drastically.
1.2 Automatic segmentation 7
developments increase the production of diagnostic image data that will have to be analyzed. Consequently, it is impossible to cope with the vast amount of image data by manual interaction and visual perception, and the combination of many different assessments per patient make interpretation a difficult task. This can lead to different analysis results between different interpreting physicians, i.e. a large interobserver variability results. Also, large intra-observer variations are observed.
It is therefore an obvious conclusion that assessment and diagnosis based on imaging data has to be automated to the maximal extent possible, minimizing time invested by the physicians and increasing analysis robustness and consistency.
1.2
Automatic segmentation
For quantification in (medical) imagery, image segmentation is required. Medical im-age segmentation in particular is a very difficult problem. Spurious edges, fuzzy edges, absent edges, low signal-to-noise ratios, (reconstruction) artifacts, image misregistra-tion, and missing data, are only a few of the problems that appear in medical image segmentation. Nevertheless, medical image segmentation has been a very active field of research for many years.
1.2.1 Knowledge-based solutions
To overcome many problems mentioned above, knowledge-based segmentation ap-proaches have been proposed. Inclusion into the segmentation framework of a pri-ori knowledge has shown to be instrumental for robust performance. This can be knowledge involving, e.g., (ordering of) gray values among tissues, organ shape related knowledge, anatomical and geometrical related knowledge. Examples of such knowl-edge include: blood appears brighter than muscle, which in turn appears brighter than air in bright blood MR and MSCT images, and, the heart is located between the lungs and the diaphragm.
Knowledge extracted from imagery in general and medical imagery in particular, is not inherently related to features immediately visible in images. A particular form of knowledge inclusion is that related to shape characteristics, that captures prior knowledge about the typical shape and spatial context of an organ.
1.2.2 Statistical shape modeling
A generalized method of shape characterization became available with the develop-ment of statistical shape models, capable of capturing statistical shape and gray level information from sets of typical examples. Statistical shape modeling has proven to be one of the most influential developments in knowledge-driven image analysis in the last decade.
Point Distribution Model
8 1.2 Automatic segmentation
into a training framework. This shape knowledge was expressed in coordinates of landmark points being commonly present and easily identifiable locations in the im-ages. A mean shape and a number of characteristic shape variations with respect to this mean shape are calculated after alignment of the shapes. With these, within certain statistical limits, shapes resembling those from the training set can be (re-) constructed by the mean shape and a linear combination of the shape variations. Cootes’ work inspired the further development of PDMs and ASMs. PDMs have been explored extensively in 2D and 3D. In 2D, Suinesiaputra et al. [12,13] uses a PDM with deformation modes determined by Independent Component Analysis (ICA) [14] for extraction of contractility patterns of the cardiac left ventricle and for detection of abnormal cardiac contraction regions. In 3D, PDMs are mainly used for morphomet-rics of complex shapes. For instance, Caunce et al. [15] use a PDM for analysis of the cortical sulci, Lorenz et al. [16] for the lumbar vertebrae, Styner et al. [17] for the femoral head and the lateral ventricles of the brain, and Frangi et al. [18] for analysis of the heart.
Active Shape Model
In 1995 Cootes et al. further developed the PDM into a versatile trainable method for shape modeling and matching in the form of the Active Shape Model (ASM) [19]. Thus, the PDM was extended with a matching algorithm that extracts, e.g., edge informa-tion from the target organ from an image. With this edge informainforma-tion the matching algorithm suggests new positions for the landmarks of the PDM. By projection of these new landmark positions on the shape sub space, spanned by the mean shape and the shape variations from the model, a new shape instance is created. By constraining the coordinates in the shape space to within certain statistical limits, the shape is forced to resemble the shapes seen in the training set.
Active Shape Models have found widespread application in 2D. Among others, Cootes et al. used ASMs for face recognition [19,20], Van Ginneken et al. [21] used an ASM with optimal features detection to chest radiographs. Li et al. [22] have used steer-able filters to extract local image edge features from which a small number of critical features is selected to classify image regions as edge or non-edge. They applied their method to the corpus callosum in a multi-resolution approach. Hamarneh et al. ex-tended ASMs to the spatio-temporal domain [23], and applied their (2D+time) ASM to echo cardiographic image data of the left ventricle and to synthetic images. In 3D, ASMs have been explored by Dickens et al. [24,25] on synthetic kidney data and Kaus et al. [26] in an application to cardiac image segmentation problems.
Active Appearance Model
1.2 Automatic segmentation 9
iterative matching process and have proven to be very powerful and robust tools for medical image segmentation.
As a consequence, AAMs too find widespread application. In 2D, Cootes et al. used their AAM for knees from MRI images and matching faces [27] and for face recog-nition [28]. Mitchell et al. developed a hybrid 2D AAM for simultaneous application to cardiac MRI and Ultrasound [29]. Lelieveldt et al. developed a multi-view or 2.5D AAM for application to multiple MRI views of the cardiac left ventricle [30]. Mitchell et al. developed the first 3D AAM, in an application to a cardiac left ventricle data set acquired with MRI [31]. Also, L¨otj¨onen et al. [32] and Stegmann et al. [33] developed 3D AAMs dedicated to the heart, the latter of which is bi-temporal.
In 2002, Bosch et al. developed an Active Appearance Motion Model [34], a 2D+time AAM, for segmentation of echo cardiographic image sequences.
Alternative approaches
In recent years, much more work on knowledge- and model driven statistics-based seg-mentation has been described. Alternative approaches to (statistical) shape modeling for shape extraction or image segmentation developed in the last decade are:
• Spherical harmonics
Kelemen et al. [35] propose a 3D model that uses a parametric shape representa-tion instead of a PDM. Using spherical harmonics, they automatically generate surface meshes with homogeneous node distributions. They calculate eigenvari-ations in the shape parameter space, whereas Cootes et al. [19] derive shape statistics from sample point coordinates. Matching performance was tested with a left hippocampus model; results were compared to manual segmentation. The same model by Kelemen et al. was also used for segmentation of the amygdala hippocampal complex by Shenton et al. [36].
• Constrained level sets
Tsai et al. [37] introduced a method for segmentation of medical imagery by incorporating a-priori knowledge into the level set framework. They derived a model-based curve evolution technique and applied it in 3D to a prostate gland segmentation example from pelvic MRI. Like Kelemen et al. , Tsai et al. do not derive shape statistics from sample point coordinates, but use the signed dis-tance function as a shape representation, and extract modes of shape variation by applying PCA to those distance functions.
• Statistical deformation models
Rueckert et al. [38] present an algorithm for the construction of 3D anatomi-cal models of the brain from MR images. They use a nonrigid registration al-gorithm based on free-form deformations and normalized mutual information. Thus, dense correspondences between subjects within a target population are derived from the deformations between them. These statistical deformation models have been applied to cardiac modeling by, e.g., Chandrashekara et al.[39] and L¨otj¨onen et al. [32], who recently presented a statistical deformation model built from combined long-axis and short-axis views of the cardiac ventricles and atria.
• Constructive Solid Geometry (CSG)
10 1.3 Motivation of this work
proposed by Ricci [40] for computer graphics and later applied to chest image segmentation by Lelieveldt et al. [41]. Danilouckine et al. used ISM for auto-matic planning of cardiac MRI acquisitions [42]. Models created for ISM usually consist of organ primitives described by parametric functions, and binary oper-ations like union, subtraction and intersection performed on the primitives. • Medial representations
Medial representations or m-reps [43,44] model object surfaces by a grid of dial atoms described by a position, a width (radius), a vector tangent to the me-dial axis and an object angle. At object boundaries an additional scalar parame-ter deparame-termines the shape of the atom from flat to circular to elongated [45]. Me-dial representations have also been explored in combination with shape statis-tics by Joshi et al. [45]. They derive features based on geometrical properties that represent growth or bending, on scale, and on location. These features are unit free and scale invariant, because they describe ratios between object-related distances. Thus the need for alignment is avoided, allowing for more localized statistical modes of variation to emerge.
1.3
Motivation of this work
Inspired by the work of Cootes et al. many groups have directed their research to-wards statistical model-based segmentation approaches for medical applications. De-formable statistical models have proven to be highly useful in medical image analysis. Especially with respect to segmentation tasks, they can be very effective and robust, and therefore they form an active field of research. The Point Distribution Model (PDM) and Active Shape Model (ASM) developed by the seminal work of Cootes et al. are the cornerstone of the algorithms presented in this thesis.
Despite the large amount of developments with respect to PDMs, ASMs and also AAMs, methods presented thus far have a number of limitations that were solved in this work. The classic ASMs and AAMs are limited to application to the same class of data that was used for the extraction of the statistics incorporated in the model. This data congruence requirement demands application of the method to equally (densely) sampled data, i.e., similarly structured image data should be acquired at the same locations of the target organ, using the same modality, the same scanner model, and the same protocol. More sparsely/densely sampled data can not be handled by such ASMs, and the modality dependence requires training of a new model.
By removing the statistical update generation scheme, the gray value constraints im-posed by the statistical gray level models currently present in the ASMs/AAMs can be released. Consequently, an alternative robust edge detection/update generation method should be developed.
The primary motivation of this work was to develop a segmentation method for the 3D cardiac left ventricle that:
• treats segmentation in an intrinsically three-dimensional manner and exploits 3D spatial continuity of the cardiac (sub)shapes,
1.4 Structure of this thesis 11
(a) (b) (c)
Figure 1.3: (a) Radially oriented cardiac image stack, (b) a combined long-axis and short-axis data set, and (c) a short-axis image stack (only showing every second image).
• can segment image data irrespective of the orientation of its constituting image slices,
• can segment a data set using only a few images. In combination with the previ-ous point, the method should be able to segment sparse image data with differ-ent oridiffer-entations, e.g., a four-chamber view, a two-chamber view and one or two short-axis views, a radially oriented long-axis stack, or a short-axis stack (see Fig.1.3). This sparse data matching enables LV function analysis without the necessity of acquiring a large number of image slices and is a major reason for us to choose the ASM approach for this segmentation task.
In this work, the focus lies on developing a segmentation method that satisfies all points above for two imaging modalities: MRI and MSCT. The method should also be applicable to multiple MRI acquisition protocols. As a consequence of the desired modality/protocol independence, common knowledge with respect to the relative or-dering of gray values in the surroundings of the heart will be included in the ap-proaches presented in this thesis, instead of including statistical gray level knowledge as present in the classic ASMs and AAMs. Furthermore, this thesis focuses completely on the cardiac left ventricle, because the majority of quantitative indices for cardiac function are derived from LV contours. However, although most of the proposed ideas were validated in application to cardiac imaging, the methods are easily extendable to other fields of computer vision.
1.4
Structure of this thesis
This thesis is further structured as follows.
12 1.4 Structure of this thesis
independence it includes only shape knowledge, and to update model position, orien-tation and shape basic edge filtering is employed. As a proof-of-concept of modality independence in combination with an ASM, this first model trained from MRI-data is evaluated on a medium sized MSCT patient data set.
In Chapter3the edge filters are replaced by a Fuzzy Inference System (FIS) to infer model updates in a more robust manner. The basis of the new matching algorithm is the classification of pixels into three tissue classes. Image patches from the sur-roundings of the model are collectively classified using Fuzzy C-means (FCM) cluster-ing. Thus, a segmentation performance improvement is achieved while maintaining modality independence. The improvement (on MSCT data) due to this development is evaluated mainly visually, although volume errors are compared quantitatively with those from Chapter2.
In Chapter4the FIS is explained more thoroughly and is applied for the first time to both MSCT data and MRI data, without any differences in the training of the model. Only parameters in the segmentation part of the model have to be tuned to either of the modalities. For weakly defined local boundaries, edge locations are estimated by means of an interpolation scheme that uses edge location information from the closest reliable outcomes from the fuzzy inference segmentation algorithm.
In this chapter, the modality independent 3D-ASM is extensively evaluated on a large MSCT data set and a medium sized MRI data set. MSCT data from different manu-facturers was used. Evaluation of the segmentation results is performed by means of point-to-point error assessment, volume regression analysis and Bland-Altman anal-ysis all with respect to manual segmentation, which serves as the gold standard. In Chapter5 a new shape model is constructed from a far bigger training data set acquired with MRI. The model construction differs from that in previous chapters in that it is achieved by building an atlas using non-rigid registration. Point correspon-dence is determined via automatic landmarking of the atlas and propagation of the landmarks to the individual shape samples.
To overcome intensity inhomogeneity commonly observed in cardiac MR data, the model is subdivided in multiple sectors. Different sectors can be assigned different rule sets in the FIS, and with respect to the FCM operations different sectors can be combined. Since intensity inhomogeneity within one sector is limited, effects of in-homogeneity on the fuzzy clustering outcome is thus diminished. Furthermore, the model from this chapter includes a closed apex for both the endocardial and epicardial surfaces, whereas in previous chapters the apex was not defined. Another improve-ment made in this chapter is a compensation for differences in respiration level be-tween different acquisitions: the heart can shift within an image with respect to its position in other images in the same acquisition. This compensation is applied during the image matching process. Such respiration level artifacts are commonly corrected using image registration [46,47,48]. However, since registration is not the topic of this thesis, two simple registration algorithms inspired by Moore et al. [49] and based on geometric assumptions are used in a pre-processing step to the application of the 3D-ASM.
1.4 Structure of this thesis 13
In Chapter6segmentation performances are compared for three models: a 3D-ASM constructed with the point correspondence and alignment method used in Chapters2,
3and4with a highly regular mesh, the model from Chapter5, and an intermediate model. Both bi-ventricle models and single-ventricle models are built, yet only the single-ventricle models are applied to real data and extensively evaluated. Settings for all models are optimized through grid computing methods. Evaluation is performed on the same evaluation database used in Chapter5and also on the ED and the ES phase.
Chapter7presents SPASM (SParse data ASM), a newly developed method for prop-agation of update information over the model surfaces, built into the automatically landmarked model of Chapters 5 and 6. In sparse image data sets regions on the model surfaces lack model updates, turning calculation of the shape parameter vector into an ill-posed problem. The solution in SPASM is realized by a propagation scheme employing Gaussian weighting related to the distance between update source and re-ceiving nodes. The method is evaluated on a medium sized MRI data set from which different data configurations are constructed, with different image orientations (SA, LA, radial LA) and different data sparsity. This chapter shows the full potential of the model, application to arbitrarily oriented data with varying sparsity, and thus large undersampled regions.
In Chapter8the limit with respect to data sparsity is sought. The model is applied to similar MRI data sets used in Chapter7, but even sparser data sets are constructed from these. This chapter shows how many image slices are minimally required for SPASM to achieve a segmentation performance that does not differ significantly to that achieved on a full MRI data set, either SA, LA radial, or multi-view (i.e., a com-bination of SA and LA).
Hamlet: ’Do you see yonder cloud that’s almost in shape of a camel?’ Lord Polonius: ’By the mass, and ’tis like a camel, indeed.’ Hamlet: ’Methinks it is like a weasel.’ Lord Polonius: ’It is backed like a weasel.’ Hamlet: ’Or like a whale?’ Lord Polonius: ’Very like a whale.’
William Shakespeare (1564–1616), in Hamlet
2
3D-Active Shape Model Matching for Left
Ventricle Segmentation in Cardiac CT
Copyright c 2003 Society of Photo-Optical Instrumentation Engineers. Reprinted from:
3D Active Shape Model Matching for Left Ventricle Segmentation in Cardiac CT
H.C. van Assen, R.J. van der Geest, M.G. Danilouchkine, H.J. Lamb, J.H.C. Reiber, and B.P.F. Lelieveldt In: Proceedings of the SPIE 2003, M. Sonka and J.M. Fitzpatrick, Eds., vol. 5032 [Medical Imaging 2003: Image Processing], 2003: 384-393
16 2.1 Introduction
Abstract
Manual quantitative analysis of cardiac left ventricular function using multi-slice CT is labor intensive because of the large data sets. We present an automatic, robust and intrinsically three-dimensional segmentation method for cardiac CT images, based on 3D-Active Shape Models (3D-ASMs).
ASMs describe shape and shape variations over a population as a mean shape and a number of eigenvariations, which can be extracted by e.g. Principal Component Analysis (PCA). During the iterative ASM matching process, the shape deformation is restricted within statistically plausible constraints (3σ). Our approach has two novel aspects: the 3D-ASM application to volume data of arbitrary planar orientation, and the application to image data from another modality than which was used to train the model, without the necessity of retraining it.
The 3D-ASM was trained on MR data and quantitatively evaluated on 17 multi-slice cardiac CT data sets, with respect to calculated LV volume (blood pool plus myocar-dium) and endocardial volume. In all cases, model matching was convergent and final results showed a good model performance. Bland-Altman analysis however, showed that blood pool volume was slightly underestimated and LV volume was slightly over-estimated by the model. Nevertheless, these errors remain within clinically acceptable margins.
Based on this evaluation, we conclude that our 3D-ASM combines robustness with clinically acceptable accuracy. Without retraining for cardiac CT, we could adapt a model trained on cardiac MR data sets for application in cardiac CT volumes, demon-strating the flexibility and feasibility of our matching approach. Causes for the sys-tematic errors are edge detection, model constraints, or image data reconstruction. For all these categories, solutions are discussed.
2.1
Introduction
In medical image analysis and segmentation, deformable statistical models have proven to be highly successful and form an important field of research [50,51]. Their success mainly stems from the a-priori knowledge contained in these models, ranging from shape and shape variation knowledge to gray level information.
In 1992, Cootes et al. introduced the Point Distribution Model as a description of the characteristic shape and shape variations of a set of example shapes [11]. In 1995 Cootes et al. further developed the PDM into a versatile trainable method for shape modeling and matching in the form of the Active Shape Model [19]. Active Shape Mod-els model the characteristic shape and shape variations over a population of example shapes, and can be applied to image segmentation to limit the segmentation result to ”statistically plausible” shape instances. In 1998, Cootes presented an extension of the ASM in the form of an Active Appearance Model, which is an extension with a statistical model of an intensity patch [27]. Both Active Shape Models and Active Appearance Models have found widespread application in 2D [11,21,22,27,29] and 2D + time [52] medical image segmentation problems.
2.1 Introduction 17
have been explored in 3D [15,16,53,54], mainly for shape analysis of complex 3D shapes. Nevertheless, application of 3D Point Distribution Models to image segmen-tation is still largely unexplored territory.
The primary motivation of this work was to develop a segmentation method for the 3D cardiac ventricle that:
• treats segmentation in an intrinsically three-dimensional manner and exploits 3D continuity of LV shape,
• is applicable to various types of 3D cardiac image data, either from different modalities or acquisition protocols without retraining of the statistical model, • can segment an LV data set using only a few images with different
orienta-tions, e.g., a four-chamber view, a two-chamber view and a short-axis view (see Fig.2.1). This sparse data matching enables LV function analysis without the necessity of acquiring a large number of image slices and is a major reason for us to choose the ASM approach for this segmentation task.
In this paper, we present a 3D extension of the Active Shape Models that meets the requirements mentioned above. We focus on the development of a novel matching mechanism for 3D Point Distribution Models, with two important properties:
• the left ventricle shape is represented in 3D by mapping the 3D PDM to a mesh structure consisting of two surfaces, which can be intersected with planes of arbitrary orientation. These planes mimic image slices from which edge infor-mation is extracted to drive the model matching. By mapping these 2D position changes from the image slices to the 3D mesh points, the 3D shape and pose parameters are updated,
• the mesh used to represent the 3D model can be intersected by planes of any ori-entation, as mentioned in the first property above. The statistical model itself contains only shape information and is therefore modality independent. The edge detection technique applied to extract edge information from the 2D im-age slices is also independent of imim-age modality. Therefore, the 3D-ASM is ap-plicable to image data from different modalities and from different acquisition protocols.
The 3D-ASM was quantitatively evaluated with stacks of parallel short-axis images. In the same evaluation, we illustrate the model’s independence of image modality with the direct application of the 3D-ASM, which was trained on cardiac MR data, to seg-mentation of multi-slice cardiac CT images. Moreover, the model was qualitatively tested on axial CT image stacks to demonstrate the validity of the concept of segment-ing left-ventricular image data of arbitrary planar orientation.
18 2.2 Methodology
Figure 2.1: Shape model with a stack of parallel image slices (left) and the same model with 3 image slices with arbitrary but different orientations (right).
2.2
Methodology
2.2.1 Model generation
Model generation in 3D involves three important issues, which are discussed in this section. First, point correspondence for 3-dimensional cardiac left-ventricular (LV) shapes is discussed. Second, shape alignment in 3D is addressed, and finally, statisti-cal modeling of 3D shape and shape variation is discussed.
Point correspondence in the 3D cardiac left ventricle
An important condition for generating an Active Shape Model is the definition of point correspondence between shapes. Together with alignment, point correspondence is one of the most important problems of the extension of the ASM from 2D to 3D. Methods for defining generic point-correspondences between shapes currently form an active field of research [53,55,56]. However, because of the relatively simple shape of the 3D cardiac left ventricle, we adopted an application specific definition for point corre-spondence in the left ventricle.
Figure2.2(a)shows the shape of a left ventricular endo- and epicardial surfaces and the parameterization that was applied. To define the parameterization representing a particular shape instance, each contour is sampled at equidistant angles with respect to the LV long axis. The sampling in each slice starts at the posterior junction of the left and right ventricle, which had been indicated by an expert. The coordinates of the samples of the endo- and epicardial contours in the stack of image slices together form one shape vector. The endocardial contours are sampled counter clockwise and from the apical to the basal slice, filling half the shape vector. The other half of the vector is filled by sampling the epicardial contours counter clockwise, from the basal slice towards the apical slice. This yields a line-parameterization with a specific point-to-point ordering.
ex-2.2 Methodology 19
(a) (b)
Figure 2.2: (a) Line parameterization, defining the point correspondence for the 3D cardiac left ventricle. (b) 3D triangular mesh constructed from the PDM. Each point is connected to its corresponding point in the next slice and to the previous point in the next slice, thus creating triangles.
pressed as a 3n element vector x containing n concatenated 3-dimensional landmark points{(xi, yi, zi)} of a particular shape.
Statistical shape modeling
In order to minimize the effect of trivial variations in pose and scale in the statis-tical shape model, shape alignment is required. Shape samples are aligned using Procrustes analysis [57,58], where all shapes in the training set are aligned using an iterative least-squares distance minimization. For the registration of correspond-ing point sets, we adopted Besl and McKay’s iterative closest points (ICP) algorithm for registration of 3D points sets [59]. This algorithm uses quaternions to represent scaling and rotation, assuming that 3D translation is represented in a separate pose vector.
After Procrustes alignment, the residual variation in shape in the training data set is modeled. The aligned set of shapes is transformed onto a basis of eigenvectors de-scribing the shape variation in the set. The eigenvectors are calculated by applying a Principal Component Analysis (PCA) on the covariance matrix of the training set. A point in the eigenspace is a linear combination of eigenvectors representing a partic-ular shape x in 3D, while the origin in the eigenspace corresponds with the average shape from the training set. Any shape synthesized from the model can be written as
x = x + Φb. (2.1)
with Φ a matrix containing the m eigenvectors and b a m-dimensional (m ≤ N ) pa-rameter vector controlling the deformation of the model.
20 2.2 Methodology
the model should represent. If pv is the required proportion of total variance Vtotal
present in the training set, the number of modes k can be determined by
k
X
i=1
λi≥ pvVtotal (2.2)
2.2.2 Matching Algorithm
To match the 3D PDM to image data, a 3D triangular mesh from the points in the PDM was constructed by connecting neighboring points within one slice. To form tri-angles, each point is connected by an edge to the corresponding point in the next slice and to the point previous to the corresponding point in the next slice (see Fig. 2.2(b)). The proposed matching procedure is based on distance minimization between two
Figure 2.3: Schematic representation of the matching algorithm used in the 3D-ASM.
Figure 2.4: Different possible intersection directions for the LV model. Left, a short-axis intersection and right, an axial intersection.
2.2 Methodology 21
the triangle mesh. The second point cloud represents the target shape, to which the model is fitted within the statistical deformation constraints to guarantee plausible left-ventricular shapes. The matching consists of the following steps (see Fig.2.3):
1. Mesh intersection. From the current state of the mesh, model sample points in the individual image planes are derived by dividing the mesh into two parts by the image plane. Each point (vertex) in the mesh is labeled to either side of the image plane. The in-plane contour points are generated by intersecting the mesh edges with differently labeled end points with the image plane. Endo-and epicardial sample points are generated simultaneously. Figure2.4shows two possible directions for intersection, a short-axis intersection and an axial intersection.
2. From each 2D intersection contour, a set of evenly distributed sample points is generated. For this, the center of gravity of the contour coordinates is calculated. Using this point as the center of rotation, the 2D contour is sampled with equal angle intervals, i.e. a fixed number of points per contour is generated.
3. For each sample point, generate a candidate boundary position. The candidate boundary position is estimated by maximizing cross correlation between a fil-ter template (step filfil-ter) and a sampled line profile perpendicular to the con-tour. This yields a 2D displacement vector for each of the sample points (see Fig.2.5(a)).
4. Projection of candidate boundary points on the mesh vertices. The candidate boundary points on the contours in the image slices do not coincide with the mesh points. Therefore, the point displacements in the image slices are prop-agated to the mesh vertices spanning the intersected triangle edge. Typically, displacements of more than one sample point contribute to the displacement of a mesh point. Multiple edges above and below any vertex can be intersected by image slices.
5. Align the in-plane displacement vector of each mesh point to the average 3D normal vector of the neighboring triangles in the mesh.
6. Align the current state to the proposed state of the mesh, using the ICP align-ment used during model generation [59].
7. Calculate shape differences between the aligned states of the mesh, and succes-sively adjust the parameter vector b controlling the model deformation:
b = ΦT(xproposed− xcurrent) (2.3)
with xproposedthe vector representing the proposed shape, and xcurrentthe
vec-tor representing the aligned current state of the mesh. Thus, the mesh in the current state is deformed to optimally match the new shape. This deformation is constrained within the bounds of the statistical description, i.e. within a fixed number of standard deviations from the average (non-deformed) model.
Repeat until convergence.
22 2.3 Experimental setup
generate a candidate position for each point in the mesh. A simple step filter profile of nine pixels served as a filter template. To find the epicardial wall at the septal wall, a rising edge filter template was used, whereas all other regions, a descending edge filter was employed. The length of the sampled in-plane profiles was fixed to 21 pixels. Matching was performed in two steps: a Euclidian transformation only in the first few iterations, followed by simultaneous shape and pose adaptation. This was motivated by the observation that the model is not yet close to its final position if it still undergoes large pose changes. This way, the risk of convergence to neighboring structures can be decreased. In case no clearly defined edge could be detected along a sampled in-plane profile, the new position of the sample point is estimated by interpolation between reliable neighboring edge results.
2.3
Experimental setup
2.3.1 Training data
To assess the clinical potential of the presented ASM matching, an ASM was generated using expert drawn epicardial and endocardial contours of the cardiac left ventricle of a group of 53 patients and normals, from 3D MR data. The basal and the apical slices of the ventricle in the model were selected as the last slice in which still both an endo-and epicardial contours were drawn by the expert. Images were 256x256 pixels; field of view: 400-450 mm; pixel sizes: 1.56-1.75 mm.
To define point correspondence between the training samples, the parameterization presented in Section 2. was applied, where each sample was divided in 16 slices, each containing 32 points for the epicardial contour and 32 points for the endocardial contour. This resulted in 1024 points for the model.
In order to reduce model dimensionality, the model was restricted to represent 99% of the shape variation present in the training data, resulting in 33 modes for statistical shape deformation description.
2.3.2 Evaluation data
The 3D-ASM performance was tested on 17 CT acquisitions of left ventricles. The number of slices in these acquisitions ranges from 15 to 39 of which between 12 and 24 contain left ventricular data. Of the 17 data sets that were used, six were acquired using a Toshiba Acquilion 4-slice CT-scanner, and had an axial slice thickness of ap-proximately 1 mm and an in-plane resolution of 0.5 mm/pixel. Eleven data sets were acquired using GE Lightspeed 4-slice CT scanners with a slice thickness of 1.25 mm and an in-plane resolution of approximately 0.5 mm/pixel. All data sets were refor-matted to yield short-axis image slices. No normals were included for evaluation; all data sets included in the evaluation population were acquired from patients.
2.3.3 Model matching parameters
2.3 Experimental setup 23
evaluation.
Throughout most of the model surface areas, the step edge filters for the endocardial surface were defined the same as those for the epicardial surface: a descending edge when traversing the surface from inside to outside. Only at the septal wall, the epicar-dial edge filters were defined with opposite edge transitions. To avoid epicarepicar-dial edges in the image attracting the endocardial surface of the model and vice versa, a-priori knowledge about the gray levels of the several tissues was utilized to reject implausi-ble edge candidates: the average gray value of the blood pool was required to be higher than a certain Houndsfield threshold. The average gray value of the lung was required to be lower than another certain Houndsfield unit.
During the model matching to the patient data, model deformation was limited by con-straining each component of the model deformation parameter vector between -3s and +3s. The ASM as described here did not have a stop criterion indicating that conver-gence had been achieved. During evaluation, the model search ran for a fixed number of iterations. The model state in the last iteration was included in a quantitative com-parison with manually drawn expert contours.
2.3.4 Quantitative evaluation
To quantitatively evaluate the model, volumes from the endocardial and epicardial contours were calculated using Simpson’s rule. Areas were calculated enclosed by the contours that resulted from the model in mm2and summed these over the slices
included in the evaluation. Slices were included in the evaluation if the model was able to produce a full contour for that slice. Slices for which the model produced only partial contours (see Fig.2.5(a), bottom-center) were excluded from the quantitative evaluation procedure. From the calculated automatic and manual epicardial volumes (LV volume) and endocardial volumes (blood pool volume) regression formulas were calculated. Bland-Altman analysis of the LV (i.e. blood pool plus myocardium) volume and the blood pool volume was performed to determine whether segmentation results of the model show a systematic error.
(a) (b)
24 2.4 Results
(a) (b)
Figure 2.6: (a) Bland-Altman plot of the ED LV volume calculated from automatically detected contours compared with LV volume resulting from hand drawn contours. This shows a systematic over-estimation by the model. The dashed lines indicate the mean difference, the mean difference +2SD and the mean difference -2SD. (b) Bland-Altman plot of the ED blood pool volume calculated from automatically detected contours com-pared to blood pool volume resulting from hand drawn contours. This shows a system-atic under-estimation by the model. The dashed lines indicate the mean difference, the mean difference +2SD and the mean difference -2SD.
2.4
Results
In all 17 cases, the matching procedure resulted in visually plausible contours. No match was excluded from quantitative evaluation, according to the exclusion criterion mentioned above. Figure 2.5 shows representative examples of plausible contours resulting from the 3D-ASM in short-axis and axial data sets respectively. From the calculated automatic and manual epicardial volumes (LV volume) and endocardial vol-umes (blood pool volume) regression formulas were calculated. For epicardial volume an excellent correlation was found: y = 1.02x + 16.4(R = 0.99), with y denoting the LV volume calculated from automatic contours and x the LV volume calculated from expert contours. For the endocardial volume, we found the same excellent correlation factor: y = 0.88x + 1.4(R = 0.99), with y the endocardial volume calculated from au-tomatically generated contours, and x the blood pool volume resulting from manually drawn contours. Bland-Altman plots for endo- and epicardial volume are shown in Figure2.6. The regression formulas and the Bland-Altman plots reveal that there is a slight systematic underestimation of the blood pool volume and a slight systematic overestimation of the epicardial volume by the model.
2.5
Discussion and conclusions
2.5 Discussion and conclusions 25
(a) (b)
Figure 2.7: (a) An example where model limitations prohibit the model to deform according to the edge-information in the image (arrow). Edge-detection resulted in good candidate points, but the model surface was unable to reproduce local myocardial thinning. (b) Contours that the same model generated for a mid-ventricular slice of the same study are shown. This shows that this model shape limitation can be a local issue.
axial data sets.
Quantitative evaluation from the short-axis data sets however, shows that ED LV volume (blood pool plus myocardium) is systematically overestimated by the model, whereas ED blood pool is systematically underestimated (see Fig.2.6). This can also be seen from the regression formulas presented for both volumes. One of the reasons for this may be the lack of apex data in the model. The apex is a well recognizable landmark of the cardiac left ventricle, and therefore including it in the statistical LV shape description may result in a more representative model. Missing the apex in the model is a cause for uncertainty about the position of the model in the direction of the LV long axis.
In some cases however, generation of an accurate segmentation failed. This can have a number of causes, which can be divided in three categories:
• image data
In three data sets, image data was truncated close to the LV frontal side due to reformatting. The truncation itself showed to be a very strong edge, attracting and thus misleading the model epicardial surface. In the short-axis test set of another patient, expected 3D continuity of the data set was not present, because of an in-plane shift of one slice with respect to the rest of the stack. If a stack of images has slices that are shifted in an in-plane direction the model may not be able to follow the surfaces. The 3D model requires 3D continuity between slices in the image data set and thus provides 3D continuity in the contours that result from the 3D segmentation process. To resolve these issues, image reformatting should be performed with care.
26 2.5 Discussion and conclusions
In one patient, the myocardium showed drastic local thinning, that the model was unable to reproduce (see Fig.2.7). The deformation proposed as a result of the edge-detection could not be accommodated by model. This may be caused by a lack of representative sample shapes in the training data set. Secondly, because of missing apical data in the model, uncertainty in the position of the model (closer to the base, or more towards the apex) arises. If the model is positioned too close to the base or apex, local deformations of the model will not match local shape characteristics any more. Incorporation of apical data in the model and extending the training data set with additional patient data should solve these issues.
• edge-detection
Edge-detection yielding candidate edge positions at transitions between fat and air instead of transitions between myocardium and epicardial fat, causes a sys-tematic overestimation of LV volume. Moreover, uncertainty about the best val-ues to choose for the average blood pool gray value and the lung gray value influ-ences edge-detection accuracy. This is a common tuning problem. And thirdly, in 3 patients, an edge between epicardial fat and lung (air) attracted the model epi-cardial surface more than the edge between myocardium and epiepi-cardial fat. The model deformed to fit the epicardial fat, thus yielding overestimation of the to-tal LV volume. Therefore, for the edge-detection algorithm, we suggest adding a preprocessing step that distinguishes between tissue types before the 3D-ASM is applied. This would also support the objective of a modality independent model. The ultimate goal of this work is the application of the model to a sparse data set, i.e., a small number of image slices that are not necessarily parallel, as shown in Figure2.1. That however, requires a weighing scheme while updating the model’s pose and shape parameters, since the sparse data set will cause a large amount of model points not to be updated. Only sample points close to model intersections by the image slices are updated with the help of edge information. Other sample points are not updated in between iterations, which means that they tend to retain their position. These points will have to be discarded while updating the model parameters, i.e. they have to be assigned a weight equal to zero.
Once this weighing scheme is implemented, it can also be used to assign weights to sample points according to the strength of the edges that attract these points. Thus, a strong edge has more influence than a weak edge, and points that have no edge in their vicinity at all can be assigned a weight zero, as with the sample points missing image data in the sparse data application.
In the current evaluation, the model was initialized manually. In the future, we want to further automate the model matching as much as possible, so initialization should also be completely automatic. Due to this automation, the risk of initialization far-ther from the final segmentation result will be increased. In order to reduce model dependency of the initialization, the model can be allowed to broaden its view in the initial matching process, and reducing its view when the matching process converges towards its final result.
’There is nothing worse than a sharp image of a fuzzy concept.’
Ansel Adams (1902–1984)
3
Cardiac LV Segmentation Using a 3D Active
Shape Model Driven by Fuzzy Inference
Copyright c 2003 Springer-Verlag Berlin Heidelberg. Reprinted from:
Cardiac LV Segmentation Using a 3D Active Shape Model Driven by Fuzzy Inference
H.C. van Assen, M.G. Danilouchkine, F. Behloul, H.J. Lamb, R.J. van der Geest, J.H.C. Reiber, and B.P.F. Lelieveldt In: MICCAI 2003, Lecture Notes in Computer Science, R.E. Ellis and T.M. Peters, Eds., vol. 2878. Berlin: Springer Verlag, 2003: 533-540
28 3.1 Introduction
Abstract
Manual quantitative analysis of cardiac left ventricular function using multi-slice CT is labor intensive because of the large datasets. In previous work, an intrinsically three-dimensional segmentation method for cardiac CT images was presented based on a 3D Active Shape Model (3D-ASM). This model systematically overestimated left ventricular volume and underestimated blood pool volume, due to inaccurate estima-tion of candidate points during the model update steps. In this paper, we propose a novel ASM candidate point generation method based on a Fuzzy Inference System (FIS), which uses image patches as an input. Visual and quantitative evaluation of the results for 7 out of 9 patients shows substantial improvement for endocardial con-tours, while the resulting volume errors decrease considerably (blood pool: −39 ± 29 cubic voxels in the previous model,−0.66 ± 6.2 cubic voxels in the current). Standard deviation of the epicardial volume decreases by approximately 50%.
3.1
Introduction
Deformable statistical models have proven to be highly useful in medical image analy-sis. Especially with respect to segmentation tasks, they can be very effective and there-fore they form an active field of research. The knowledge in the model ranges from shape and shape variation to gray level information. Since 1992, Cootes et al. intro-duced several statistical modeling methods, the Point Distribution Model (PDM) [11], the Active Shape Model (ASM) [19], which is an extension to the PDM with a match-ing algorithm, and the Active Appearance Model (AAM) [27]: an extension of the ASM using a statistical intensity model of image patches. PDMs have been explored in 3D [16,53], mainly for the analysis of complex 3D shapes. AAMs have also been de-veloped in 3D, and applied to medical image segmentation problems [31]. Applications for ASMs, however, have mainly been limited to 2D or 2D plus time [52,23].
In previous work, we developed a cross-modality 3D-ASM [60] for cardiac left ventri-cle (LV) segmentation of MR and CT images, while avoiding the necessity of intensity model retraining for each modality. We showed that the 3D-ASM is a promising tool for this purpose. However, due to the basic edge detection applied to define model up-dates, an underestimation of blood pool volume inside the LV and an overestimation of total LV-volume resulted. This was our primary motivation to design a method to more accurately localize the candidate boundary positions in the image slices. For this purpose, we chose to adopt fuzzy inference (FI) for two reasons:
• by applying a classification approach to image patches instead of pixel scan lines, sensitivity to local disturbances and noise can be greatly reduced,
3.1 Introduction 29
(a) (b)
Figure 3.1: (a) Line parameterization, defining the point correspondence for the 3D cardiac left ventricle (b) Mesh used for matching the model to the image information.
Figure 3.2: Schematic representation of the matching procedure used in this 3D-ASM. The focus of this paper is the generation of new candidate positions for the sample points that construct the mesh (indicated in gray).
30 3.2 Methodology
3.2
Methodology
3.2.1 3D model generation
To perform statistical analysis on 3D-shapes, three important issues were addressed: point correspondence for 3-dimensional cardiac LV shapes, 3D shape alignment, and statistical modeling of 3D shape and shape variation.
The definition of point correspondences is an active field of research. Apart from a manual definition of point correspondence, different automatic methods have been ex-plored. Styner et al. [17] presented an evaluation of four automatic point correspon-dence methods. This evaluation suggests the use of either DetCov or the MDL-based approach, however the DetCov method is not easily extendable to 3D. For our ASM however, we adopted an application specific solution and used a manually defined point correspondence [60] (see Fig.3.1).
After line-parameterization of every shape in the training set, the resulting point set is resampled in the direction of the long axis. This is done to generate sets of sample points, that represent the same number of slices, and consequently the same number of sample points. Each shape sample can then be expressed as a 3n element vector x containing n concatenated 3-dimensional landmark points (xi, yi,zi) of a particular shape. Both the LV endo- and epicardial surface are represented in the same vec-tor. Finally, the shape samples are aligned. The remaining differences are solely shape related, and thus the effect of trivial variations in pose and scale is eliminated. For alignment of the training samples, Procrustes analysis was applied. We adopted Besl and McKays iterative closest points (ICP) algorithm for registration of 3D points sets [59]. Shape modeling is analogous to the 2D case, except that 3D coordinates are used. To extract the modes of variation of the training set, the eigenvectors and covariance matrix are calculated using principal component analysis.
3.2.2 Model matching
The model was extended with a matching algorithm: an iterative procedure to deter-mine update steps according to information from the target image set. During each update, the shape parameters are adjusted to best match the image evidence. For the 3D-ASM of the cardiac LV, a 3D triangular mesh was constructed from the sam-ple points (see Fig.3.1). During the matching, this mesh is intersected by the image planes, thus generating 2D contours spanned by the intersections of the mesh tri-angles. Model update information is represented by 2D point-displacement vectors, extracted from the 2D image slices. These 2D vectors are propagated to the 3D mesh representing the complete model update. This propagation is achieved by alignment of the 2D in-plane displacement vectors to the 3D surface normal vector. Similar to the training stage, scaling, rotation and translation differences between the current state of the model and the point cloud representing the model update step are eliminated by alignment. Both mesh states are aligned using the ICP method [59]. Successively, the parameter vector b controlling model deformation is adjusted:
b = ΦT(xproposed− xcurrent). (3.1)
with xproposedthe vector representing the proposed shape, and xcurrentthe vector
