3D active shape modeling for cardiac MR and CT image
segmentation
Assen, Hans Christiaan van
Citation
Assen, H. C. van. (2006, May 10). 3D active shape modeling for cardiac MR
and CT image segmentation. Retrieved from https://hdl.handle.net/1887/4460
Version:
Corrected Publisher’s Version
License:
Licence agreement concerning inclusion of doctoral
thesis in the Institutional Repository of the University
of Leiden
Downloaded from:
https://hdl.handle.net/1887/4460
’Wheresoever you go, go with all your heart.’
Confucius (551–479 BC)
1.1
Background
1.1.1 Cardiac anatomy
The heart is the organ that maintains blood circulation through the body. It contains four cavities, the left and right atria, and the left and right ventricles (see Fig.1.1). In the heart the left and right side are separated, each supporting one of two different circulations. The systemic circulation is maintained by the left side and runs through the aorta, the organ tissues, the brain, and the extremities, and the pulmonary circu-lation is maintained by the right side of the heart and runs through the pulmonary artery and the lungs. To avoid backward flow of the blood, valves between the atria and the ventricles and between the ventricles and the outflow tracts exist that open and close at the right moment in the cardiac cycle.
The cardiac cycle itself consists of an electrically organized sequence of contractions of the atria and ventricles. Hypoxic blood enters the heart from the veins at the right atrium. From there it is injected into the right ventricle through the tricuspid valve by contraction of the atrium. The filling of the right ventricle is followed by its con-traction, causing the blood to flow through the pulmonary valve, the pulmonary artery and lungs, where it is oxygenated and releases carbon dioxide. The oxygenated blood returns to the heart at the left atrium. Together with the right atrium, the left atrium contracts injecting the blood through the mitral valve into the left ventricle. Shortly after, the left ventricle contracts, pumping the blood through the aortic valve into the aorta. The first branches in the aorta, and the only branches in the ascending aorta, are the coronary arteries. These supply the myocardium itself with oxygenated blood.
1.1.2 Heart disease
The cardiac contraction relies on a delicate balance of events, where the coronary circu-lation, electrical system, valves and myocardium all function in a coordinated manner. The major causes for cardiovascular disease (CVD) are:
• atherosclerosis, depositing plaque in vessels like, e.g., the renal arteries, the carotid arteries and the coronary arteries. Plaque is composed of fatty sub-stances, cholesterol, cellular waste products, calcium and other subsub-stances, and may cause stenosis, a local decrease in vessel diameter that leads to changes in blood flow. When plaques rupture, embolies (blood clots) can drift through the vascular system, with the risk of blocking arteries and/or veins (embolism). An embolism in the coronary arteries can lead to myocardial infarction.
• hypertension (elevated blood pressure), forcing the heart to pump against higher pressures. Causes for hypertension can be, among others, reduced kidney func-tion, renal artery stenosis, and stress. Hypertension can lead to hypertrophic obstructive cardiomyopathy (HOCM) and result in congestive heart failure. • congenital heart diseases, causing abnormal heart morphology. Congenital heart
1.1 Background 3
• valvular failure/dysfunction, i.e. valves that either don’t close or open normally causing either regurgitation or limited flow. A number of causes of valvular fail-ure are inflammation of the cardiac valves, valvular displacement, and valvular stenosis.
• arrythmia, caused by failure in the electrical system and resulting in unbalance in the sequence of contractions of the chambers. Arrythmia may occur when any portion of the propagation of the trigger signal given by the sino-atrial node is interrupted or disturbed. Normally this signal is transmitted from the right atrium, travels across the atria, through the septum to specialized tissues slow-ing down its progression and passslow-ing it on to the ventricles.
In the US, cardiovascular disease (CVD) kills more people every year than cancer, chronic lower respiratory diseases, accidents, diabetes mellitus, and influenza and pneumonia combined. Because CVD is the primary cause of death, its prevalence is monitored closely. In 2002, 38% of all deaths were caused by CVD. Of over 2,400,000 deaths from all causes, nearly 60% had CVD as primary or contributing cause. Since 1984, CVD has experienced higher prevalence in women than in men. From the 2005 update on heart disease and stroke statistics, on average every 34 seconds someone in the US dies of CVD [1]. CVD does not only affect old people, but also children. For children under age 15, CVD is the number 2 cause of death. In 2003, in the Nether-lands, 47,992 of in total 142,355 deaths were caused by cardiac and (cardio-)vascular diseases.
1.1.3 Diagnosis: cardiac imaging and quantification
Cardiac Left Ventricular (LV) function and mass are important prognostic factors in risk assessment and management of heart disease [2]. LV function can be divided in global function, regional function, function related to perfusion, to infarcted tissue vi-ability and to metabolism. In order to assess cardiac function, patients can undergo several tests. Cardiac function can be monitored by ElectroCardioGrams (ECGs), and measuring (systolic and diastolic) blood pressure. In addition, patients can be diag-nosed using cardiac imaging. Cardiac imaging can be performed with multiple imag-ing modalities, e.g., (3D) cardiac Ultrasound (US), cardiovascular cine-angiography (XA), Computed Tomography (CT, or MSCT: Multi-slice CT), Single Photon Emission Computed Tomography (SPECT), Positron Emission Tomography (PET), and Mag-netic Resonance Imaging (MRI). For typical images produced by these modalities, see Figure1.2. For advantages and disadvantages of the different modalities with respect to quantification of cardiac function see Table1.1.
Figure 1.1: The human heart. ( c Edwards Lifesciences Corporation, used with per-mission)
A recently proposed MRI technique is based on acquisition of the radially (RAD) ori-entated long-axis (LA) images, with the common intersection line coinciding with the long LV axis. This provides a clearer depiction of the basal part of the heart and can be alternatively utilized for the quantification of LV volume and mass [4]. However, more sparsely sampled regions exist in the mid-ventricular and basal endocardium and epicardium. A third alternative is the use of 2- and 4-chamber views for quantifi-cation. Because those views rely on geometric assumptions, quantification based on these views is less accurate.
1.1.4 Automation in diagnostic quantification
1.1 Background 5
Table 1.1: Imaging modalities, their properties with respect to quantification of car-diac function, and their applicability. ∗MI=Myocardial Infarction, CHD=Coronary Heart Disease, VD=Valvular Dysfunction, EmSc=Emergency Screening, HF=Heart Failure, GF=Global Function, RF=Regional Function, MP=Myocardial Perfusion, MV=Myocardial Viability, Met=Metabolism
Modality Advantages Disadvantages Pathology∗ Analysis∗
2D US fast acquisition, high temporal resolution
limited spatial resolu-tion, relies on geometric shape assumptions MI, CHD, HOCM, EmSc, VD GF, RF 3D US (rotational) 3D volumetric data sets [5]
suboptimal image qual-ity
MI, CHD, HOCM, EmSc, VD
GF, RF
(MS)CT fast acquisition, high spatial resolution, nearly isotropic voxel size
ionized radiation, ad-ministration of contrast agent (semi-invasive, toxic), moderate tem-poral resolution, axial image acquisition
MI, CHD, HOCM, HF
GF, RF, MV
MRI non-invasive, high spa-tial resolution, high temporal resolution, intrinsically high blood-myocardium contrast, arbitrary image orien-tation
prolonged examination times, breath holding is difficult for patients, low through-plane res-olution, reproducibility of quantitative results depends on the accuracy of the positioning of the image slices [6]
MI, Ischemia, HOCM, VD, HF
GF, RF, MV, MP
XA high spatial resolution, high temporal resolu-tion
ionized radiation, ad-ministration of contrast agent (invasive, toxic), low image quality for ES, strong dependence on geometrical models
MI, CHD GF, RF
PET natural way of perform-ing functional measure-ments: perfusion, func-tion, oxygenafunc-tion, pro-tein concentration
nuclear radiation, ad-ministration of nuclear tracer (semi-invasive), low spatial resolution
MI, Ischemia [7] MV [7], MP, Met
SPECT natural way of perform-ing functional mea-surements: perfusion function, oxygenation, protein concentration; investigations over longer time interval than PET possible, because radionuclides have a longer physical half-life than with PET
nuclear radiation, nuclear tracer (semi-invasive), lower spatial resolution than PET, lower temporal resolu-tion than PET, images hard to interpret
(a) (b) (c)
(d) (e) (f)
(g) (h) (i)
Figure 1.2: (a) 2D Cardiac Ultrasound (b) 3D Cardiac Ultrasound (c) X-Ray LV-angiography (d) Short-axis cardiac MRI view (e) Long-axis cardiac MRI view (f) Car-diac PET (g) Axial carCar-diac CT view (h) Short-axis carCar-diac CT view (with reconstruction artifacts) (i) Cardiac SPECT . (Image (b) courtesy Marco M. Voormolen MSc, Erasmus MC)
that additional capacity be built for research and development and that institutional frameworks that facilitate cardiovascular disease prevention and control be developed. Due to the increasing prevalence of cardiac and cardiovascular diseases, and if the recommendations of the IOM are supported and executed, an increasing number of diagnostic assessments and interventions will be performed in the future. Already, the amount of diagnostic assessments is increasing rapidly, and as a consequence an increasing number of imaging operations is ordered. Moreover, for the purpose of com-prehensive diagnostic analysis, multiple types of assessments are performed which in-clude: functional, anatomical, perfusion, and rest-stress imaging. This also increases the amount of acquired data drastically.
1.2 Automatic segmentation 7
developments increase the production of diagnostic image data that will have to be analyzed. Consequently, it is impossible to cope with the vast amount of image data by manual interaction and visual perception, and the combination of many different assessments per patient make interpretation a difficult task. This can lead to different analysis results between different interpreting physicians, i.e. a large interobserver variability results. Also, large intra-observer variations are observed.
It is therefore an obvious conclusion that assessment and diagnosis based on imaging data has to be automated to the maximal extent possible, minimizing time invested by the physicians and increasing analysis robustness and consistency.
1.2
Automatic segmentation
For quantification in (medical) imagery, image segmentation is required. Medical im-age segmentation in particular is a very difficult problem. Spurious edges, fuzzy edges, absent edges, low signal-to-noise ratios, (reconstruction) artifacts, image misregistra-tion, and missing data, are only a few of the problems that appear in medical image segmentation. Nevertheless, medical image segmentation has been a very active field of research for many years.
1.2.1 Knowledge-based solutions
To overcome many problems mentioned above, knowledge-based segmentation ap-proaches have been proposed. Inclusion into the segmentation framework of a pri-ori knowledge has shown to be instrumental for robust performance. This can be knowledge involving, e.g., (ordering of) gray values among tissues, organ shape related knowledge, anatomical and geometrical related knowledge. Examples of such knowl-edge include: blood appears brighter than muscle, which in turn appears brighter than air in bright blood MR and MSCT images, and, the heart is located between the lungs and the diaphragm.
Knowledge extracted from imagery in general and medical imagery in particular, is not inherently related to features immediately visible in images. A particular form of knowledge inclusion is that related to shape characteristics, that captures prior knowledge about the typical shape and spatial context of an organ.
1.2.2 Statistical shape modeling
A generalized method of shape characterization became available with the develop-ment of statistical shape models, capable of capturing statistical shape and gray level information from sets of typical examples. Statistical shape modeling has proven to be one of the most influential developments in knowledge-driven image analysis in the last decade.
Point Distribution Model
into a training framework. This shape knowledge was expressed in coordinates of landmark points being commonly present and easily identifiable locations in the im-ages. A mean shape and a number of characteristic shape variations with respect to this mean shape are calculated after alignment of the shapes. With these, within certain statistical limits, shapes resembling those from the training set can be (re-) constructed by the mean shape and a linear combination of the shape variations. Cootes’ work inspired the further development of PDMs and ASMs. PDMs have been explored extensively in 2D and 3D. In 2D, Suinesiaputra et al. [12,13] uses a PDM with deformation modes determined by Independent Component Analysis (ICA) [14] for extraction of contractility patterns of the cardiac left ventricle and for detection of abnormal cardiac contraction regions. In 3D, PDMs are mainly used for morphomet-rics of complex shapes. For instance, Caunce et al. [15] use a PDM for analysis of the cortical sulci, Lorenz et al. [16] for the lumbar vertebrae, Styner et al. [17] for the femoral head and the lateral ventricles of the brain, and Frangi et al. [18] for analysis of the heart.
Active Shape Model
In 1995 Cootes et al. further developed the PDM into a versatile trainable method for shape modeling and matching in the form of the Active Shape Model (ASM) [19]. Thus, the PDM was extended with a matching algorithm that extracts, e.g., edge informa-tion from the target organ from an image. With this edge informainforma-tion the matching algorithm suggests new positions for the landmarks of the PDM. By projection of these new landmark positions on the shape sub space, spanned by the mean shape and the shape variations from the model, a new shape instance is created. By constraining the coordinates in the shape space to within certain statistical limits, the shape is forced to resemble the shapes seen in the training set.
Active Shape Models have found widespread application in 2D. Among others, Cootes et al. used ASMs for face recognition [19,20], Van Ginneken et al. [21] used an ASM with optimal features detection to chest radiographs. Li et al. [22] have used steer-able filters to extract local image edge features from which a small number of critical features is selected to classify image regions as edge or non-edge. They applied their method to the corpus callosum in a multi-resolution approach. Hamarneh et al. ex-tended ASMs to the spatio-temporal domain [23], and applied their (2D+time) ASM to echo cardiographic image data of the left ventricle and to synthetic images. In 3D, ASMs have been explored by Dickens et al. [24,25] on synthetic kidney data and Kaus et al. [26] in an application to cardiac image segmentation problems.
Active Appearance Model
1.2 Automatic segmentation 9
iterative matching process and have proven to be very powerful and robust tools for medical image segmentation.
As a consequence, AAMs too find widespread application. In 2D, Cootes et al. used their AAM for knees from MRI images and matching faces [27] and for face recog-nition [28]. Mitchell et al. developed a hybrid 2D AAM for simultaneous application to cardiac MRI and Ultrasound [29]. Lelieveldt et al. developed a multi-view or 2.5D AAM for application to multiple MRI views of the cardiac left ventricle [30]. Mitchell et al. developed the first 3D AAM, in an application to a cardiac left ventricle data set acquired with MRI [31]. Also, L¨otj¨onen et al. [32] and Stegmann et al. [33] developed 3D AAMs dedicated to the heart, the latter of which is bi-temporal.
In 2002, Bosch et al. developed an Active Appearance Motion Model [34], a 2D+time AAM, for segmentation of echo cardiographic image sequences.
Alternative approaches
In recent years, much more work on knowledge- and model driven statistics-based seg-mentation has been described. Alternative approaches to (statistical) shape modeling for shape extraction or image segmentation developed in the last decade are:
• Spherical harmonics
Kelemen et al. [35] propose a 3D model that uses a parametric shape representa-tion instead of a PDM. Using spherical harmonics, they automatically generate surface meshes with homogeneous node distributions. They calculate eigenvari-ations in the shape parameter space, whereas Cootes et al. [19] derive shape statistics from sample point coordinates. Matching performance was tested with a left hippocampus model; results were compared to manual segmentation. The same model by Kelemen et al. was also used for segmentation of the amygdala hippocampal complex by Shenton et al. [36].
• Constrained level sets
Tsai et al. [37] introduced a method for segmentation of medical imagery by incorporating a-priori knowledge into the level set framework. They derived a model-based curve evolution technique and applied it in 3D to a prostate gland segmentation example from pelvic MRI. Like Kelemen et al. , Tsai et al. do not derive shape statistics from sample point coordinates, but use the signed dis-tance function as a shape representation, and extract modes of shape variation by applying PCA to those distance functions.
• Statistical deformation models
Rueckert et al. [38] present an algorithm for the construction of 3D anatomi-cal models of the brain from MR images. They use a nonrigid registration al-gorithm based on free-form deformations and normalized mutual information. Thus, dense correspondences between subjects within a target population are derived from the deformations between them. These statistical deformation models have been applied to cardiac modeling by, e.g., Chandrashekara et al.[39] and L¨otj¨onen et al. [32], who recently presented a statistical deformation model built from combined long-axis and short-axis views of the cardiac ventricles and atria.
• Constructive Solid Geometry (CSG)
proposed by Ricci [40] for computer graphics and later applied to chest image segmentation by Lelieveldt et al. [41]. Danilouckine et al. used ISM for auto-matic planning of cardiac MRI acquisitions [42]. Models created for ISM usually consist of organ primitives described by parametric functions, and binary oper-ations like union, subtraction and intersection performed on the primitives. • Medial representations
Medial representations or m-reps [43,44] model object surfaces by a grid of dial atoms described by a position, a width (radius), a vector tangent to the me-dial axis and an object angle. At object boundaries an additional scalar parame-ter deparame-termines the shape of the atom from flat to circular to elongated [45]. Me-dial representations have also been explored in combination with shape statis-tics by Joshi et al. [45]. They derive features based on geometrical properties that represent growth or bending, on scale, and on location. These features are unit free and scale invariant, because they describe ratios between object-related distances. Thus the need for alignment is avoided, allowing for more localized statistical modes of variation to emerge.
1.3
Motivation of this work
Inspired by the work of Cootes et al. many groups have directed their research to-wards statistical model-based segmentation approaches for medical applications. De-formable statistical models have proven to be highly useful in medical image analysis. Especially with respect to segmentation tasks, they can be very effective and robust, and therefore they form an active field of research. The Point Distribution Model (PDM) and Active Shape Model (ASM) developed by the seminal work of Cootes et al. are the cornerstone of the algorithms presented in this thesis.
Despite the large amount of developments with respect to PDMs, ASMs and also AAMs, methods presented thus far have a number of limitations that were solved in this work. The classic ASMs and AAMs are limited to application to the same class of data that was used for the extraction of the statistics incorporated in the model. This data congruence requirement demands application of the method to equally (densely) sampled data, i.e., similarly structured image data should be acquired at the same locations of the target organ, using the same modality, the same scanner model, and the same protocol. More sparsely/densely sampled data can not be handled by such ASMs, and the modality dependence requires training of a new model.
By removing the statistical update generation scheme, the gray value constraints im-posed by the statistical gray level models currently present in the ASMs/AAMs can be released. Consequently, an alternative robust edge detection/update generation method should be developed.
The primary motivation of this work was to develop a segmentation method for the 3D cardiac left ventricle that:
• treats segmentation in an intrinsically three-dimensional manner and exploits 3D spatial continuity of the cardiac (sub)shapes,
1.4 Structure of this thesis 11
(a) (b) (c)
Figure 1.3: (a) Radially oriented cardiac image stack, (b) a combined long-axis and short-axis data set, and (c) a short-axis image stack (only showing every second image).
• can segment image data irrespective of the orientation of its constituting image slices,
• can segment a data set using only a few images. In combination with the previ-ous point, the method should be able to segment sparse image data with differ-ent oridiffer-entations, e.g., a four-chamber view, a two-chamber view and one or two short-axis views, a radially oriented long-axis stack, or a short-axis stack (see Fig.1.3). This sparse data matching enables LV function analysis without the necessity of acquiring a large number of image slices and is a major reason for us to choose the ASM approach for this segmentation task.
In this work, the focus lies on developing a segmentation method that satisfies all points above for two imaging modalities: MRI and MSCT. The method should also be applicable to multiple MRI acquisition protocols. As a consequence of the desired modality/protocol independence, common knowledge with respect to the relative or-dering of gray values in the surroundings of the heart will be included in the ap-proaches presented in this thesis, instead of including statistical gray level knowledge as present in the classic ASMs and AAMs. Furthermore, this thesis focuses completely on the cardiac left ventricle, because the majority of quantitative indices for cardiac function are derived from LV contours. However, although most of the proposed ideas were validated in application to cardiac imaging, the methods are easily extendable to other fields of computer vision.
1.4
Structure of this thesis
This thesis is further structured as follows.
independence it includes only shape knowledge, and to update model position, orien-tation and shape basic edge filtering is employed. As a proof-of-concept of modality independence in combination with an ASM, this first model trained from MRI-data is evaluated on a medium sized MSCT patient data set.
In Chapter3the edge filters are replaced by a Fuzzy Inference System (FIS) to infer model updates in a more robust manner. The basis of the new matching algorithm is the classification of pixels into three tissue classes. Image patches from the sur-roundings of the model are collectively classified using Fuzzy C-means (FCM) cluster-ing. Thus, a segmentation performance improvement is achieved while maintaining modality independence. The improvement (on MSCT data) due to this development is evaluated mainly visually, although volume errors are compared quantitatively with those from Chapter2.
In Chapter4the FIS is explained more thoroughly and is applied for the first time to both MSCT data and MRI data, without any differences in the training of the model. Only parameters in the segmentation part of the model have to be tuned to either of the modalities. For weakly defined local boundaries, edge locations are estimated by means of an interpolation scheme that uses edge location information from the closest reliable outcomes from the fuzzy inference segmentation algorithm.
In this chapter, the modality independent 3D-ASM is extensively evaluated on a large MSCT data set and a medium sized MRI data set. MSCT data from different manu-facturers was used. Evaluation of the segmentation results is performed by means of point-to-point error assessment, volume regression analysis and Bland-Altman anal-ysis all with respect to manual segmentation, which serves as the gold standard. In Chapter5 a new shape model is constructed from a far bigger training data set acquired with MRI. The model construction differs from that in previous chapters in that it is achieved by building an atlas using non-rigid registration. Point correspon-dence is determined via automatic landmarking of the atlas and propagation of the landmarks to the individual shape samples.
To overcome intensity inhomogeneity commonly observed in cardiac MR data, the model is subdivided in multiple sectors. Different sectors can be assigned different rule sets in the FIS, and with respect to the FCM operations different sectors can be combined. Since intensity inhomogeneity within one sector is limited, effects of in-homogeneity on the fuzzy clustering outcome is thus diminished. Furthermore, the model from this chapter includes a closed apex for both the endocardial and epicardial surfaces, whereas in previous chapters the apex was not defined. Another improve-ment made in this chapter is a compensation for differences in respiration level be-tween different acquisitions: the heart can shift within an image with respect to its position in other images in the same acquisition. This compensation is applied during the image matching process. Such respiration level artifacts are commonly corrected using image registration [46,47,48]. However, since registration is not the topic of this thesis, two simple registration algorithms inspired by Moore et al. [49] and based on geometric assumptions are used in a pre-processing step to the application of the 3D-ASM.
1.4 Structure of this thesis 13
In Chapter6segmentation performances are compared for three models: a 3D-ASM constructed with the point correspondence and alignment method used in Chapters2,
3and4with a highly regular mesh, the model from Chapter5, and an intermediate model. Both bi-ventricle models and single-ventricle models are built, yet only the single-ventricle models are applied to real data and extensively evaluated. Settings for all models are optimized through grid computing methods. Evaluation is performed on the same evaluation database used in Chapter5and also on the ED and the ES phase.
Chapter7presents SPASM (SParse data ASM), a newly developed method for prop-agation of update information over the model surfaces, built into the automatically landmarked model of Chapters 5 and 6. In sparse image data sets regions on the model surfaces lack model updates, turning calculation of the shape parameter vector into an ill-posed problem. The solution in SPASM is realized by a propagation scheme employing Gaussian weighting related to the distance between update source and re-ceiving nodes. The method is evaluated on a medium sized MRI data set from which different data configurations are constructed, with different image orientations (SA, LA, radial LA) and different data sparsity. This chapter shows the full potential of the model, application to arbitrarily oriented data with varying sparsity, and thus large undersampled regions.
In Chapter8the limit with respect to data sparsity is sought. The model is applied to similar MRI data sets used in Chapter7, but even sparser data sets are constructed from these. This chapter shows how many image slices are minimally required for SPASM to achieve a segmentation performance that does not differ significantly to that achieved on a full MRI data set, either SA, LA radial, or multi-view (i.e., a com-bination of SA and LA).
