Towards Computerized Diagnosis of Prostate Cancer

Bachelor Thesis

Towards Computerized Diagnosis of Prostate Cancer

Analysis of the Morphological Features of Prostate Glands

Name: Josephien Bakker

Student number: 10766235

Study: Bèta-Gamma

Major: Biomedical Sciences

Daily supervisor: Marit Lucas

Amsterdam University Medical Centre

Department of Biomedical Engineering and Physics

Date: June 29

_{, 2018}

2

Abstract

The gold standard for prostate cancer diagnosis is by histologic assessment performed by a pathologist. By this method, the pathologist assigns a Gleason grade to the prostate tissue. This is a time-consuming procedure that is also sensitive to inter-observer variation. The reproducibility of the Gleason grading varies from 60-90%. Therefore, a more objective method of disease detection is required. Due to the advancements in computer processing power and the invention of whole-slide imaging (WSI) scanners, it became possible to digitally assess prostate cancer tissue. This computerized disease detection can potentially minimize inter-observer variation, therewith improving the current standard diagnostic method. For this research, biopsies from the pathology archives of the VU Amsterdam were examined. In this project, the morphological features of prostate glands are analyzed and compared between healthy, Gleason 3 and Gleason 4 glands. For this purpose, the ratios between gland components, the shape and size of lumina and the color variation inside the nuclei are analyzed and compared between different stages of cancer. A significant increase was noticed in the cytoplasmic and nuclear area, while a significant decrease was found in the luminal area and perimeter in higher Gleason grades. Additionally, color variation within the nucleus was higher in progressed stages of cancer. This research project provides evidence that the morphological features of prostate glands are significantly different in various stages of cancer.

Introduction

Prostate cancer is the most prevalent and the second deadliest type of cancer in men (Siegel et al., 2017). One in every nine men will develop prostate cancer during his life (American Cancer Society, 2018). Currently, the definitive diagnosis of prostate cancer is via histologic assessment of biopsied tissue (Kwak & Hewitt, 2017). This diagnosis is based on the identification of several morphological features of prostate glands, that are characteristic for malign tissue (Moch et al., 2016). To detect prostate cancer, prostate specific antigen (PSA) testing is a screening method that is commonly used. Patients with an elevated PSA level are considered candidates for prostate biopsies (Liu et al., 2016). In this method, the biopsies are taken systematically from the prostate, from base to apex (Lucia & van Bokhoven, 2012). Afterwards, the samples are usually stained with hematoxylin and eosin (H&E) and microscopically analyzed by a pathologist (Mosquera-Lopez et al., 2015; Gurcan

et al., 2009). Pathologists can then assign Gleason patterns (GP) to the prostate cancer tissue, which are

numerical grades ranging from 1 to 5 that indicate the stadium of the tumor. The grade depends on several morphological features in the tissue, such as the shape and size of the glandular structures. Grade 1 is characterized by well-differentiated, individual glands whereas grade 5 is characterized by poorly differentiated, infiltrative glands (Lucia & van Bokhoven, 2012; Naik et al., 2007). A tumor usually does not consist solely of one homogeneous Gleason pattern but has various different heterogeneous patterns. Therefore, a Gleason Score (GS) is assigned to the biopsy, which is the sum of the primary Gleason pattern (most prevalent pattern) and the secondary Gleason pattern (second most prevalent pattern) in the tissue (Lucia & van Bokhoven, 2012). The GS ranges from 6 (GP 3+3) to 10 (GP 5+5).

3

In the 2014 International

Society of Urological Pathology (ISUP) consensus conference, five new prognostic Grade Groups (GG) have been proposed to be used in parallel to the Gleason grading system (Table 1) (Magi-Galuzzi et al., 2016). In this new grading system, the Gleason scores are categorized into five risk groups (Offermann et al., 2017). This new grading system offers a better indication for the severity of the tumor and guides better clinical care than the initial Gleason Score system. This is due to the distinction that is now made between Gleason score 3+4 (GG2) and Gleason score 4+3 (GG3). The difference between these groups is that Gleason 3 is the most

prevalent pattern in Grade Group 2 and Gleason 4 is the most frequent pattern in Grade Group 3. It is important to distinguish Grade Group 2 and 3 as their prognosis and treatment options are different. The 5-year biochemical recurrence-free survival rate of patients in these groups after biopsies are 83% respectively 65% (Magi-Galuzzi

et al., 2016). For low-risk patients (GG1), The American Society of Clinical Oncology recommends active

surveillance (AS). This surveillance includes regular PSA tests, digital rectal examinations and repeat prostate biopsies. This therapy option can also be suitable for intermediate-risk patients (GG2). However, higher-risk patients of GG3 and higher, should be offered an active treatment. This therapy can include radiotherapy or a radical prostatectomy (Chen et al., 2016). These therapies have displeasing consequences such as incontinence. To ensure a correct diagnosis, the distinction between GG2 and GG3 must thus be made properly to avoid any unnecessary side-effects following radical treatment (Magi-Galuzzi et al., 2016).

However, the assignment of a Gleason grade to prostate cancer tissue biopsies is an error-prone and time-consuming procedure. The diagnosis is sensitive to inter- and intra-observer variation, as not every pathologist is equally experienced and a pathologist’s decision can vary per day. Due to this subjectivity, the inter-observer reproducibility of the of the Gleason grading system ranges from 60% to 90% (Mosquera-Lopez et al., 2015; Humphrey, P. 2004). Therefore, an objective diagnostic method is needed to minimize the variation and to increase the accuracy and pace of diagnosis. Recently, an automated and computerized analysis of prostate tissue shows to be a promising concept in cancer pathology (Kwak & Hewitt, 2017).

Computer-aided diagnosis (CAD) is increasingly implemented due to the increases of computer processing power and improvements in image analysis algorithms. As it is possible to digitize histopathology slides with whole slide imaging (WSI) digital scanners, these slides can be processed by computerized image analysis and machine learning techniques (Gurcan et al., 2009). This computer-aided diagnosis is developed to aid the disease detection that usually is performed by pathologists. It improves the accuracy and decreases the reading time of the diagnosis. In contrast to pathologists, CAD is not subject to inter-observer variation (Mosquera-Lopez et al., 2015). It also has the potential to minimize the work of pathologists. For instance, about 80% of all prostate biopsies in the USA are benign, implicating that pathologists spend roughly 80% of their time examining benign prostate tissue. A computer-based approach can be a solution for faster and more accurate diagnosis of prostate cancer. It may pre-select clear cases of prostate tumor or benign tissue, giving pathologists more time to focus on rare cases that are difficult to diagnose (Gurcan et al., 2009).

In light of the above, this paper examines if prostate cancer tissue can be graded using a computerized diagnosis method, determining morphological features of the glands. This experiment focuses on distinguishing benign (healthy) glands from Gleason 3 and Gleason 4 glands, as the prevalence of these glands give different prognoses and treatment options for patients. Examples of these glands are shown in Figure Appendix 1. Healthy glands are characterized by big and well-separated glands, containing branched and large lumina. Gleason pattern 3 is typified by smaller glands with tiny, circular lumina. Gleason pattern 4 is defined by fusion of glands and overall loss of glandular structures (Nguyen & Jain, 2010). Several morphological features will be compared between

Table 1: Grade Groups

Grade Group Gleason Score

Grade Group 1 Gleason ≤ 6 Grade Group 2 Gleason 3 + 4 Grade Group 3 Gleason 4 + 3 Grade Group 4 Gleason 8 Grade Group 5 Gleason ≥ 9

4

healthy, Gleason 3 and Gleason 4 glands. As cancerous glands are characterized by smaller size of lumina, increased size of nuclei and presence of nucleoli, the hypothesis is that these features will be significantly different between healthy, Gleason 3 and Gleason 4 glands (Leze et al., 2014; Uemura et al., 2012; Naik et al., 2015)

Materials and Methods

Data collection

Acquisition of prostate samples

Patient samples were acquired from the pathology archives of the Amsterdam UMC, location VUmc. This dataset included biopsies from patients with elevated PSA levels (> 3.0-4.0 ng/mL). Taken from 53 patients, a total of 105 biopsies were analyzed. The prostate samples were obtained by taking transrectal prostate biopsies. These biopsies were formalin-fixed and paraffin-embedded, then cut into 4 µm sections and placed on glass microscope slides. Hereafter, the biopsies were stained with hematoxylin and eosin (H&E). Hematoxylin colors the nuclei blue and eosin stains the cytoplasm pink. The slides were digitized using the Philips UltraFast scanner and stored in the Philips Digital Pathology Solutions server. The digitized histology slides were exported at 20x magnification. Only Gleason grade 3, Gleason grade 4 and healthy glands were analyzed, as it is known that Gleason grade 1 and 2 lead to diagnostic errors (Mosquera-Lopez et al., 2015). It was decided to combine Gleason pattern 4 and 5, because the prevalence of Gleason 5 was low in the available sample and the disease management strategy of Gleason 5 does not differ from Gleason 4 (Mottet et al., 2014). The approval of the Institutional Review Board (IRB) was not required as an IRB waiver was obtained.

Analysis of the prostate tissue

The H&E-stained and digitized prostate histology slides were exported to OCT Patho Analyzer (an in-house developed free-hand annotation tool). This program is developed to assign a Gleason grade to the individual prostate glands, by outlining the boundary of each gland or cluster of fused glands with a specific color that represents the corresponding Gleason grade. Healthy glands were outlined in green, Gleason 3 glands in yellow and Gleason 4 glands in orange (Figure Appendix 2). Some areas were excluded from the analysis, such as blurry regions or spaces that included contamination such as hairs or colon tissue. Glands that could not be identified were also excluded from the analysis, as no corresponding immunohistochemically stained slides with markers either p63-AMACR or 34betaE12 were available. A random sample of delineations in the digitized prostate tissue was confirmed by an experienced pathologist.

Construction of binary masks

The outlined glands from the previously mentioned OCT Patho Analyzer program were converted into coordinates and stored in a “.txt” file. This file was exported to MATLAB (MATLAB 2017b, The MathWorks Inc., Natick, MA), where three individual binary masks were produced for each group, representing the healthy, Gleason 3 and Gleason 4 glands (Figure Appendix 3). The glands in the binary masks were further analyzed. The tissue is detected and healthy and Gleason 3 glands that are (partially) located on the border of the tissue were excluded. Gleason 4 glands situated on the border of the biopsy were not excluded, as Gleason 4 glands often appear in large masses that extend over the whole width of a biopsy.

Figure 1: A schematic overview of the workflow for the analysis of the binary masks

5

Analysis of the glands

Color deconvolution and color variation

Color deconvolution was performed on each individual annotated gland and some surrounding tissue (40 pixels in each direction). The aim of this color deconvolution was to extract the nuclei from the mask by separating the stains of hematoxylin (blue) and eosin (pink). By using singular value decomposition on the optical density of the image, the two vectors (belonging to H and E) were separated (Figure Appendix 4). This resulted in two separate images, in which the first image contained the regions with hematoxylin and the second image the regions with eosin. The hematoxylin channel is smoothed using Gaussian filtering, resulting in a nuclei mask (Figure Appendix 5A & 5B), which is subtracted from the original image. This resulted in the original image with black dots where the nuclei were located (Figure Appendix 5C). Afterwards, the color variation in the nuclei was determined with the CIELAB delta E equation in Lab-space. Delta E was calculated for a random selection of pixels within each nucleus and the median color differences were noted for each gland. The color variation is a numerical value that ranges from 0 to 1, in which 0 implicates two exactly the same colors and 1 means two exactly opposite colors. Color clustering

After converting the image into Lab-space, color clustering was performed using K-means clustering to identify three different categories within the glands, cytoplasm/stroma, nuclei and lumen. As the cytoplasm and stroma have a similar color, it was not possible to make a distinction between the two segments. The cluster with the on average lowest intensity represented nuclei, while the cluster with the highest intensity represents the lumina. The intensity of stroma/cytoplasm lies in between these values. This results in three masks of the region: nuclei, cytoplasm/stroma and lumen (Figure Appendix 6A & 6B), which are present within the delineated region. The amount of pixels within each binary mask were determined per gland component and then analyzed. Analysis of lumen properties

The masks of the lumen were analyzed using the “Regionprops” function in MATLAB (MathWorks Inc., USA), which measured the properties of image regions. This resulted in the area, eccentricity and perimeter of the individual lumina. The area and perimeter were measured in pixels. The eccentricity is a numerical value that ranges from 0 to 1, representing the roundness of each lumen, in which 0 is a perfect circle and 1 is a straight line.

Statistical analysis

From the K-means clustering, the amount of pixels of cytoplasm, lumen and nuclei within the glands was estimated for the healthy, Gleason 3 and Gleason 4 group. Afterwards, the percentages of cytoplasm, lumen and nuclei of the total segmented area were calculated and the ratios of nucleus/cytoplasm and cytoplasm/lumen (n/c and c/l) were determined. Subsequently, the color variety inside the nuclei was determined. Finally, the shape of lumina was evaluated by measuring the area, eccentricity and perimeter of each individual lumen. The ratio of perimeter/area was also determined and compared between the three groups. A Shapiro-Wilk test was performed to check for a normal distribution within each group and a Levene test was performed to check for equal variance. No normal distribution was noticed in any of the data. Therefore, the groups were compared with a Kruskal-Wallis test. Afterwards, a Pairwise Wilcoxon rank sum test was done to examine which groups differed significantly (P < 0.05). Additionally, the medians and interquartile ranges of all the measured features were determined per group. All of the statistical tests were performed in R Studio (version 3.5.0, GNU General Public License, June 2007).

6

Results

A total of 105 prostate biopsies from 53 patients were analyzed, resulting in a sum of 1380 healthy glands, 356 Gleason 3 glands and 1335 Gleason 4 glands. The areas of cytoplasm, lumen and nuclei were measured in addition to the area, eccentricity and perimeter of each individual lumen. Finally the color variation within each nucleus was determined. These features were compared between healthy, Gleason 3 and Gleason 4 glands. Ratios of gland segments

To examine the architectural differences of prostate glands in the various stages of cancer, the ratios of cytoplasm, lumen and nuclei were compared between healthy, Gleason 3 and Gleason 4 glands. For this purpose, the ratio of pixels of each segment to the total amount of pixels was determined. This ratio was expressed in percentages of total gland area. The medians, interquartile ranges and corresponding P values were determined of each gland segment and compared between the groups (Table 2 and Figure 2).

Table 2: Median percentages of each segment of total gland area

Gland segment Group Median (%) IQR P value P value between groups

Cytoplasm Healthy 47.7 9.8 < 0.001 H vs G3 : < 0.001 Gleason 3 52.2 11.7 H vs G4: < 0.001 Gleason 4 57.7 11.9 G3 vs G4: < 0.001 Lumen Healthy 38.7 12.2 < 0.001 H vs G3: < 0.001 Gleason 3 33.9 15.7 H vs G4: < 0.001 Gleason 4 25.5 14.5 G3 vs G4: < 0.001 Nuclei Healthy 12.6 8.2 < 0.001 H vs G3: 0.37 Gleason 3 13.2 9.9 H vs G4: < 0.001 Gleason 4 15.4 7.7 G3 vs G4: < 0.001

The cytoplasm ratio was significantly increased (P < 0.001) in higher stages of cancer. The median percentages of glandular area including cytoplasm were 47.7% for healthy, 52.2% for Gleason 3 and 57.7% for Gleason 4 glands (Figure 2a, Table 2). Likewise, the nuclear ratio was significantly increased between the groups (P < 0.001). A significant difference was noticed between H/G4 and G3/G4 (P < 0.001), but not between H/G3 (P = 0.37). The median percentages of nuclei were 12.6% for healthy, 13.2% for Gleason 3 and 15.4% for Gleason 4 glands (Figure 2c, Table 2). The luminal ratio was significantly decreased in higher stages of cancer (P < 0.001). The median percentages of lumen were 38.7% for healthy, 33.9% for Gleason 3 and 25.5% for Gleason 4 glands (Figure 2b, Table 2). Based on these results it can be concluded that the cytoplasmic and nuclear ratios were increased while the luminal ratio was decreased in higher stages of cancer.

7

Figure 2: The median percentages of cytoplasm, lumen and nuclei of total gland area

Boxplots of the percentages of glandular area that include cytoplasm (2a), lumen (2b) or nuclei (2c), per healthy, Gleason 3 and Gleason 4 glands. ***, P < 0.001

Additionally, the ratios of nucleus/cytoplasm and cytoplasm/lumen were determined. No significant difference was detected in the nucleus/cytoplasm ratio between the three groups (P > 0.05) (Figure 3a, Table 3). The cytoplasmic/luminal ratio was significantly increased in progressed stages of cancer (Figure 3b, Table 3). The median ratio of nucleus/cytoplasm was 0.262 for healthy, 0.252 for Gleason 3 and 0.262 for Gleason 4 glands. The median cytoplasmic/luminal ratio was 1.232 for healthy, 1.569 for Gleason 3 and 2.293 for Gleason 4 glands. Table 3: Ratios between gland segments and color variation inside the nuclei

Feature Group Median IQR P value P value between groups

Nuclear/cytoplasmic ratio Healthy 0.262 0.185 0.2 H vs G3: 0.74

Gleason 3 0.252 0.206 H vs G4: 1.00

Gleason 4 0.262 0.152 G3 vs G4: 0.16

Cytoplasmic/luminal ratio Healthy 1.232 0.622 < 0.001 H vs G3: < 0.001 Gleason 3 1.569 1.112 H vs G4: < 0.001 Gleason 4 2.293 1.764 G3 vs G4: < 0.001 Color variation in nucleus Healthy 0.123 0.038 < 0.001 H vs G3: 0.03 Gleason 3 0.124 0.036 H vs G4: < 0.001

Gleason 4 0.124 0.023 G3 vs G4: 1.00

Color variation inside nuclei

The color variation inside the nuclei was measured and compared between the three groups. An increase of color variation can implicate a higher availability of nucleoli. The color variation is a numerical value ranging from 0 to 1. A significant increase in color variation was noticed in higher Gleason grades (P < 0.001) as seen in Figure 3c and Table 3. The median color variation was 0.123 for healthy, 0.124 for Gleason 3 and 0.124 for Gleason 4 glands. The color variation in healthy was significantly lower than in Gleason 3 and Gleason 4 (P = 0.03 & P < 0.001). However, no significant difference was detected between Gleason 3 and Gleason 4 (P > 0.05).

8

Figure 3: Ratios of nucleus/cytoplasm, cytoplasm/lumen and color variation inside the nucleus

Boxplots of the ratios of nucleus/cytoplasm (3a), cytoplasm/lumen (3b) and the color variation inside the nucleus (3c), per healthy, Gleason 3 and Gleason 4 glands. *, P < 0,05. ***, P < 0.001.

Luminal shape

To study the differences in the appearance of the lumina in various stages of cancer, the shape and size of the lumina were assessed in different Gleason grades. Therefore, the area and perimeter of each lumen was measured in pixels, along with the eccentricity, a numerical value ranging from 0 to 1 indicating the roundness of each lumen. The medians, interquartile ranges of each feature with corresponding P values are depicted in Table 4.

Table 4: The area, eccentricity, perimeter and perimeter/area ratio of individual lumina

Feature Group Median IQR P value P value between groups

Area (px) Healthy 5,935 10,075 < 0.001 H vs G3 : < 0.001 Gleason 3 3,933 4,443 H vs G4: < 0.001 Gleason 4 4,137 5,960 G3 vs G4: 0.25 Eccentricity Healthy 0.875 0.145 < 0.001 H vs G3: < 0.001 Gleason 3 0.853 0.179 H vs G4: 1.00 Gleason 4 0.877 0.153 G3 vs G4: < 0.001 Perimeter (px) Healthy 834.755 869.660 < 0.001 H vs G3: < 0.001 Gleason 3 524.368 407.666 H vs G4: < 0.001 Gleason 4 741.434 835.015 G3 vs G4: < 0.001 Ratio perimeter/area Healthy 0.124 0.102 < 0.001 H vs G3: 0.015

Gleason 3 0.114 0.083 H vs G4: < 0.001

Gleason 4 0.140 0.142 G3 vs G4: < 0.001

A significant decrease was noticed in the luminal area and perimeter in higher Gleason grades (P < 0.001). The median luminal area (in pixels) was 5,935 for healthy, 3,933 for Gleason 3 and 4,137 for Gleason 4 (Figure 4a, Table 4). The median luminal perimeter (in pixels) was 834.755 for healthy, 524.368 for Gleason 3 and 741.434 for Gleason 4 (Figure 4c, Table 4). The eccentricity varied significantly between the groups (P < 0.001), where the lowest median was observed in Gleason 3 glands (Figure 4b & Table 4). The median eccentricity was 0.875 for healthy, 0.853 for Gleason 3 and 0.877 for Gleason 4. The ratio of luminal perimeter/area was significantly increased in higher Gleason grades (P < 0.001). The median perimeter/area ratios were 0.124 for healthy, 0.114 for Gleason 3 and 0.140 for Gleason 4 (Figure 4d & Table 4). Concluding, the lowest medians for area, eccentricity, perimeter and ratio of perimeter/area were noticed in Gleason 3.

9

Figure 4: Luminal area, eccentricity, perimeter and perimeter/area ratio

Display of the medians and interquartile ranges of the luminal areas in pixels (4a), eccentricity (4b),

perimeter in pixels (4c) and ratio of perimeter/area (4d) for healthy, Gleason 3 and Gleason 4 glands. *, P < 0.05, ***, P < 0.001.

Discussion

From these results it can be concluded that the cytoplasmic and nuclear ratios were significantly increased. Additionally, the luminal ratio was significantly decreased in higher Gleason grades compared to healthy glands. A significant increase of color variation was noticed within the nuclei of malignant tissue compared to benign tissue. There was a significant increase in the cytoplasm/lumen ratio, however no significant difference could be detected in the nucleus/cytoplasm ratio between the three groups. The luminal area and perimeter were significantly decreased in higher Gleason grades and the eccentricity varied between the three groups. The lowest medians of area, eccentricity and perimeter were found in Gleason 3 glands. This implicates that Gleason 3 glands had the smallest, most round and most regular lumina. Concluding, significant differences were noticed in the morphological features of prostate glands between different Gleason grades. However, a clear distinction cannot be made yet because the measured features had overlapping interquartile ranges between the groups. Therefore, prostate cancer diagnosis is not yet possible based on these results.

In this experiment, a realistic dataset that is representative for patients with elevated PSA levels was assessed. This dataset included multiple cases that an experienced pathologist was not able to identify. The immunohistochemically stained slides with markers p63-AMACR and 34betaE12 were not available to confirm malignancies in H&E stained slides. These markers stain the basal cell epithelial layer (Boran et al., 2011). As malignant cells do not contain this layer, these cells can be identified by absence of the staining. Because these

10

immunohistochemically stained slides were not available, the glands that could not be clearly identified were excluded from the analyses. The excluded glands mostly included Gleason 3 glands, as they often show resemblance with healthy glands. Therefore, a lower number of Gleason 3 glands was evaluated compared to healthy and Gleason 4 glands, which might have led to a loss of statistical power due to the smaller sample size. Also, the extraction of nuclei was not optimal. Because of the large color differences in the used slides, a generic color deconvolution method was used. Therefore, this method was less specific and did not extract all the nuclei from the mask. Nevertheless, an increase in nuclear ratio was noticed. The increase in nuclear ratio can be interpreted as an enlargement of nuclei that is generally observed in malignant cells. The mechanisms behind the growth of nuclei in cancer are largely unknown, but they are associated with changes in gene expression, nuclear architecture and chromatin organization during the growth and progression of a tumor (Jevtic&Levy, 2014). The increase of nuclear color variation found in this experiment can be interpreted as a higher prevalence of nucleoli, as nucleoli are darker compared to the surrounding nuclear plasma. This increase can be explained by various reasons. Increase and magnification of nucleoli is a diagnostic hallmark of prostate cancer cells. The nucleolus is the location where ribosomal RNA (rRNA) is transcribed, processed and altered (Uemura et al., 2012). The increase in size and amount of nucleoli in malignant cells is a consequence of an increased production of ribosomes that is caused by overexpression of c-MYC, p27, retinoblastoma, p53 and different growth factors (Rashid & Haque, 2011).

Research by Leze et al. (2014) have clarified that the mean nuclear volume was significantly increased in higher Gleason grades. Furthermore, they stated that prostate grading can be performed based on this feature. Sorensen et al. (1996) also presented that both nuclei and nuclear/cytoplasmic ratio are increased in higher stages of cancer. The results from this study are partly in alignment with their earlier findings, as there was an increase seen in the nuclear ratio, but not in the nuclear/cytoplasmic ratio. This can be explained because the k-means clustering in MATLAB was unable to distinguish stroma and cytoplasm. Therefore, the clustering can mistakenly have included parts of stroma as cytoplasm within Gleason 4 gland areas in the analysis, which can explain the increase of cytoplasm ratio and an indifferent nuclear/cytoplasmic ratio in higher Gleason grades. The decrease in luminal size with higher Gleason glands was also noticed by Naik et al. (2015). Additionally, Iczkowski et al. (2011) stated that patients’ outcome can be predicted based on the size and perimeter of lumina. In their research, a significant decrease was found in the size and perimeter of lumina and an increase was noticed in the perimeter/area ratio of the lumina. These conclusions are coherent with the results from our experiment. The lowest area, perimeter and eccentricity were measured in Gleason 3 glands. This corresponds to an earlier statement that Gleason 3 glands are characterized by small round lumina (Nguyen et al., 2010). Overall, the results in this experiment show that the gland features differed significantly between groups. However, there was still overlap of the features between healthy, Gleason 3 and Gleason 4 glands. Therefore, a clear diagnosis cannot be performed yet based on these results. To improve this computer-aided diagnosis (CAD) method, more data has to be acquired on the features of prostate glands. Therewith, a distinction can be made between healthy, Gleason 3 and Gleason 4 glands based on quantitative features. The CAD can apply as an aid for pathologists, by pre-selecting clear cases of prostate cancer that can be easily identified by a computer. This can not only ease the labor-intensive work of pathologists, but also improve the accuracy and reliability of diagnoses by minimizing the inter-observer variation (Kwak & Hewitt, 2017).

For further research, the color clustering must be improved to allow a distinction between stroma and cytoplasm. In this experiment, the inclusion of stroma has presumably led to an unexpected increase of the cytoplasm ratio. Also, the color deconvolution has to be optimized, as not all nuclei were extracted from the mask. Ideally, the analysis of H&E tissue is combined with the examination of immunohistochemically stained slides. The latter give more clarity about the malignancy of the tissue, allowing to distinguish benign glands from Gleason 3. Further, the delineation tool that was used in this project must be automated to enable faster analysis of the tissue. Ultimately, a deep-learning method has great potential for detection of prostate cancer (Liu et al., 2017). The development of a deep-learning method requires a lot more delineated data compared to the morphological

11

features obtained in this study. However, the delineated data as obtained within this experiment can be a first step towards a deep learning approach.

To conclude, a significant increase was measured in the nuclear and cytoplasmic ratio and in the nuclear color variation in higher Gleason grades. The lowest perimeter, area and eccentricity were measured in Gleason 3 glands. Thus, this research project provides evidence that the morphological features of prostate glands were significantly different in various Gleason grades. However, a clear diagnosis cannot be made yet based on these results. Computerized diagnosis can me possible if techniques are optimized and more data is acquired. This computerized grading can be used parallel to the current diagnosis method performed by pathologists, improving the accuracy of disease detection.

12

References

Articles:

Boran, C., Kandirali, E., Yilmaz, F., Serin, E. and Akyol, M., 2011, November. Reliability of the 34βE12, keratin 5/6, p63, bcl-2, and AMACR in the diagnosis of prostate carcinoma. In Urologic Oncology: Seminars and Original Investigations (Vol. 29, No. 6, pp. 614-623). Elsevier.

Chen, R.C., Rumble, R.B. and Jain, S., 2016. Active surveillance for the management of localized prostate cancer (Cancer Care Ontario guideline): American Society of Clinical Oncology clinical practice guideline endorsement summary. Journal of

oncology practice, 12(3), pp.267-269.

Gurcan, M.N., Boucheron, L.E., Can, A., Madabhushi, A., Rajpoot, N.M. and Yener, B., 2009. Histopathological image analysis: A review. IEEE reviews in biomedical engineering, 2, pp.147-171.

Humphrey, P.A., 2004. Gleason grading and prognostic factors in carcinoma of the prostate. Modern pathology, 17(3), p.292.

Iczkowski, K.A., Torkko, K.C., Kotnis, G.R., Wilson, R.S., Huang, W., Wheeler, T.M., Abeyta, A.M. and Lucia, M.S., 2011. Pseudolumen size and perimeter in prostate cancer: correlation with patient outcome. Prostate cancer, 2011.

Jevtić, P. and Levy, D.L., 2014. Mechanisms of nuclear size regulation in model systems and cancer. In Cancer Biology and the

Nuclear Envelope (pp. 537-569). Springer, New York, NY.

Kwak, J.T. and Hewitt, S.M., 2017, March. Lumen-based detection of prostate cancer via convolutional neural networks. In Medical Imaging 2017: Digital Pathology (Vol. 10140, p. 1014008). International Society for Optics and Photonics.

Leze, E., Maciel-Osorio, C.F. and Mandarim-de-Lacerda, C.A., 2014. Advantages of Evaluating Mean Nuclear Volume as an Adjunct Parameter in Prostate Cancer. PloS one, 9(7), p.e102156.

Liu, S., Zheng, H., Feng, Y. and Li, W., 2017, March. Prostate cancer diagnosis using deep learning with 3D multiparametric MRI. In Medical Imaging 2017: Computer-Aided Diagnosis(Vol. 10134, p. 1013428). International Society for Optics and Photonics.

Lucia, M.S. and Bokhoven, A.V., 2012. Temporal changes in the pathologic assessment of prostate cancer. Journal of the

National Cancer Institute Monographs, 2012(45), pp.157-161.

Magi-Galluzzi, C., Montironi, R. and Epstein, J.I., 2016. Contemporary Gleason grading and novel grade groups in clinical practice. Current opinion in urology, 26(5), pp.488-492.

Mosquera-Lopez, C., Agaian, S., Velez-Hoyos, A. and Thompson, I., 2015. Computer-aided prostate cancer diagnosis from digitized histopathology: a review on texture-based systems. IEEE reviews in biomedical engineering, 8, pp.98-113.

Mottet, N. P. J. B., Bellmunt, J., Briers, E., Van den Bergh, R. C. N., Bolla, M., Van Casteren, N. J., ... & Mason, M. D. (2014). Guidelines on prostate cancer. Eur Urol, 65(1), 124-37.

Naik, S., Doyle, S., Feldman, M., Tomaszewski, J. and Madabhushi, A., 2007. Gland segmentation and computerized gleason grading of prostate histology by integrating low-, high-level and domain specific information. In MIAAB workshop(pp. 1-8). Citeseer.

Nguyen, K., Jain, A.K. and Allen, R.L., 2010, August. Automated gland segmentation and classification for Gleason grading of prostate tissue images. In Pattern Recognition (ICPR), 2010 20th International Conference on (pp. 1497-1500). IEEE.

13

Offermann, A., Hohensteiner, S., Kuempers, C., Ribbat-Idel, J., Schneider, F., Becker, F., Hupe, M.C., Duensing, S., Merseburger, A.S., Kirfel, J. and Reischl, M., 2017. Prognostic Value of the new Prostate cancer international society of Urological Pathology grade groups. Frontiers in medicine, 4, p.157.

Rashid, F. and Haque, A.U., 2011. Frequencies of different nuclear morphological features in prostate adenocarcinoma. Annals of diagnostic pathology, 15(6), pp.414-421.

Siegel, R.L., Miller, K.D., Fedewa, S.A., Ahnen, D.J., Meester, R.G., Barzi, A. and Jemal, A., 2017. Colorectal cancer statistics, 2017. CA: a cancer journal for clinicians, 67(3), pp.177-193.

Sørensen, F.B., 1996. Quantitative analysis of nuclear size for prognosis-related malignancy grading. Advances in

oncobiology, 1, pp.221-255.

Uemura, M., Zheng, Q., Koh, C.M., Nelson, W.G., Yegnasubramanian, S. and De Marzo, A.M., 2012. Overexpression of ribosomal RNA in prostate cancer is common but not linked to rDNA promoter hypomethylation. Oncogene, 31(10), p.1254.

Book:

Moch, H., Humphrey, P.A. and Ulbright, T.M., 2016. Who classification of tumours of the urinary system and male genital organs. In Who classification of tumours of the urinary system and male genital organs.

Website:

American Cancer Society (2018)

https://www.cancer.org/cancer/prostate-cancer/about/key-statistics.html.

14

Appendix

Figure A1: Healthy, Gleason 3 and Gleason 4 glands

Healthy glands are typified by large glands with large, branched lumina (A1A). Gleason 3 glands are characterized by smaller glands with small round lumina (A1B). Gleason 4 glands are typified by fused glands and loss of glandular structures (A1C)

Figure A2: Delineation of glands with OCT Pathoanalyzer

Benign glands are outlined in green. Gleason 3, typified by smaller glands with round lumina, are delineated in yellow. Gleason 4 glands are outlined in orange. Regions that could not be identified or were not relevant were excluded from the analysis by outlining them in blue.

15

Figure A3: Binary masks of a prostate biopsy

A3A: H&E stained prostate biopsy. A3B: Binary mask of healthy glands. A3C: Binary mask of Gleason 3 glands. A3D: Binary mask of Gleason 4 glands.

Figure A4: Separation of the H&E vectors using singular value decomposition.

The angle of the vectors eosin and hematoxylin is maximized. Eosin is plotted on the y-axis. The values of the axes represent the optical density (given by –log10(normalized_image), in which the normalized image runs between 0 and 1). The color of the dots represents the real color as can be observed in the original image. The vectors, representing hematoxylin and eosin are separated to extract the nuclei from the mask.

16

Figure A5: Isolation of nuclei by color deconvolution

Color deconvolution is performed to isolate nuclei from the mask. A5A: a H&E-stained biopsy. A5B: Extraction of the nuclei, generated by separation of the hematoxylin and the eosin signal. A5C: Fusion of the nuclei mask with the original image, the black dots represent the areas where the nuclei are located

Figure A6: K-means clustering

A6A: Original H&E biopsy. A6B: Color clustering in an outlined gland. Dark grey represents nuclei, light grey represents lumina, white indicates cytoplasm