Development of an automatic segmentation method for kidneys in support of robot-assisted partial nephrectomy surgery

(1)

nephrectomy surgery

for kidneys in support of robot-assisted partial

Development of an automatic segmentation method

Academic year 2019-2020

Master of Science in Biomedical Engineering

Master's dissertation submitted in order to obtain the academic degree of Backer (UGent)

Counsellors: Prof. dr. Karel Decaestecker, Dr. ir. Danilo Babin, Ir. Pieter De Supervisors: Prof. dr. ir. Charlotte Debbaut, Prof. dr. ir. Patrick Segers Student number: 01405055

(2)

(3)

nephrectomy surgery

for kidneys in support of robot-assisted partial

Development of an automatic segmentation method

Academic year 2019-2020

Master of Science in Biomedical Engineering

Master's dissertation submitted in order to obtain the academic degree of Backer (UGent)

Counsellors: Prof. dr. Karel Decaestecker, Dr. ir. Danilo Babin, Ir. Pieter De Supervisors: Prof. dr. ir. Charlotte Debbaut, Prof. dr. ir. Patrick Segers Student number: 01405055

(4)

(5)

PREFACE v

Preface

Ever since I changed from electrical engineering to biomedical engineer, I wanted to have the feeling that I could have a positive impact on the healthcare world through (technical) innovation. This feeling was triggered by my sister, who is a nurse. Whenever I am injured or sick, I can always rely on her for advice and care. Of course as a biomedical engineer I do not have the medical knowledge to help patients in the same way a doctor or a nurse can do. But as a biomedical engineer there is the possibility to help improve the healthcare world using technology and as a result have a positive influence on peoples lives.

When the moment was there to find a thesis subject, I really wanted to find a thesis that possibly will have a positive impact on peoples lives and not having to write a thesis that will end up at the bottom of somebodies desk drawer. I sincerely hope that this thesis can be the start of a successful and helpful application for urologists and could possibly be expanded to other medical fields.

Of course I could not have realised this all on my own. Therefore this seems like the perfect moment to thank some important people without whom this thesis would not have been a success.

First of all I want to thank Dr. Karel Decaestecker for suggesting this thesis topic and also for allowing me to be present during a robot-assisted partial nephrectomy which was very helpful to really understand the problem. Next I want to thank my promotors Prof. dr. ir. Charlotte Debbaut and Prof. dr. ir. Patrick Segers for making this thesis possible and

(6)

an extra thanks to Prof. dr. ir. Charlotte Debbaut for the insightful remarks and useful feedback during our meetings. A very special thanks goes out to Ir. Pieter De Backer for really guiding me throughout this project, always being available for questions, giving feedback, challenging me and proof reading this paper. I also want to thank Dr. ir. Danilo Babin for helping me with some question I had during this project. The last person I most definitely should not forget to mention is Ir. Dimitri Buytaert. He granted me remote control access to his powerful computer which allowed me to work on my own computer while his computer was training my deep learning model and for that a big thank you.

Of course I can not forget to thank my friends and family for always being supportive. Special gratitude goes out to my grandmother, who has always been my biggest motivator throughout my entire university career and also to my girlfriend who always helped me calm down in moments of stress and never stopped believing in me.

(7)

PERMISSION OF USE ON LOAN vii

Permission of use on loan

“The author gives permission to make this master dissertation available for consultation and to copy parts of this master dissertation for personal use.

In all cases of other use, the copyright terms have to be respected, in particular with regard to the obligation to state explicitly the source when quoting results from this master dissertation.”

(8)

(9)

Abstract

Surgery is the standard use of care for a renal cell carcinoma (RCC), the most common form of renal cancer. Robot-assisted partial nephrectomy (RAPN) is becoming the treatment of choice in case a healthy part of the kidney can safely be spared. To perform the resection of the tumor, the main renal artery is clamped to be able to remove the tumor without excessive bleeding. If the surgeon knows which arteries perfuse the tumor and the renal tissue around the tumor, it is possible to clamp arteries further down the vascular tree instead of the main artery. As a result, more renal tissue remains perfused during the resection and does not risk losing function due to irreversible ischemic damage. Recently, 3D models of the vascular tree and parenchyma gained interest to plan this procedure. At present, such a 3D model is often developed by a manual process called segmentation, in which anatomical structures are indicated slice per slice in order to obtain a 3D model. In this work, we present the development of a fully automated approach to retrieve a 3D model. To do so, a segmentation method for the renal parenchyma is developed using a convolutional neural network (CNN) with U-net architecture. This CNN is trained on 2D CT images of pathological kidneys that are cropped around the kidney. Trained models are evaluated using the dice coefficient. The CNN with U-net architectures detects features in a down-sampling phase and localizes these features in an up-sampling phase. The network is trained with two different datasets of about 20000 slices, with varying model features. A test set of over 4000 slices and 8 individual scans are used for testing several models, obtaining a maximal dice score of 0.93 on the test set and 0.94 on the individual scans. Testing on the individual scans reveals that the models misclassify tumor tissue as renal parenchyma in case of low contrast difference between the tumor tissue and the parenchyma. A succesful method to automatically segment the renal parenchyma on scans where there is a clear distinction between renal parenchyma and tumor tissue is realised.

Keywords: Robot-assisted partial nephrectomy (RAPN), automatic segmentation, con-volutional neural network (CNN), 3D modeling, surgical planning

(10)

(11)

Development of an automatic segmentation method

for kidneys in support of robot-assisted partial

nephrectomy surgery

Jordi Martens

Supervisor(s): Charlotte Debbaut, Patrick Segers

Counsellor(s): Pieter De Backer, Karel Decaestecker, Danilo Babin

Abstract— 3D renal models may help planning of robot-assist partial nephrectomies (RAPN). In this work, the renal parenchyma is segmented automatically and the segmentation is used to create a 3D model. The seg-mentation is realised by use of a convolutional neural network (CNN) with a U-net architecture.

Keywords—Robot-assisted partial nephrectomy (RAPN), automatic seg-mentation, convolutional neural network (CNN), 3D modeling, surgical planning

I. INTRODUCTION

S

URGERY is the standard use of care for a renal cell car-cinoma (RCC) [1], the most common form of renal can-cer. Robot-assisted partial nephrectomy (RAPN) is becoming the treatment of choice in case a healthy part of the kidney can safely be spared [1]. The objective of RAPN is to remove the re-nal tumor(s) while preserving as much healthy tissue as possible to maintain maximal renal function. During resection, the main renal artery is clamped to prevent excessive bleeding, giving the surgeon 20 minutes to perform the resection. A longer ischemic period for the healthy tissue would risk a loss of renal function. If the surgeon can selectively clamp arteries further down the vascular tree, making only the tumor and the region around the tumor ischemic, more healthy tissue remains perfused, decreas-ing the risk of losdecreas-ing function due to ischemic damage. Deciddecreas-ing which arteries to clamp can be done based on 2D CT images, but is much easier and efficient using a 3D model.

A 3D model is created in a manual process through segmen-tation of the 2D slices where anatomical structures are indicated slice per slice. An example of a renal 3D model created from a manual segmentation is given in figure 1.

Fig. 1. Example of complete renal 3D model [2]

The segmentation can also be performed automatically. In this work, a method will be presented for the automatic

segmen-tation of renal parenchyma using a CNN with U-net architec-ture trained on CT images of pathological kidneys. Based on a predicted segmentation of a trained CNN, a 3D model can be created.

II. MATERIALS ANDMETHODS

To accomplish the task of automatic segmentation and con-verting the segmentation to a 3D model, several steps have to be taken. First of all, a ground truth of the structure of interest is required. For this the masks of the manual segmentation of the renal parenchyma will be used. Both mask images and orig-inal CT images require pre-processing before they can be used to train a CNN. After training the network, it can be used to predict a segmentation of new CT images. This automatic mentation can then be transformed into a 3D model. The seg-mentation is handled as a classification task, where every pixel needs to be classified into one of two classes: renal parenchyma or background (everything that is not renal parenchyma). A. Data

50 abdominal CT scans were manually segmented by use of of Mimics Research 21.0. All scans contained a left and/or right sided renal malignancy and different scanning phases (arterial, venous, blanco, excretory phase) were used for segmentation, resulting in 53 segmented kidneys. 5 extra scans were added to the dataset by the end of the project.

B. Pre-Processing

The mask images require four pre-processing steps: extrac-tion from the CT images, cropping, labeling and resizing. The masks are exported from Mimics on top of the CT images. By subtracting the original CT images from these, only the mask images remain. To eliminate the need for a kidney localization method, the images are cropped around the kidney. To distin-guish the renal parenchyma class and the background class, the pixel values of the mask images are transformed to 0, represent-ing background or 1, representrepresent-ing renal parenchyma. Finally all images are resized to 128x128 pixels because all images need to be the same size to be used as input for the CNN.

The CT images need to be cropped in the same way as the mask images and therefore also require to be resized to 128x128 pixels. Additionally the CT images are normalized. Each pixel of a slice is divided by the maximum pixel value of that slice.

(12)

In order to have a balanced dataset that contains an equal amount of background and renal parenchyma pixels, slices con-taining no renal parenchyma are removed. Making the dataset more balanced will lead to an unbiased result.

C. Data Augmentation

In biomedical deep learning applications, a lack of big data sets often necessitates expanding the dataset by artificial meth-ods. The data augmentation methods used are: mirroring, elas-tic transformation and the addition of Gaussian noise. In order to find out if different data augmentation methods could lead to different results, two datasets are created. One dataset ‘1’, is built up with the original images, mirrored images and elas-tically transformed images. The second dataset ‘2’, is created by using the original images, mirrored images and images with added Gaussian noise. After data augmentation, both dataset 1 and 2 contain somewhat over 20 000 slices compared to 5000 slices in the original datasets. Dataset 1 involves all segmented kidney, while 3 out of 53, randomly selected, segmented kid-neys are left out of dataset 2. This was done to use as much as possible scans for training, but still have some complete scans for testing. Dataset 1 is split up in a training and test set where a fraction of the training set is used as validation set (table I). Dataset 2 is completely used for training, but similar to dataset 1 is a fraction of the dataset used as validation set (table I).

Dataset 1 Dataset 2

Train Validation Test Total Train Validation Total 13980 3495 4369 21814 16326 4082 20408

TABLE I

OVERVIEW OF THE AMOUNT OF SLICES IN THE DATASETS

D. Architecture of Neural Network

In 2015, Ronneberger et al [3] designed a U-shaped CNN specifically for the segmentation of biomedical images. Because of its shape, the network is called a U-net. With this U-net archi-tecture, less training examples are required, which is a big ad-vantage for applications with biomedical images. The U-net ar-chitecture has a contracting path, this is a down-sampling phase and an expansive path, this is an up-sampling phase. The down-sampling phase detects features, while the up-down-sampling phase accurately localizes the detected features [4].

The architecture used for the automatic segmentation of renal parenchyma (see figure 2) is an adapted version of the original U-net of Ronneberger et al [3]. Dropout layers are added to prevent overfitting.

E. Model Evaluation

Evaluating a trained model is done with the dice coefficient which represents the similarity between the prediction by the model and the ground truth of the renal parenchyma:

D = 2∗ |G ∩ S| |G| + |S|

Where D is the dice coefficient, G the ground truth and S the predicted segmentation.

III. RESULTS

Dataset 1 is used to train 13 different models with vary-ing model features (optimizer, learnvary-ing rate (LR), batch size, epochs). All models are trained with the same loss function: binary cross-entropy. The loss function calculates the error be-tween the predicted output and the desired output. The goal is to minimize the error by training the network. The different mod-els were all tested on the test set of dataset 1 with the results presented in table II.

Model Nr Optimizer LR Batch Size Epochs Dice

1 Adam 0.001 1 15 0.67 2 Adam 0.001 1 20 0.58 3 Adam 0.0001 1 20 0.93 4 Adam 0.0001 100 20 0.63 5 SGD 0.01 1 20 0.43 6 AdaGrad 0.001 1 20 0.83 7 Nadam 0.0001 1 20 0.92 8 Nadam 0.0005 1 20 0.89 9 Nadam 0.0001 1 30 0.93 10 Nadam 0.0001 100 20 0.21 11 AdaMax 0.0001 1 20 0.87 12 AdaMax 0.001 1 20 0.92 13 AdaMax 0.001 100 20 0.88 TABLE II

OVERVIEW OF TRAINED MODELS WITH DATASET1

We note that the best performance was obtained by model 3 and 9. An example for model 9 is depicted in figure 3. The result of a prediction are probability values between 0.0 and 1.0, the prediction shown in figure 3 is thresholded at 0.5. All values above 0.5 are changed to 1, values equal to or below 0.5 are changed to 0.

When training a model with deep learning, it is important to make sure that you are not overfitting. The validation set is used to make sure that this is not happening. There is a model feature that shows the performance of the model but is not actually used to train and improve the model. The dice coefficient can be used as a custom metric for this feature. To know that you are not overfitting the loss on validation set should be slightly higher than on the training set and the dice coefficient on the validation set should be slightly lower than on the training set. Figure 4 and 5 show the comparison of the loss and dice coefficient between the training and validation set of model 9. From these graphs, it is clear that there is no overfitting.

Dataset 2 is used to train a model with the same model fea-tures as model 3 and tested on the three kidneys that were left out of the dataset. The results are presented in table III

Scan Dice a 0.78 b 0.92 c 0.86

TABLE III

OVERVIEW OF DICE COEFFICIENT FOR TEST SCANS OF A MODEL TRAINED WITH DATASET2

(13)

Fig. 2. Illustration of the used U-net architecture

Fig. 3. Example of CT slice (left) with the ground truth (middle) and the pre-dicted segmentation by a trained model (right)

Model 9 and the model trained with dataset 2 are also tested on the five additional scans with the results presented in table IV. Dice 1 represents the results for the prediction of the test scans with model 9. Dice 2 represents the results for the prediction of the test scans with the model trained with dataset 2.

Low dice scores in table III and IV where always caused by misclassifaction of the renal tumor(s) as renal parenchyma. The

Scan Dice 1 Dice 2 1 0.58 0.56 2 0.94 0.92 3 0.74 0.71 4 0.91 0.90 5 0.94 0.94 TABLE IV

OVERVIEW OF DICE COEFFICIENT FOR THE EXTRA TEST SCANS

contrast difference between renal tissue and tumor tissue is often limited, causing error zones on the slices where both renal tissue and tumor tissue are present (figure 6). In some scans the tumor was very small, so misclassification of the tumor(s) did not af-fect the dice score too much for these scans. If there was enough contrast difference between the tumor and the parenchyma, the models were able to segment the renal parenchyma without

(14)

mis-Fig. 4. Loss of training set vs loss of validation

Fig. 5. Dice of training set vs dice of validation set

classifying tumor tissue (figure 7). In case there was no tumor tissue on the slices, there were only some minor misclassifica-tions of structures lying close to the parenchyma with similar intensity or on slices were only a few pixels were parenchyma.

A predicted segmentation can be used to create a 3D model. Saving the predicted segmentation as bitmap (BMP) files makes it possible to load them into Mimics, create a mask of the seg-mentation and calculate a 3D model. It is also possible to put this mask over the CT images, that way the mask can be manu-ally corrected using the CT images as reference. The 3D model of the ground truth and the predicted segmentation of scan b (table III) is shown in figure 8 and figure 9.

IV. CONCLUSION

The method is able to automatically segment the renal parenchyma on scans where there is a clear distinction between the renal parenchyma and the tumor. The method does not yet give a perfect segmentation and there is still room for

improve-Fig. 6. Example of misclassification of renal tumor. Left: original CT image, Middle: ground truth. Right: prediction

Fig. 7. Example of a good predicted segmentation when tumor tissue is present Left: original CT image, Middle: ground truth. Right: prediction

Fig. 8. Example of 3D model of ground truth

Fig. 9. Example of 3D model of predicted segmentation

ment. Different combinations of parameters can be tested, mod-els can be trained for a longer period of time, more images that include tumor tissue can be added to the datasets and all of this might lead to an even better result than there is at this moment.

REFERENCES

[1] Umberto Capitanio and Francesco Montorsi, “Renal cancer,” The Lancet, vol. 387, no. 10021, pp. 894–906, 2016.

[2] Maryse Lejoly and Stefanie Vanderschelden, “Analysis and optimization of 3d renal anatomy reconstruction for rapn planning,” M.S. thesis. [3] Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net:

Convolu-tional networks for biomedical image segmentation,” in InternaConvolu-tional Con-ference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241.

[4] Ahmed Taha, Pechin Lo, Junning Li, and Tao Zhao, “Kid-net: convolu-tion networks for kidney vessels segmentaconvolu-tion from ct-volumes,” in Inter-national Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2018, pp. 463–471.

(15)

CONTENTS xv

List of Figures

2.1 Kidney anatomy and positioning . . . 4

2.2 Kidney anatomy . . . 5

2.3 Microanatomy of the nephron . . . 6

2.4 Renal vascular segmentation by Graves (right kidney). . . 7

2.5 Standard anatomy of the renal arteries . . . 8

2.6 Standard anatomy of the renal veins . . . 9

2.7 Examples of variations in renal arterial anatomy . . . 10

2.8 Examples of variations in renal venous anatomy . . . 11

2.9 World wide crude rates for renal cancer . . . 12

2.10 Overview of RAPN . . . 14

3.1 Example of a commercial virtual 3D model of a kidney with tumors provided by VisiblePatient© . . . 18

3.2 Example of virtual 3D model made with Mimics . . . 18

3.3 A hilar renal carcinoma of the left kidney . . . 19

3.4 Molds for casting silicone kidney and silicone tumor . . . 20

3.5 Workflow for the presurgical simulation . . . 21

3.6 Silicone kidney - resection of tumor with suture on the site of resection . . 22

3.7 Pre-operative simulation of robotic partial nephrectomy using 3D model . . 22

3.8 Cast of porcine kidneys . . . 24

4.1 Basic concept of machine learning and deep learning . . . 27

4.2 Schematic representation of a random forest . . . 29

4.3 Illustration of coarse-to-fine kidney localization . . . 30

4.4 Illustration of the two-step kidney segmentation on two cases . . . 31

4.5 Experimental results of kidney and cortex segmentation on one slice by the method of Chen et al [23] . . . 33

(20)

LIST OF FIGURES xx

4.7 Perceptron: basic building block of a neural network . . . 35

4.8 Interpretation of a perceptron . . . 36

4.9 Simple neural network . . . 36

4.10 Example of combining different linear classifiers to create a more complex function . . . 37

4.11 Gradient descent algorithm . . . 38

4.12 Example of convolutional neural network . . . 38

4.13 Example of a filter of convolutional layer going over a 6x6 image with stride 1 40 4.14 CNN used during the training stage . . . 41

4.15 Sample slices illustrating the results of the framework for kidney segmentation 42 4.16 Background sampling approach . . . 44

4.17 Qualitative evaluation results . . . 44

4.18 Examples of data augmentation . . . 47

4.19 U-net architecture of Ronneberger et al [32] . . . 49

4.20 Graphical representation of 3D U-net architecture of Habijan [34] . . . 51

4.21 SGD algorithm . . . 53

4.22 SGD with and without momentum . . . 53

4.23 Adam algorithm . . . 55

4.24 AdaMax algorithm . . . 56

4.25 Nesterov momentum . . . 58

4.26 Nadam algorithm . . . 58

4.27 Training of multilayer neural network on MNIST images . . . 59

4.28 Convolutional neural network training cost . . . 59

4.29 Different types of activation functions . . . 60

4.30 Leaky ReLu function . . . 62

5.1 Full sized CT image vs cropped version of that CT image . . . 67

5.2 CT images vs rotated version of that CT image . . . 68

5.3 Example of feature maps . . . 71

5.4 Example of CT image used as input . . . 71

5.5 Illustration of used U-net architecture . . . 72

5.6 Convert BMP files to dicom files with MevisLab . . . 74

5.7 . . . 75

6.1 Example of CT slice with the ground truth and the predicted segmentation by a trained model . . . 78

(21)

LIST OF FIGURES xxi

6.2 Graphs showing the performance of model 9 . . . 79

6.3 Examples of predictions of scans 1 . . . 80

6.4 Examples of predictoins of scan 2 . . . 81

6.5 Examples of prediction of scan 3 . . . 82

6.6 Comparison of 3D model of ground truth and prediction of scan 2 . . . 83

6.7 Examples of the prediction of scan 1 by both the model trained with dataset 2 and model 9 . . . 85

6.12 Comparison of 3D model of ground truth and prediction of scan 2 . . . 91

6.13 Comparison of 3D model of ground truth, prediction and manually corrected prediction of scan 5 . . . 92

(22)

LIST OF TABLES xxii

List of Tables

3.1 Advantages and disadvantages of renal 3D models . . . 25 5.1 Overview of the amount of slices in the datasets . . . 69 5.2 Alienware Aurora R7 specifications . . . 75 6.1 Overview of trained models with dataset 1 . . . 77 6.2 Overview of dice coefficient for test scans of a model trained with dataset 2 79 6.3 Overview of dice coefficient for the extra test scans . . . 83

(23)

(24)

LIST OF ABBREVIATIONS xxiv

List of Abbreviations

Adam AdaGrad ANN BMP CNN Dicom GV HU ML Nadam NAG NN RAPN ReLU RCC RMSProp SGD

Adaptive Moment estimation Adaptive Gradient algorithm Artificial Neural Network Bitmap

Convolutional Neural Network

Digital Imaging and Communications in Medicine Grayscale Value

Hounsfield Unit Machine Learning

Nesterov-accelerated Adaptive Moment estimation Nesterov Accelerated Gradient

Neural Network

Robot Assisted Partial Nephrectomy Rectified Linear Unit

Renal Cell Carcinoma

Root Mean Squared propagation Stochastic Gradient Descent

(25)

LIST OF SYMBOLS xxv

List of Symbols

xi wi b ϕ y J yi ˆ yi θ α gt Gt γ mn β1 β2 mt vt D G S Inputs Weights Bias

Activation function Output Loss function

Desired output Predicted output

Parameter of the neural network Learning rate

Gradient

Sum of squares of past gradients

Smooting term to avoid division by zero Decay term

N-th moment

Exponential decay rate for first moment estimate Exponential decay rate for second moment estimate Estimate of first moment

Estimate of second moment

Dice coefficient Ground truth

(26)

INTRODUCTION 1

Chapter 1 Introduction

1.1 The Project

Robot-assisted partial nephrectomy is a minimal invasive surgery to remove kidney tumors. When a patient suffers from one or multiple renal tumors, the objective is to preserve as much healthy kidney tissue as possible. To plan a partial nephrectomy, the surgeon can solely base the planning on the 2D medical images or in some cases a 3D model. The goal of this project is to develop a 3D planning tool that can be used by the surgeon to plan the partial nephrectomy. The tool is based on a 3D reconstruction of the patient’s kidney and tries to predict the perfusion regions of the arteries. During surgery the main artery to the kidney is clamped to be able to remove the tumor without excessive bleeding. The kidney can only remain ischemic for about 20 min, anything longer increases the risk of tissue damage. With a better understanding of which arteries perfuse the region around the tumor and the tumor itself, it is possible to clamp arteries further down the vascular tree instead of the main artery. With this selective clamping technique, more renal tissue remains perfused during the tumor removal and does not risk to become ischemic and hence permanently lose function. The 3D model itself includes the parenchyma of the kidney, tumor(s), the arteries, veins and the collecting system.

The entire project consists of two parts. The first part is the automation of the 3D kidney reconstruction. The second part is the actual perfusion planning tool. The planning tool will have the 3D reconstructed kidney as input and will allow the surgeon to make an estimation of what will possibly be the best option to clamp in order to remove the tumor while at the same time preserving as much kidney function as possible. This dissertation will focus on the automatic reconstruction or ’segmentation’ of the kidney.

(27)

1.2 Overview 2

1.2 Overview

This dissertation is built up in seven chapters. This first chapter is a short introduction into the topic of this dissertation. The second chapter will comprise the renal anatomy and physiology. Here the general function and anatomy will be discussed together with the kidneys vascular system and its variants, renal cancer and the importance of minimizing renal ischemia during partial nephrectomy. Chapter 3 handles patient-specific 3D modeling of kidneys, this gives more information about virtual and physical 3D models of kidneys and how they can be used. In chapter 4, different kidney segmentation techniques, going from manual segmentation to automatic segmentation methods are discussed with a focus on the use of deep learning. Chapter 5 explains the steps of the workflow to accomplish the automation of 3D kidney reconstruction through automatic segmentation. The sixth chapter gives an overview of the results of the models for the automatic segmentation. In the final chapter these results are discussed, a conclusion is made and we look at the possible tasks and improvements for the future.

(28)

(29)

RENAL ANATOMY AND PHYSIOLOGY 4

Chapter 2 Renal Anatomy and Physiology

2.1 General Anatomy and Physiology

The kidney is a retroperitoneal bean-shaped structure. They have an average weight of 135 g and 150 g in respectively female and male, with a length of 10-12cm, width of 5-7 cm and 2-3 cm in thickness, what can be compared to the size of a closed fist [1]. Both kidneys can be located somewhere between the transverse processes of vertebrae T12 and L3, but typically the left kidney is positioned superior to the right kidney [1]. On figure 2.1 the positioning of the kidneys in the human body is shown as well as the main (macro)structures of the kidney, figure 2.2 shows some more (macro)structures.

(30)

2.1 General Anatomy and Physiology 5

Figure 2.2: Kidney anatomy [1]

The kidneys have several important functionalities [1]:

• filtration and excretion of metabolic waste products (urea and ammmonia) • regulation of necessary electrolytes, fluid and acid-base balance

• stimulation of red blood cell production • blood pressure regulation

• controlling reabsorption of water • maintenance of intravascular volume • reabsorption of glucose and amino acids • hormonal functions

• vitamin D activation

The nephron (see figure 2.3), the functional unit of the kidney, is responsible for most of these functionalities.

(31)

2.2 Vascular Anatomy 6

Figure 2.3: Microanatomy of the nephron [1]

2.2 Vascular Anatomy

Like all organs in the body, the kidneys have arteries bringing in the O2 rich blood towards

the kidneys and veins draining the CO2 rich blood. Every individual is different and so

is everyone’s renal vascular anatomy. Knowledge of these individual variations in vascular anatomy is of importance for partial nephrectomy [3].

What is now being used as the reference model for the anatomical description of the kidney, is the model described by F.T. Graves. In 1954, Graves described the first segmental renal classification [4]. According to the model of Graves, the parenchyma of the kidney can be divided into five segments. Of these five segments, four are located on the anterior side of the kidney and one on the posterior side. The four anterior segments are the apical, superior, middle and inferior segment. Each segment is theoretically supplied by its own arterial branch. Figure 2.4 shows the different segments with their supplying arterial branch.

(32)

Figure 2.4: Renal vascular segmentation by Graves (right kidney). Pattern of arterial distribu-tion: (A) anterior view; (B) three-quarter view; (C) lateral view; and (D) posterior view. Segmentation: 1 = apical segment; 2 = superior segment; 3 = middle seg-ment; 4 = inferior segseg-ment; and 5 = posterior segment. [4]

2.2.1 Standard Renal Vascular Anatomy

Renal Arteries

In the general case, the kidney of an individual is supplied by a single renal artery orig-inating from the abdominal aorta. The renal arteries typically arise at the level of the superior margin of the second lumbar vertebral body, slightly below the origin of the su-perior mesenteric artery, with the renal artery being posterior to the renal vein. There is a difference between the orientation of the left and the right main renal artery. Going towards the right kidney, the artery follows a downward path, since the right kidney is positioned at a lower level than the left kidney. The course of the left renal artery towards the left kidney is more horizontal. This is clearly visible in figure 2.5a. The main renal artery of both kidneys branches into two parts, a posterior and anterior part. This results in a posterior and anterior segmental artery lying, respectively, posterior and anterior to the renal pelvis. The division of the renal artery into the segmental arteries takes place near the renal hilum. The posterior segmental artery is typically the first one to branch from the main renal artery, providing blood to most of the posterior part of the kidney. Next, the main artery will branch into the four anterior segmental arteries. These four arteries are the apical, upper, middle, and lower anterior segmental artery, similar to the segments of the Graves model. They are depicted on figure 2.5b. These four arteries supply different areas of the kidney with blood. The anterior and posterior surfaces of the upper renal pole are supplied with blood from the apical anterior segmental artery, while the lower anterior segmental artery is providing blood to both the anterior and posterior sur-faces of the lower renal pole. The remainder of the anterior kidney surface is provided with blood by the middle and upper segmental arteries. Following this, the segmental arteries

(33)

continue through the renal sinus. At this point the segmental arteries will branch again, into the lobar arteries. Furthermore there is even additional branching of the arteries, they include interlobar, arcuate, and interlobular arteries. Between the anterior and posterior arterial division, there is a relatively avascular plane. A clear depiction of this avascular plane is of importance to a surgeon, the region can be used during surgery to make a clean incision towards the renal pelvis. Usually the avascular plane is located posteriorly, one third of the distance between the posterior and anterior surfaces of the kidney. There is also another avascular plane that is situated between the posterior renal segment and the polar renal segments. [2, 3]

(a) CT image [3] (b) Graphical image [2]

Figure 2.5: Standard anatomy of the renal arteries

Renal Veins

The standard anatomy of the renal veins is depicted in figure 2.6. From figure 2.6 it is clear that the renal cortex is drained starting at the arcuate vein, following the arcuate vein, the blood flows to the interlobar vein. The interlobar veins join together, forming the superior and inferior trunk, these then make up the main renal vein. There are similarities and differences between the left and the right renal vein. At the renal hilum both veins usually lie anterior to the renal artery, which is similar for both left and right kidney. The first difference is a difference in length. The average length of the left renal vein is approximately 6.8-7.5 cm and for the right renal vein this is 2.5-2.6 cm, so the left renal vein is about three times as long as the right renal vein. The left renal vein will course anteriorly between the superior mesenteric artery and the aorta, to enter the medial side of

(34)

the inferior vena cava. The right renal vein will enter the inferior vena cava at the lateral side, at the level of the L1 vertebra. Before joining the inferior vena cava, the left renal vein receives several tributaries, but this is not the case for the right renal vein. [2, 3]

Figure 2.6: Standard anatomy of the renal veins [2]

2.2.2 Aberrant Vascular Anatomy

Renal Arteries

Only 70% of the population has a single renal artery as presented in the standard anatomy. The most common and clinically important variations are accessory renal arteries. It is possible to have multiple renal arteries both unilaterally or bilaterally. Unilateral accessory arteries are seen more often than bilateral accessory arteries. The accessory arteries are considered to be persistent embryonic lateral splanchnic arteries. Their origin can be located on several places. The most common case is that the accessory arteries have their origin on the abdominal aorta. This can be both at a high or a low position on the abdominal aorta. In case the origin is at a low position, the origin may be near the aortic bifurcation or from the iliac arteries. If accessory arteries originate from the abdominal aorta, most commonly they will supply the inferior pole of the kidney. In more rare cases, they find their origin in the coeliac, mesenteric, lumbar, middle colic or middle sacral artery. The accessory renal arteries can be divided into two different categories based on their course. They can be polar, piercing the upper or lower pole of the kidney directly, or they can be hilar, entering the kidney at the hilum. There is a size difference noticeable between these two categories. The polar ones are usually smaller, but the hilar ones are not

(35)

necessarily smaller than the principal renal arteries. A final common variant is pre-hilar (early) arterial branching, where a renal artery branches within 1.5-2.0 cm from the aorta. Figure 2.7 shows some examples of variations in the arterial anatomy of kidneys.

[2, 3]

(a) Four right renal arteries. The lowest origi-nates from the right iliac artery and pierces the lower pole of the right kidney directly [3]

(b) Pre-hilar branching in renal artery. [2]

Figure 2.7: Examples of variations in renal arterial anatomy

Renal Veins

Where variations in renal arterial anatomy occurred in 30% of the population, variations in the renal venous anatomy are less common. The most common variant seen in the venous system of the kidney are multiple renal veins. Multiple renal veins will occur more often on the right side. It is also possible that a single vein branches before it joins the inferior vena cava. Another variant in the venous anatomy is the late venous confluence. This is possible on both the left and the right sides. This variant is diagnosed when on the left side venous branches join within 1.5 cm from the left wall of abdominal aorta or when on the right side venous branches join within 1.5 cm of the confluence with the inferior vena cava. The next venous variant is the most common one of the left renal venous system. The variant is called the circumaortic renal vein. In this variant there is a bifurcation of the left renal vein. The renal vein is divided into ventral and dorsal limbs that encircle the abdominal aorta. This can again be split up in two different variants. In the first

(36)

2.3 Renal Cancer 11

variant one renal vein at the renal hilum subsequently divides before entering the inferior vena cava. This variant is the most common of the two. In the second variant, there are two distinct veins that originate from the renal hilum. When a circumaortic renal vein is present, typically the adrenal vein joins the pre-aortic limb and the gonadal vein joins the retroaortic limb. A last and less common variant in the renal venous anatomy that will be discussed is the completely retroaortic renal vein. In this case the single left renal vein courses posterior to the aorta, it then drains either into the lower lumbar portion of the inferior vena cava or into the iliac vein. Figure 2.8 shows some examples of variations in the venous anatomy of kidneys.

[2, 3]

(a) Circumaortic renal vein [2] (b) Accessory renal artery to the right lower pole [2]

Figure 2.8: Examples of variations in renal venous anatomy

2.3 Renal Cancer

2.3.1 Background

Detection of a renal mass is often an incidental finding. It is very common to detect a renal mass thanks to abdominal imaging that was originally intended for a different reason [51]. Detecting a renal mass does not necessarily mean the detection of renal cancer, it can always be a renal cyst or benign lesion (eg angiomyolipomas, oncocytomas) that does not

(37)

2.3 Renal Cancer 12

require any treatment or follow-up, but in a notable proportion of cases the renal masses are not innocent [51].

Renal Cell Carcinoma (RCC), the most common form of renal cancer, comprises a het-erogeneous group of cancers, with different genetic and molecular alterations, arising from renal tubular epithelial cells [50, 51]. RCC accounts for 5% of all oncological diagnoses in men and 3% in women, making it the sixth most frequently diagnosed cancer in men and the 10th in women, worldwide [53]. Figure 2.9 shows the worldwide crude rate for renal cancer in both men and women. The three most common RCC subtypes that account for 85% of all renal malignancies are clear-cell RCC, papillary RCC and chromophobe RCC [50, 51]. There are also some less common cancers like papillary adenoma, multilocular cys-tic clear-cell carcinoma, hybrid oncocycys-tic chromophobe tumour, carcinoma of the collecting ducts of Bellini, renal medullary carcinoma, carcinoma associated with neuroblastoma, and mucinous tubular and spindle-cell carcinoma [51].

Figure 2.9: World wide crude rates for renal cancer [54]

Although renal cancer is often diagnosed during examinations for other medical problems a patient is having, there are some signs and symptoms that could indicate renal cancer. Symptoms include blood in the urine, pain or lump in the lower back or abdomen, fatigue,

(38)

2.3 Renal Cancer 13

weight loss, fever and swelling in the legs and ankles [52]. But only 30% of patients with renal cancer will be diagnosed on the basis of one or multiple of these symptoms [51].

2.3.2 Tumor Vascularization

From section 2.2 it has become clear that there is not a single model to describe the renal vascularization of the kidney segments for all individuals. It is demonstrated that a single segment can be supplied by multiple arteries. From this conclusion, the hypothesis rises that this would also be the case for a renal tumor. So, a renal tumor can be vascularized by branches that originate from an artery leading to another segment [4]. A study by Angers University Hospital analyzed the vascularization of renal tumors by using data from preoperative 3D arteriography, while performing tumor embolization, prior to partial nephrectomy [4]. In order to analyze the tumor vascularization, the following data were collected: tumor localization, number of tumor arteries and segmental origin of each one. The aspect of tumor vascularization (hypovascularization, moderate vascularization, or hypervascularization) was assessed by comparing the tumor and parenchymal vascular enhancement, according to the expertise of the interventional radiologist at the time of arteriography [4]. From the study it has become clear that the hypothesis is true. A renal tumor can be vascularized by a single artery as well as multiple arteries originating from different segments. The more arteries that supply blood to the tumor, the bigger the tumor will be. [4]

Knowing which arteries supply the tumor are of importance for the surgeons. Selective clamping, rather than clamping the main artery, can only be done properly and without any risk if there is a good knowledge of the arteries perfusing the tumor. It is obvious if you do not clamp all arteries going to the tumor, there will possibly be excessive bleeding when the tumor is being removed.

2.3.3 Treatment

The most frequently used treatment for renal cancer is surgery [51, 52], but surgery is not always necessary or sometimes not even an option. For patients with small tumors active surveillance is a possibility [52]. In case a patient is not fit for surgery or it is not necessary there are alternatives like cryotherapy and ablation to remove a tumor [51, 52]. Immunotherapy is also used in the treatment of renal cancer, this can be a stand-alone treatment or in combination with other treatments like surgery.

(39)

2.3 Renal Cancer 14

The perfusion tool is being developed for robot-assited partial nephrectomy (RAPN), hence a little bit more information will be given about this procedure. RAPN is a minimally invasive, nephron sparing, surgery. This means that the goal of RAPN is to remove the entire tumor while preserving as much as possible of the renal tissue and in that way keeping the maximum amount of renal function. Some more information will be given in the next section. Figure 2.10 gives an overview of a RAPN.

Figure 2.10: Overview of RAPN. (A) In robot-assisted surgery, instead of directly moving the instruments, the surgeon performs the normal movements associated with the surgery, and the robotic arms make those movements and use end-effectors and manipulators to perform the actual surgery on the patient. One arm is dedicated to the laparoscope and the two others hold forceps, monopolar curved scissors, a cautery hook, and a large needle driver. The patient is positioned in a modified flank position. Port configuration can vary based on tumor location to optimise the working angles. Surgical excision of the tumor is done by (B) kidney mobilisation, (C) tumor resection (with or without a rim of normal parenchyma according to anatomical and tumor features), and (D) final reconstruction (renorrhapy). [51]

(40)

2.4 Renal Ischemia During Partial Nephrectomy 15

2.4 Renal Ischemia During Partial Nephrectomy

In the previous section it was mentioned that surgery is the most common treatment for renal cancer and RAPN was given as an example. When, instead of removing only the tumor, the whole kidney is removed, this is called a radical nephrectomy. As a result of a radical nephrectomy, the patient will lose a lot of kidney parenchyma and possibly require dialysis if there would be a problem with the remaining kidney. The benefit of partial nephrectomy in renal mass patients, compared with radical nephrectomy, is preservation of renal function [12]. To accomplish partial nephrectomy, the blood supply to the kidney where the tumor is present, has to be clamped. Since the renal arteries are end vessels, the kidney is particularly sensitive to ischemia [4]. When clamping the main renal artery, the surgeon has about 20 minutes to remove the tumor [11].

If the warm ischemic period takes longer, there might be side-effects. Renal ischemia primarily has an effect on the vascular system, leading to vaso-obstruction and reperfusion injury that reduces renal blood flow and decreases organ activity. On a vascular level, multi-inflammatory responses by interleukins lead to vasoconstriction and vascular spasms. A vicious cycle is activated by endothelial injury, causing the activation of a cytokine cascade that results in arteriolar vasoconstriction. The low renal blood flow releases angiotensin II and eicosanoids. In this state of ischemic insult, failure of oxidative phosphorylation and adenosine triphosphate (ATP) depletion cause cellular swelling by passive water diffusion into the cells, and, consequently, both the ’no reflow’ phenomenon and vascular obstruction during renal reperfusion occur. Free radicals resulting from ATP degradation cause further cell damage. [11]

A solution for this problem, as already mentioned earlier, is that the clamping is not done on the main artery, but selectively on arteries closer to the tumor. This would result in only a small area around the tumor and the tumor itself becoming ischemic, while the remaining part of the kidney is still supplied with blood giving the surgeon more time to perform the tumor removal. Finding the best arteries to clamp is the goal of the planning tool that is part of this research.

(41)

(42)

PATIENT-SPECIFIC 3D RENAL MODELING 17

Chapter 3 Patient-Specific 3D Renal Modeling

A lot of surgeries are planned based on 2D images, but a 3D image makes it often easier for a surgeon to interpret the anatomy and plan the surgery. At present, this is often very costful, since current models are often provided by external companies. The companies provide a virtual 3D that can also be 3D printed to acquire a physical model.

3D models can be separated into two different categories: virtual 3D models and physical 3D models. Of course the physical models result from a virtual model.

3.1 Virtual 3D Models

2D images from eg CT or MRI scans are often manually transformed into 3D images by using different types of software in a process called segmentation. Software used includes Mimics Medical, 3D Slicer, Simpleware and many more. Transforming the 2D images to a 3D image is often a very time consuming and manual process which makes it costly. Furthermore, the quality of the 3D image will be highly dependent on the quality of the 2D images which in his turn is influenced by aspects like patient movement, artifacts, scanning protocol, etc. A final difficulty lies in capturing the details of the vascular system, which is of great importance in the case of partial nephrectomy. It is rather difficult to get quality images of vessels further than the third or fourth generation of branching. Nevertheless, having a virtual 3D model can already be very useful for the surgeon in many ways, even without a good representation of the vessels. It improves the understanding of the surgical anatomy, it facilitates the surgical planning and the surgeon can also use this model to educate the patient on his disease or surgical procedure. A doctor has experience with interpreting 2D medical images, but for a patient these can be hard to understand, so

(43)

3.1 Virtual 3D Models 18

having a 3D model that can be shown on a computer is a very helpful tool to get the patient more involved and informed. Figures 3.1 and 3.2 show some examples of virtual 3D models. Figure 3.1 shows an example of a commercial virtual 3D model that was made by the company VisiblePatient ©. Figure 3.2 shows some examples of a virtual model made by two medical students using Mimics.

(44)

3.2 Physical 3D Model of Kidneys 19

3.2 Physical 3D Model of Kidneys

Having to make the physical model takes extra time and money, however it is often an even better way to visualize the pathology, understand the anatomy and anticipate on possible problems for both the surgeon and patient. Physical models can also be used for a more extensive planning, because some models mimic the real life tissues and can be used to perform a ’practice’ surgery. These models can also be used for training and educating surgeons. A great example regarding the advantages of a 3D model is described in the article of Zhang et al [6]. Zhang et al evaluated 3D printed models for laparoscopic partial nephrectomy of renal tumors. The objective was to investigate the impact of 3D printing on the surgical planning, potential training and patients’ comprehension of minimally invasive surgery for renal tumors [6]. The 3D model was created from images obtained from contrast enhanced CT scans, these images were processed and transformed into a virtual 3D model that was used to print a 3D model. The printing material was a thermoplastic polymer and the different parts of the model were manually colored to represent different anatomical parts (figure 3.3).

Figure 3.3: A hilar renal carcinoma of the left kidney. a) Standard contrast enhanced CT scan, b) CT reconstruction, c) 3D printed kidney model with the tumor, d) model sectioned with the maximal diameter of tumor. [6]

(45)

The model was evaluated by four experienced urologists and ten patients to be able to evaluate all the different aspects of the model. The urologist scored the model good on overall usefulness (7.8 _{± 0.7) and usefulness in surgical planning and training (8.0 ± 1.1),} but details of the vasculature and collecting system scored only 6_{± 0.6 on a scale from 0 to} 10 [6]. Creating a quality reconstruction of the vasculature is a difficult task and one of the initial goals of this dissertation is to improve this with an automatic method. The patient scored all aspects of the model related to them a 9 out of 10. This was for satisfaction of the conversation with the surgeon and the usefulness to understand the disease and procedure with the model.

The overall conclusion from the urologists perspective was that all participating urologists agreed and advocated that it would be an advancement for surgical planning and potential training of demanding procedures, when combined with the 2D data [6]. From the patients’ point of view, it was very clear that the 3D model helps in understanding the problem and makes the conversation with the urologist more understandable.

3.2.1 Surgical Simulation

To my knowledge four studies simulate RAPN by the use of a patient specific silicone 3D model [5, 7, 8, 9]. All of these studies contained only a rather small group of subjects, ranging from 3 to 10 subjects. All four studies created a virtual 3D model from a pre-operative CT or MRI. Different types of software were used for the creation of the 3D model, like 3Dslicer [5, 7], Invesalius [9] or collaborated with a specialized company like 3DSystems [8]. For the fabrication of the 3D models, different methods are used, ranging from 3D printing the model completely, to silicone molding part per part. Examples of such molds are given in figure 3.4.

(46)

Once the model is finished, it can be used to simulate a RAPN. The simulation surgery was executed in at least 3 of the 4 studies on a da Vinci® surgical robot [5, 8, 9] and one with an unspecified simulator. To perform the simulation surgery, the model is placed in a laparoscopic trainer box (see figure 3.5) or put on a tripod (see figure 3.6) or just placed on a table (see figure 3.7). Some of the studies specify that the positioning of model is adapted in such a way that it approximates the actual positioning during surgery as close as possible.

Figure 3.5: Workflow for the presurgical simulation. A patient specific 3D digital model is reconstructed on the basis of the available preoperative axial imaging (CT or MRI) (1-2) and then converted into a cast silicone soft tissue model (3). Robotic-assisted laparoscopic enucleation of the kidney tumor is rehearsed for all patients using the 3D printed model in a three-arm configuration (4-5). Both the resected model tumor and the patient’s actual resected tumor are 3D laser scanned in the operating room for volumetric measurements (6-7). All scans are then reprocessed and each pair (specimen scan from 3D kidney model and patient) is digitally overlaid to visualize the concordance in mass contour. [5]

(47)

Figure 3.6: Silicone kidney - resection of tumor with suture on the site of resection [7]

Figure 3.7: Pre-operative simulation of robotic partial nephrectomy using 3D model [8]

To convincingly prove outcomes for studies like this, a larger case control study would be necessary. Nevertheless some useful information can still be gathered from these studies.

The first similar conclusion from all the studies is that the simulation of the surgery can be very useful in training novice surgeons. For beginning surgeons it is a good way to acquire some new skills or even improve their already existing skills. Thanks to the sim-ulation they have the ability to gain these new skills in a stress-free environment without compromising the life of any patient. This teaching ability will result in a flattening of the early steep learning curve novel surgeons have to overcome. There are several reasons for this steep learning curve. First of all is the kidney an important organ that receives 20% of the body’s cardiac output per minute, making intraoperative bleeding a significant pos-sibility despite renal artery clamping [8]. Being able to train novice surgeon better should also improve patient outcome. Next, warm ischemia time is very important, the longer the kidney is not perfused, the higher the risk of function loss. For these reasons, renorrhapy is commonly the most difficult portion of the surgery for young surgeons [8]. Renorrhapy is suturing the kidney. This has to happen fast, because the warm ischemia time has to

(48)

be minimized, but it also has to be done with great care. The sutures will compress the renal parenchyma, risking damage to the intraparenchymal vessels, it also reduces renal blood flow and there is also a chance that the compression of the parenchyma results in the formation of renal artery pseudoaneurysms or vascularized parenchymal volume reduction [48].

The simulation is not only beneficial for novice surgeons, but also for experienced surgeons. Especially in more complex cases the simulation gives the experienced surgeons helpful insight and they used the experience for particular cases in the actual surgery. By doing the simulation first, they have a better idea on what the best approach would be to perform the actual surgery.

Another result that was similar between the different studies was the realism of the model. This is maybe more of a subjective result, but all participating surgeons were positive about the similarity between the model and a real life kidney, for example when it comes to needle driving or cutting.

Some studies also looked at the surgery margins. In one study there were only negative margins in the simulation and one positive margin in the actual surgery [5]. While in another study there were only negative margins in both the simulations and actual surgeries [8].

The next interesting aspect is the resection time. This differed in the different studies, some showed a striking similarity in tumor resection time when comparing the simulation to the real surgery, another showed a decrease in resection time. Another showed that the resection time of the actual surgery was longer that the resection time of surgeries that were performed without a simulation prior to the actual surgery, but this was probably the consequence of the complexity of the cases in the study, which was higher than the average complexity of the cases in their database.

An aspect that was only handled by [8] was blood loss. By comparing the blood loss in the cases of the study and cases outside the study, where no simulations were performed, there was a significant difference in the amount of blood loss. The surgeries that were simulated prior to the actual surgery showed a lower amount of estimated blood loss.

The most meaningful measure of any surgical training model is the extent to which model performance is associated with operation room performance improvements [9]. Im-proved operation room performance could lead to imIm-proved patient outcome, but it is difficult to asses these aspects.

To conclude the discussion about the surgical simulations for RAPN, it was notable that most of the studies were performed at the same time, since 3 out of the 4 studies assume

(49)

they were the first to perform the study and seeing that they show a lot of similar results, gives extra confidence that some of the result might be valid, but larger studies are required to get more definite proof. The first results are very promising, but there is still room for improvement given the realism of the simulations. The models used now are very good and accurate replicas of the patient-specific anatomy, but only include the renal parenchyma, tumor and some arteries. The models are missing smaller arterial branches as well as venous branches which all might be important during the actual RAPN. Additionally, peri-nephric fat is also missing in the models. Peri-nephric fat can make identification and dissection of the vasculature challenging. So, peri-nephric fat and active blood supply are aspects that should be added in future models.

3.2.2 Casting

Creating a physical 3D model is, as is clear from previous sections, done by either 3D printing or casting. However, the examples from previous sections all created the physical 3D model from a virtual model. It is also possible to cast a real kidney. For the master dissertation of Nathan De Kock [59] two porcine kidneys were casted using the actual kidneys (see figure 3.8). The casting procedure includes three steps: injection of the casting fluid, polymerisation of the casting fluid and maceration of the tissue [59]. The maceration is the removal of the tissue by a treatment with a solution of potassium hydroxide (KOH) [59]. The result is a perfect replica of the vascular tree of the kidney. A porcine kidney shows both anatomical and physiological similarity with the human kidney [59]. This mean that this method could possibly be used to evaluate automatically created 3D models by comparing a cast of an actual porcine kidney with a print or cast of a virtually created 3D model.

(50)

3.3 Summary 25

3.3 Summary

To summarize, a quick overview of the advantages and disadvantages of 3D models:

Advantages Disadvantages

Can be used for: - Expensive

- surgical planning - Labour intensive

- surgical training - Suboptimal 3D image in case of low quality image acquisition systems - medical education - 3D engineering knowledge often not available in surgical department - patient counseling

(51)

(52)

KIDNEY SEGMENTATION TECHNIQUES 27

Chapter 4 Kidney Segmentation Techniques

4.1 Goal of This Thesis

The goal of this dissertation is to develop an automatic method for the segmentation of renal parenchyma. 2D CT images are the starting point, from these images the kidney will be segmented and converted into a 3D image. The most obvious and probably the most simple way to do this is manually segment the images. However this is also a very time consuming process. In order to reduce the required time to segment a kidney, the objective is to be able to do this as automatically as possible, with minimal human intervention. Over the years different methods have been developed to (semi-)automatically segment the kidney. With the increasing popularity of machine learning, this became a frequently used method and now also deep learning is being deployed. A very simple way to visualize how both machine learning and deep learning work is given in figure 4.1: here we see that input data and the desired output are used to compute a program.

Figure 4.1: Basic concept of machine learning and deep learning [44]

The main difference between machine learning and deep learning can be best explained by giving a general definition of both concepts. Machine learning can be defined as using training data to learn how to classify findings based on handcrafted features [49]. Deep learning on the other hand uses training data or labeled data to iterate on the dataset in

(53)

4.2 Manual Segmentation 28

order to learn the features, which are now no longer defined by the designer [49]. Hence, deep learning is more of a black box than machine learning since with machine learning you have to select features, while in deep learning this is done by the computer.

4.2 Manual Segmentation

As mentioned before, manual segmentation can be performed with a lot of different types of software. An example of how you can manually segment a kidney (with tumor), including the vessels and collecting system is giving by Maryse Lejoly and Stefanie Vanderschelden in their thesis [13]. The tutorial shows how to perform the kidney segmentation with Mimics Research 21.0. The tutorial is a step by step explanation of how to segment each part of the kidney separately. It shows different possibilities on how to tackle the segmentation task, showing which Mimics functions to use.

4.3 Automatic Segmentation Using Machine

Learn-ing

A lot of different methods to automatically segment from CT images have been developed in order to try and minimize the time necessary to perform the segmentation task. However this is not an easy task. The automatisation of segmentation of the kidneys is quite challenging because of grey levels similarities of adjacent organs, contrast media effect and relatively high variation of the position of organs and their shapes in abdominal CT images [14]. Nevertheless, there are several methods to automatically segment the kidneys from 2D images. A method that is often used is a coarse-to-fine strategy [14, 15, 16, 17]. This strategy first determines a rough estimate of the kidney position and after this it makes a more accurate localization of the kidney, followed by the actual segmentation of the kidney. The different segmentation methods will first shortly be described and some are explained in more detail in the next sections. The segmentation itself can also be done with a coarse-to-fine approach. In some cases, the spine is used as a reference point to determine the kidney position [14, 16]. The difference in these methods lies in the approach on how to locate the kidneys and do the actual segmentation. Other methods make use of active shape models [18, 19]. The active shape model framework can constrain the segmentation while learning the kidney mean shape and principal modes of variation [15]. An active shape model can be used to find the expected shape and local grey level structure of a target object in an image [24]. Given a rough starting approximation, an instance of a

(54)

4.3 Automatic Segmentation Using Machine Learning 29

model can be fit to an image [24]. An iterative approach is used to improve the fit in every step [24]. Yet another way to do the automatic segmentation is by use of a deformable model based approach [20, 21]. Or even other strategies like the use of spatial-appearance models [22] or a leave-one-out strategy [23]. An important detail about all these different methods is that they are not all fully automatic, some require manual initialization, other use images that are already cropped around the kidney. It is clear that there are a lot of different approaches to this problem.

4.3.1 Coarse-To-Fine Strategy Using Random Forest

The first example of a segmentation method is a coarse-to-fine method, specifically ‘Auto-matic detection and segmentation of kidneys in 3D CT images using random forests’ [15]. A random forest is an ensemble of decision trees. To create a random forest, the original dataset is randomly sampled to create new datasets, one or a few attributes to split on are randomly sampled and different trees are grown for each subset of samples and features [56]. To make a decision, the prediction of the different trees is averaged or a majority voting is taken [56]. The principle of a random forest is again graphically explained in figure 4.2.

Figure 4.2: Schematic representation of a random forest [55]

The method itself is used for the segmentation of renal parenchyma. They claim to have a fully automatic method, that works on any kind of CT image, regardless the contrast

(55)

4.3 Automatic Segmentation Using Machine Learning 30

phase (this includes also without contrast) and this for both healthy kidneys or patients with kidney tumors. The first step is locating the kidneys with regression forests following a coarse-to-fine approach. For the coarse localization, the goal is to find a rough estimate of the kidney location. This is achieved by finding rectangular bounding boxes around the kidneys based on two points. Hence, a bounding box is parameterized by a vector which is just two points in 3D. A random forest is trained with known bounding boxes on CT images. The features that are used here are the mean intensities over displaced, asymmetric cubiodal regions. This is because the intensities on CT images have a physical meaning. After training, the random forest should be able to give an estimate of the kidneys position as well as size. After the coarse localization, the region of interest is being refined. This refinement step is based on local information only and is done for each kidney separately. For this step, a regression forest is trained to predict, from every voxel located in its neighbourhood, the relative position of the kidney’s center. The features that are being used here are, for each voxel, its intensity, its gradient magnitude and its neighbours’. An overview of the coarse-to-fine kidney localization is given in figure 4.3.

Figure 4.3: Illustration of coarse-to-fine kidney localization. (a) Initial bounding boxes detected using global contextual information. (b) Refinement step: going from the initial bounding box (red) to the refined bounding box (green) by letting each voxel around the center of the initial bounding box vote for a location of the kidney’s center. (c) Comparison between the initial (red) and refined (green) bounding box. [15]

Development of an automatic segmentation method for kidneys in support of robot-assisted partial nephrectomy surgery

nephrectomy surgery

for kidneys in support of robot-assisted partial

Development of an automatic segmentation method

nephrectomy surgery

for kidneys in support of robot-assisted partial

Development of an automatic segmentation method

Preface

Permission of use on loan

Abstract

Development of an automatic segmentation method

for kidneys in support of robot-assisted partial

nephrectomy surgery

S

Contents

List of Figures

List of Tables

List of Abbreviations

List of Symbols

Chapter 1

Introduction

1.1

The Project

1.2

Overview

Chapter 2

Renal Anatomy and Physiology

2.1

General Anatomy and Physiology

2.2

Vascular Anatomy

2.2.1

Standard Renal Vascular Anatomy

2.2.2

Aberrant Vascular Anatomy

2.3

Renal Cancer

2.3.1

Background

2.3.2

Tumor Vascularization

2.3.3

Treatment

2.4

Renal Ischemia During Partial Nephrectomy

Chapter 3

Patient-Specific 3D Renal Modeling

3.1

Virtual 3D Models

3.2

Physical 3D Model of Kidneys

3.2.1

Surgical Simulation

3.2.2

Casting

3.3

Summary

Chapter 4

Kidney Segmentation Techniques

4.1

Goal of This Thesis

4.2

Manual Segmentation

4.3

Automatic Segmentation Using Machine

Learn-ing

4.3.1

Coarse-To-Fine Strategy Using Random Forest