Bachelor Informatica
Integrating digital microscopy
images in an Electronic Data
Capture system
Tim van Zalingen
June 11, 2017
Supervisor(s): Emile van der Donk (FormsVision), dhr. dr. R.G.
Inf
orma
tica
—
Universiteit
v
an
Ams
terd
am
Abstract
Clinical research increasingly uses advanced tools built into Electronic Data Capture systems for diagnostics purposes and reporting of patient conditions. Digital microscopy is used in the diagnostic stage of patient treatment, producing large JPEG or TIFF objects that need to be accessible online in the EDC system.
A whole slide imaging system is created for FormsVision, commissioned by the VUmc. The size of the images supplied gives complexity to the project. The size varies between 100 MB and 10 GB. The system features the uploading of a JPEG or TIFF, preprocessing into an image pyramid and a virtual slide viewer to view the slide without having to download the whole image. In order to facilitate future demand, a model is created to predict the required hardware specifications. The validity of this model is tested for the current clinical trial. This also shows the hardware specifications required for the current trial and the bottlenecks in the whole slide imaging system.
Contents
1 Introduction 7 1.1 Introduction . . . 7 1.2 Project . . . 8 1.2.1 Requirements . . . 8 1.2.2 Research . . . 8 1.3 Related work . . . 9 2 Design 11 2.1 Whole slide imaging system . . . 112.2 Uploading . . . 12
2.3 Preprocessor . . . 12
2.4 Virtual slide viewer . . . 12
2.4.1 Specifications . . . 12 2.4.2 Considerations . . . 13 2.4.3 Implementation . . . 13 2.5 Hardware . . . 14 2.6 Model . . . 14 3 Experiments 17 3.1 Outline . . . 17 3.2 Results . . . 18 3.3 Evaluation of model . . . 20 4 Conclusions 23 4.1 Conclusion . . . 23 4.2 Discussion . . . 23 4.3 Future work . . . 23
CHAPTER 1
Introduction
1.1
Introduction
Digitisation for clinical research allows for better sharing of measurements and data between institutes [7, 8]. Part of this is the sharing of microscopy images for clinical pathology, called telepathology. Whole slide imaging is the creation of digital images from a microscopy slide and viewing these images through a computer instead of through a microscope using a virtual slide viewer [17]. An example of a digital microscopy slide image and the same image magnified is shown in figures 1.1 and 1.2, respectively.
Whole slide imaging has several advantages over the regular method of viewing the physical microscope slide through a real microscope. First of all, whole slide imaging offers the possibility of viewing a slide image from outside the institute [17]. Related to the previous, long-term storage of a slide image is easier than the storage of a physical slide [13]. It also reduces the time it takes to switch a slide, reduces the risk of breaking a slide and allows for better distribution of the workload [3].
The accuracy of digital microscopy for telepathology compared to conventional pathology, has already been proven [7, 5].
Figure 1.2: Zoomed in on the example in figure 1.1 of a digital microscopy slide image.
1.2
Project
This project focuses on creating a digital microscopy viewer for FormsVision, commissioned by the VUmc. FormsVision is a company that creates software to support clinical trials [1]. One of their products is ALEA Clinical, an Electronic Data Capture (EDC) system, which provides the ability to view measurements and data taken from patients during a trial. The whole slide imaging product, integrated in ALEA Clinical, will allow a user to view images inside the browser and upload digital microscopy images. For this pathological study digital microscopy is a must, since the images need to be shared between institutes.
1.2.1
Requirements
The clinical trial, performed by the VUmc, will last for four to five years. Thirty different institutes will be involved, with around ninety pathologists sharing their microscopy images. These images are either in JPEG or TIFF format, ranging from 100MB to 10GB in size. The trial will include 1100 patients, with ten slides per patient. So this adds up to 11000 slides in total. Each slide will be examined once by the researcher at the VU. The expected upload rate is at least one slide per week during the beginning of the trial and a maximum of six slides per day at the later stages of the trial. The slides have to be stored for a duration of one week [14, 16].
The main challenge lies in the sheer size of these images. It will not be possible to simply send a user an image as one single large file, this would take too long with a common network connection.
1.2.2
Research
A theoretical model of the system will be developed that describes the performance of the de-signed solution for different hardware configurations. This model can be used by FormsVision to calculate the hardware needs for future demand. Experiments will be run on the system to test the validity of the model. First, the size and expected system use for this trial will be determined by interview with a pathologist and the trial manager. Afterwards simulations will be run on
the system based on the expected use. This will be compared to the outcome of the model for the same parameters.
1.3
Related work
A framework similar to what is presented here is OpenSlide [4]. OpenSlide is a C library for reading and manipulating digital slides. Despite its comprehensiveness, it does not easily fit within the automated process that FormsVision needs for the duration of the trial. This process will be explained in section 2.1.
The challenge in handling large images is similar to the challenge found in viewing and storing geographic images [9]. An image pyramid, as in OpenSlide, is usually used to store the images. These existing systems, however, once more do not fit within the required automated process as explained in section 2.1.
CHAPTER 2
Design
2.1
Whole slide imaging system
The whole slide imaging system has to allow for the uploading and viewing of large images. The front end of the system will be inside ALEA Clinical. Firstly a user, with a session in ALEA Clinical, needs to be able to upload a large image to the whole slide imaging server. Once uploaded, a user has to be able to view the image in near real-time.
In order to view the image without having to load the whole image object, an image pyramid is used [15, 12]. An image pyramid consists of multiple layers (see figure 2.1). Each layer has the original image partitioned, with a higher partitioning for a layer with more detail. This allows for the sending of less images from a more partitioned layer, when it is not necessary to show the whole image. All images, except for the ones on the edge of a layer, will be of the same resolution. In order to create an image pyramid, a preprocessing step is executed.
After the preprocessing step, the images can be viewed from ALEA Clinical. Once the topmost layer is loaded, it has to be possible to navigate through the image. This is so the pathologist can find the sections he or she deems interesting. Translation is done by ways of holding and dragging the left mouse button. Zooming is done by scrolling. While zooming, the image pyramid level of detail can change.
Figure 2.1: 3 layer image pyramid illustration. Number of images per layer specified for each layer.
2.2
Uploading
In figure 2.2 the process of adding a new microscopy image is illustrated. The uploading is relevant to step 1 to 5 in the graph. When in ALEA Clinical (1), a user can start the upload of a new image. ALEA Clinical will keep a reference that the image exists, store this in the virtual slide viewer (2 and 3) and will redirect the upload to the whole slide imaging (4). Afterwards the user can commence uploading a new image (5). The status, stored in step 3 and 9, is there to let ALEA Clinical know whether an image is ready. In the same step 3, the session information is passed on to ALEA Clinical that the user needs to connect to the whole slide imaging system.
User ALEA Clinical Info and image upload
Preprocessor
Image pyramid
Image pyramid description Whole slide imaging server 1.Start 4.Session 2.Info 5.Upload 3.Status 6.Upload complete 7.Create images 8.Create pyramid description
9.Update status
Figure 2.2: The process of adding a new microscopy image.
2.3
Preprocessor
The preprocessor will have to convert an uploaded image into an image pyramid. These images are so large, that they can not simply be stored in memory to read and write. These operations will have to be done on the disk. In order to overcome this challenge, VIPS (VASARI Image Processing System) is used [2, 11]. VIPS allows for image processing of images with unlimited size. VIPS is mostly used in museum-driven applications for image mosaicing. VIPS divides images into subregions, so images can be processed as small sections rather than as single large objects.
The preprocessor, after the upload (6), comprises the last three steps of adding a microscopy image as illustrated in figure 2.2. Once the actual preprocessing (7) is done, a file describing the image pyramid and where to find each image file, is stored (8). The location of this file and status in ALEA Clinical (9) will be updated. From now on, the images can be viewed through the virtual slide viewer.
2.4
Virtual slide viewer
2.4.1
Specifications
The virtual slide viewer, integrated in ALEA Clinical, will allow for the viewing of a preprocessed image. The images should be loaded near real-time for common network connections. In order
to do so, the images are loaded from the image pyramid. The user experience during translation and zooming of the image should be smooth. As mentioned before, translation is done by holding and dragging the left mouse button and zooming is done by scrolling the mouse wheel. When switching between layers, a less detailed layer should be shown as long as the required layer is not yet loaded.
2.4.2
Considerations
The virtual slide viewer is written in JavaScript and HTML. This allows it to function across a wide range of platforms, as does ALEA Clinical.
Before the virtual slide can be viewed, a description of the pyramid is loaded. This is done so that the viewer knows where to find the images, what size the images are and the number of layers.
In order to prevent a black background from appearing in the virtual slide viewer while loading images in a not yet ventured area, the layer with the lowest detail is always shown. Since this image would be highly blurred when zoomed in, the layer above the current layer is also loaded. While this gives the appearance of loading detail faster, it is a trade off, since it costs up to a quarter more in terms of bandwidth. When zooming past layers, these layers will be loaded anyway. When these are cached, they will not be loaded again when the layer above the current layer is loaded.
2.4.3
Implementation
The process of viewing a virtual slide image is illustrated in figure 2.3. A user can initiate viewing a virtual slide image (1). When doing so, ALEA Clinical will return the location of the image pyramid description and a session to the user (2). This image pyramid description file and the JavaScript code for the virtual slide viewer are loaded from the server (3,4). The virtual microscopy viewer will be shown inside ALEA Clinical. In JavaScript the coordinates of the user in the whole viewer are kept and changed on scrolling and dragging events. When changed, the image is drawn. If an image is not yet loaded, it will be downloaded from the image pyramid (5,6). Depending on the zoom level the user is currently in, the corresponding level of detail is loaded from the image pyramid. The images shown in HTML and added or removed from JavaScript.
Image pyramid description User
ALEA Clinical
Image pyramid 1.Request
2.Session 3.Request pyramid description
4.Load pyramid description 5.Request images 6.Load images
Whole slide imaging server Figure 2.3: The process of viewing a microscopy image.
The virtual slide viewer works by keeping a record of the position and size of the full image, or in other words, where the image as a whole would be. When the user drags and holds the mouse, this full image is translated. Likewise when the user zooms, the full image width is changed and
it is translated to keep the image centred at the location of the mouse.
The following equations describe the translation of the image when the user drags and holds the mouse:
x = xold+ dx
y = yold+ dy (2.1) With coordinates (x, y), the previous coordinates (xold, yold) and a change due to dragging of (dx, dy).
The following equations describe the math behind zooming of the image when the user scrolls the mouse wheel. The first two equations describe the translation in order to keep the image centred on the mouse, the other two describe the change in width and height.
x = (xoffset/wold) ∗ (w − wold) + xold y = (yoffset/hold) ∗ (h − hold) + yold w = wold− m ∗ s
h = hold− m ∗ s
(2.2)
With size (w, h), previous size (wold, hold), offset coordinates to the left-top corner of the full image (xoffset, yoffset), mouse wheel movement m and scale of the image compared to the original s.
From the coordinates and size of the full image, it is possible to determine what the level of detail is and which of the images need to be shown. The images in the part that is currently shown, will be drawn and, if necessary, downloaded.
As mentioned before, other layers may also be loaded to make zooming seem more smooth. These images will be there, but behind the current layer. So when the current layer is loaded, the other layers will not be shown.
2.5
Hardware
Experiments will be run on the system in order to validate the model. The outcome of these experiments is subject to the hardware they are run on.
As of this time the whole slide imaging system is run on a Ubuntu 17.04 Virtual Machine. The server features a hard disk of 134 GB. The CPU is a single core Intel Xeon X5670 running at 2.93GHz. The RAM features one bank of 2 GiB. The server runs Apache 2.
The available bandwidth will determine how many users can simultaneously upload or view an image. In order to determine the available bandwidth, a bandwidth test was run using iperf. Since the throughput observed in iperf might be influenced by external factors, the test does not show the maximum throughput. It does however show a minimum of what is possible. If this minimum throughput exceeds the bandwidth needs for the system, it is safe to assume that the available bandwidth is enough. For this test iperf was connected with a public server in the Netherlands. During 10 tests, the average bandwidth was calculated to be 12.5 MB/s.
2.6
Model
In order to match hardware specifications to demand, the following model is proposed. The model answers how much storage space is required, how much bandwidth needs to be provided and the client side performance of the virtual slide viewer.
Firstly, since these images vary between 100MB and 10GB, the amount of storage space will be an important factor. Not having enough storage space available can impede the adding of new images. The storage space required is described in formula 2.3.
storage space = packages + code + slides ∗ (pyramid info + image pyramid) (2.3) The packages consist of the server software and VIPS. These are unlikely to be larger than one or two of these images. The number of slides is a parameter that is expected to change across studies or an increase in size of an existing study. The size of a pyramid is also subject to
change, this depends on the size of the input image. The pyramid description file however, does not make much of a difference. In testing for this trial, the size did not exceed a few megabytes. The size of the pyramid itself depends on the images used to create it. In testing with several microscopy images, the average size of the image pyramid created was of a factor 1.02 compared to the original image. This was tested by running six example microscopy images through VIPS. Bandwidth is important for the duration of uploading and performance of the virtual slide viewer. Both will compete for this resource. In adding a new image, the uploading will mostly use bandwidth and the preprocessing will mostly use processing power and some disk. The upload speed depends on the bandwidth available to the user. The maximum amount of uploads per day is known, the server must have enough bandwidth to supply for this. Bandwidth is also used by the slide viewer. This would result into the following formula:
bandwidth = max(uploads∗image size+viewer use∗average viewer bandwidth, server bandwidth) (2.4) Where uploads is the expected maximum number of new images added per day, image size is the expected image size, viewer use is the expected amount of users for the virtual slide viewer, average viewer bandwidth is the average bandwidth the virtual slide viewer needs and server bandwidth is the maximum bandwidth available to the server as explained in 2.5. Keep in mind the worst case scenario should be used for all parameters in order to be safe from running into bandwidth issues.
The behaviour of the bandwidth used for file transfer will be correct in this model until the maximum is reached. Thereafter the bandwidth will reach a high point, since the server can not transfer more data. The overhead created by the TCP/IP stack will make the bandwidth used decrease for each user past the maximum. This decrease will slow down once the used bandwidth approaches zero. In conclusion, after the maximum the linear equation used in 2.4 will be incorrect.
Finally the performance of the viewer itself can be described. Undoubtedly this depends on the architecture the user is running for the rendering time. For the latency and download time this depends on the quality of the connection to and load on the server. What can be considered are the limits of the virtual slide viewer with different maximum bandwidths. The following formula describes the time it takes to load an image:
time to draw image = rendering + TTFB + download time (2.5) Where rendering is the client side rendering time, TTFB is Time To First Byte and download time the time the file transfer takes.
To summarise, this model allows for the calculation of a required hardware configuration for given parameters depending on the clinical trial. Storage space needed can be calculated, depending on the size of the clinical trial. Bandwidth required can be calculated depending on the bandwidth use of uploading and viewing a virtual slide. Finally the performance of the viewer itself is dependent on the client architecture and on the connection to and performance of the server.
CHAPTER 3
Experiments
3.1
Outline
In order to evaluate the performance of the virtual slide viewer, the download and TTFB times are measured. This is done by recording a use of the viewer and storing the HTTP Archive (HAR) file from the Chrome Developer Tools [6]. This HAR shows information about each request done by the website. This includes the amount of time it takes to download and how long the browser has to wait before a download can commence (TTFB). This is measured with different bandwidth throttling. By measuring the performance of the slide viewer it can be deduced what the minimum connection that a client needs is and what the average bandwidth used by the virtual slide viewer is. By throttling the connection with different settings, we can find out at what point the performance will not be good enough. The different network settings are shown in table 3.1. The results are shown in figure 3.1 and 3.2.
Throttling Latency (RTT, ms) Download (MB/s) Upload (MB/s)
None - -
-WiFi 2 30 15
4G 20 4 3
3G (good) 40 1.4 0.74 3G (regular) 100 0.75 0.25
Table 3.1: The different network settings used in the testing.
It is important that the virtual slide viewer is still accessible while VIPS is converting a single image into a pyramid. In order to test the effect of VIPS on the server, the TTFB with and without VIPS running are compared. This is tested with a 300MB JPEG sample file. No throttling is done during this test, in order to prevent network related latency to affect the measurement. The result is shown in figure 3.3.
As mentioned before, the clinical trial this whole slide imaging system is being developed for, only features a single pathologist examining the uploaded slides. Despite that, it is important to test what happens when multiple users connect to the virtual slide viewer for future use of the system. In order to test the difference, an experiment is run scaling the number of users while measuring the throughput of image data the server is providing. In order to do so Apache benchmark is used. Apache benchmark is used to measure how quickly it can provide the data for a given request. The same image pyramid as in the tests before is used, with a single image out of the pyramid. The size of this image corresponds with the average size of the pyramid (14 KB). Both experiments are run on a LAN to prevent interference from other network activities or routing.
3.2
Results
Figures 3.1 and 3.2 show the performance of the image viewer in terms of download time. Figure 3.2 is the same as 3.1, but for a change in range of the y axis for readability. Down to a 4G connection the viewer loads all images within a second, however at 3G this time reaches far above one second. Interestingly there are some peaks in the download times. This is likely to be caused by either the difference in size of the images or that some images are downloaded with more other images downloaded at the same time than others. Since the test reproduces real behaviour through the virtual slide viewer, the number of images downloaded is not evenly distributed over time.
Figure 3.1: The time it takes to download the images for different connection speeds.
Figure 3.2: Same as figure 3.1 but with a scaled down y axis to more clearly show low-amplitude distributions.
The difference between using the virtual slide viewer with and without VIPS running seems to be minimal, as can be seen in figure 3.3. In both cases the TTFB does not exceed 100 ms. Furthermore, running VIPS seems to delay only around 20 ms in the worst case in this test. Converting the 300MB file, without using the virtual slide viewer, takes 2 minutes and 46 seconds.
Figure 3.3: The TTFB to the server with and without VIPS running.
Figure 3.4 features the average server side throughput measured for different number of connections. As expected, and mentioned in section 2.6, the throughput increases almost linearly until the limit of the server is reached. After which the throughput decreases and slowly starts to move towards zero, decreasing less on the way. The line showing 0.5 ∗ x is the outcome of the model for this server bandwidth and average slide viewer bandwidth use, as further explained in section 3.3.
Figure 3.4: Plot of the maximum bandwidth, outcome of the model by partially using equation 2.4 and the experiment using Apache benchmark. Where x in 0.5∗x is the number of connections.
3.3
Evaluation of model
The storage space would be the easiest part of the model to calculate for expansion of the system. For this trial, 11000 slides are expected to be used. In the worst case scenario, the original images can be up to 10 GB in size. This results in an expected 1.02 ∗ 10 = 10.2 GB size for the pyramids. So this would mean 11000 ∗ 10.2 = 112200 GB or over 112 TB of storage space required throughout the whole study. Since the images only need to be stored up to a week time and the worst case input is expected to be six images per day, so 42 images a week, this would require a maximum storage space of 42 ∗ 10.2 = 428.4 GB. In addition to the images, the operating system, packages and code currently account for 12 GB of storage space used on the server. Of course in real use, the worst case scenario is unlikely to be true. Average filesize is likely to be much smaller. Average image size of 500 MB, resulting in 510 MB pyramids, would required storage space of 42 ∗ 510 = 21420 MB or 21.420 GB. For this clinical trial the slides do not constantly have to be available. Once viewed they can be stored and recalled once required. Staying with this worst case scenario, the expected amount of image data uploaded each day is 60 GB. Despite this resulting in 0.7 MB/s in terms of average bandwidth use, it is more likely that uploading will cause bandwidth peaks of several MB/s or more per upload [10]. With up to six users, this can result in tens of MB/s in one peak. The virtual slide viewer, during the run from the above experiments, used 0.5 MB/s. For only six uploads per day and just one pathologist viewing the images, this is unlikely to cause server side issues. However if the uploads increase, a solution has to be found to handle the amount of uploads.
Figure 3.1 shows the download speed for the client can peak at 4 MB/s. When the client bandwidth is throttled below 4 MB/s, the download times will take more than tenths of seconds. As mentioned before, the available server bandwidth in this configuration is 12.5 MB/s and the average slide viewer bandwidth use is 0.5 MB/s. When using equation 2.4 and leaving out the uploads, 12.5 MB/s should be reached for 25 users. This is shown in the following equation:
bandwidth = viewer use ∗ average viewer bandwidth = 25 ∗ 0.5 = 12.5 (3.1) The following equation is plotted in figure 3.4 as 0.5 ∗ x:
The increase in throughput is what is expected in the model, the same can be said for the maximum throughput. The maximum throughput however seems to be somewhat below 12.5 MB/s. This is most likely because the overhead cause by the TCP/IP stack is already present.
CHAPTER 4
Conclusions
4.1
Conclusion
As seen by the results, the current system meets the performance demand for this clinical trial. The model will allow FormsVision to adjust the hardware configuration for future demand. In testing the current hardware configuration, the most likely problem to occur when more bandwidth is used, is running multiple connections of the virtual slide viewer.
Equation 2.3 should be used to calculate the required storage space. Each user requires a minimum of 4 MB/s for the virtual slide viewer. The expected uploads peaks per day should not exceed the rest of the available bandwidth. Each preprocessing step, using VIPS, currently needs an average of three minutes. If the average image size increases or processing power changes, this will have to be reevaluated. The sum of these times should not exceed the time available.
The expected behaviour of the bandwidth until the maximum is reached clearly shows in figure 3.4. However the behaviour after the maximum is not explained by the model.
4.2
Discussion
Given the parameters for the model, it is now possible to use the model for different hardware configurations. Bandwidth, storage space or processing power can be changed. The virtual slide viewer itself should be able to provide a similar experience as OpenSlide. However this whole slide imaging system allows for integration into ALEA Clinical and possible future extensibility. With only 12.5 MB/s of bandwidth available, handling a peak of uploads can cause an issue. Once several uploads are executed simultaneously, this is likely to exceed the 12.5 MB/s of available bandwidth. Subsequently the upload speed will slow down. This might also have an effect on the performance of the slide viewer. Six uploads at this speed, with the worst case size of 10 GB, can take just over an hour. It would be good advice to improve the bandwidth with larger demand or if the maximum upload time is too long.
Currently the hard disk on the server does not have enough storage space to account for the demand specified in 3.3. Before the deployment of the whole slide imaging system, it is necessary to upgrade the storage space on the current device or move to another device with enough storage space. With 122 GB of free space and a minimum of 428.4 GB required, an additional 306.4 GB has to be provided.
4.3
Future work
Due to the time constraints in this project, the virtual slide viewer software is quite simplistic. In order to reduce the bandwidth required for using the viewer, it might be possible to make some optimisation steps. Firstly, it is not always necessary to load the level of detail above the current level required as described in section 2.4.3. Detecting whether this might be needed can prevent
excessive bandwidth use. Secondly, no research is done in the rendering of the slide viewer itself. Using other frameworks like HTML5 Canvas or WebGL might be faster than plain HTML.
Further understanding and experimentation might improve the model for bandwidth. Cur-rently the model works until the maximum bandwidth the server can provide is reached. After-wards the TCP/IP stack creates overhead and less image data is transferred. This decrease in throughput is not modelled.
Bibliography
[1] FormsVision B.V. ”http://formsvision.com/”.
[2] John Cupitt and Kirk Martinez. VIPS: an image processing system for large images. In Electronic Imaging: Science & Technology, pages 19–28. International Society for Optics and Photonics, 1996.
[3] Andrew J Evans, Mohamed E Salama, Walter H Henricks, and Liron Pantanowitz. Imple-mentation of whole slide imaging for clinical purposes: Issues to consider from the perspec-tive of early adopters. Archives of Pathology & Laboratory Medicine, 2017.
[4] Adam Goode, Benjamin Gilbert, Jan Harkes, Drazen Jukic, Mahadev Satyanarayanan, et al. Openslide: A vendor-neutral software foundation for digital pathology. Journal of pathology informatics, 4(1):27, 2013.
[5] Peter Hufnagl, Hans Guski, J¨org Hering, Thomas Schrader, Klaus Kayser, Cornelia Tennstedt-Schenck, Manfred Dietel, and Klaus-J¨urgen Winzer. Comparing conventional and telepathological diagnosis in routine frozen section service. Diagnostic Pathology, 2(1):112, 2016.
[6] Google Inc. Chrome developer tools. ”https://developers.google.com/web/tools/ chrome-devtools/”, June 2017.
[7] Andrew K Graham Peter Schwarzmann Leong, F. Joel W.-M and James O’D Mcgee. Clinical trial of telepathology as an alternative modality in breast histopathology quality assurance. Telemedicine Journal and E-Health, 6(4):373–77, 2000.
[8] FJ Leong and JO’D McGee. Automated complete slide digitization: a medium for simulta-neous viewing by multiple pathologists. The Journal of pathology, 195(4):508–514, 2001. [9] Yu Liu, Kexiong Chen, Feng Gu, Jicheng Quan, and Guang Yu. Research and
implementa-tion of mass remote sensing image data storage and management. In Progress in Informatics and Computing (PIC), 2010 IEEE International Conference on, volume 1, pages 609–612. IEEE, 2010.
[10] TestMy Net LLC. Average upload per country. ”http://testmy.net/rank/countrycode. up/”, June 2017.
[11] Kirk Martinez and John Cupitt. VIPS-a highly tuned image processing software architecture. In Image Processing, 2005. ICIP 2005. IEEE International Conference on, volume 2, pages II–574. IEEE, 2005.
[12] Petter Ranefall, Alexandra Pacureanu, Christophe Avenel, Anne E Carpenter, and Carolina W¨ahlby. The giga-pixel challenge: Full resolution image analysis–without losing the big picture: An open-source approach for multi-scale analysis and visualization of slide-scanner data. In SSBA 2014, Symposium of the Swedish Society for Automated Image Analysis, 10-12 March, 2014, Lule˚a, Sweden, 2014.
[13] Marc M Triola and William J Holloway. Enhanced virtual microscopy for collaborative education. BMC medical education, 11(1):4, 2011.
[14] Jurriaan Tuynman and Thomas Koedam. Personal communication, May 4 2017. VUmc. [15] Antoine Vandecreme, Tim Blattner, Michael Majurski, Peter Bajcsy, Keana Scott, and
John Henry J Scott. From image tiles to web-based interactive measurements in one stop. Microscopy and Microanalysis, 21(S3):89–90, 2015.
[16] VUmc. COLOR III. ”https://rectalcancersurgery.eu/color-3-trial/ professionals/color-iii/”, June 2017.
[17] Ronald S Weinstein, Anna R Graham, Lynne C Richter, Gail P Barker, Elizabeth A Krupinski, Ana Maria Lopez, Kristine A Erps, Achyut K Bhattacharyya, Yukako Yagi, and John R Gilbertson. Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. Human pathology, 40(8):1057–1069, 2009.
