PROCEEDINGS OF SPIE
SPIEDigitalLibrary.org/conference-proceedings-of-spie
Color intensity projections with hue
cycling for intuitive and compressed
presentation of motion in medical
imaging modalities
Keith S. Cover, Frank J. Lagerwaard, Rudolf M.
Verdaasdonk
Keith S. Cover, Frank J. Lagerwaard, Rudolf M. Verdaasdonk, "Color intensity
projections with hue cycling for intuitive and compressed presentation of
motion in medical imaging modalities ," Proc. SPIE 8574, Multimodal
Biomedical Imaging VIII, 85740Q (13 March 2013); doi: 10.1117/12.2003495
Event: SPIE BiOS, 2013, San Francisco, California, United States
Color intensity projections with hue cycling for intuitive and
compressed presentation of motion in medical imaging modalities
Keith S. Cover*
a, Frank J. Lagerwaard
b, Rudolf M. Verdaasdonk
aa
Department of Physics and Medical Technology,
bDepartment of Radiotherapy,
VU University Medical Center, Amsterdam, The Netherlands
ABSTRACT
Color intensity projections (CIPs) has been employed to improve the accuracy and reduce the workload of interpreting a series of grayscale images by summarizing the grayscale images in a single color image. CIPs – which has been applied to grayscale images in angiography, 4D CT, nuclear medicine and astronomy – uses the hue, saturation and brightness of the color image to encode the summary information. In CIPs, when a pixel has the same value over the grayscale images, the corresponding pixel in the color image has the identical grayscale color. The arrival time of a signal at each pixel, such as the arrival time of contrast in angiography, is often encoded in the hue (red-yellow-green-light blue-blue-purple) of the corresponding pixel in the color image. In addition, the saturation and brightness of each pixel in the color image encodes the amplitude range and amplitude maximum of the corresponding pixel in the grayscale images. In previous applications of CIPs the hue has been limited to less than one cycle over the color image to avoid the aliasing due to a hue corresponding to more than one arrival time. However, sometimes in applications such as angiography and astronomy, in some instances the aliasing due to increasing the number of cycles of hue over the color image is tolerable as it increases the resolution of arrival time. Key to applying hue cycling effectively is interpolating several grayscale images between each pair of grayscale images. Ideally, the interpreter is allowed to adjust the amount of hue cycling in realtime to find the best setting for each particular CIPs image. CIPs with hue cycling should be a valuable tool in many fields where interpreting a series of grayscale images is required.
Keywords: color intensity projections, 4D CT, medical imaging, astronomy, maximum intensity projections,
hue cycling, flow imaging.
1. INTRODUCTION
Many modalities in medical imaging acquire images at a sequence of time points. Examples include angiography acquired both with 2D images in fluoroscopy, angiography acquired in 3D with CT, time of flight angiography in MRI and perfusion imaging in MRI.
Interpreting the huge number of images generated by such modalities is a major burden on radiologists and discourages their use. To interpret the images, the current practice of radiologist is to view the sequence of grayscale images produced by the modality as a short movie repeating it over and over again studying what stays the same and what changes.
A recent innovation, called color intensity projections (CIPs), is a method for summarizing the information in the sequence of grayscale images in a single color image with the hue, saturation and brightness of the color encoding the changes over the sequence [1-3]. Since its initial publication, the usefulness of CIPs has been demonstrated in angiography studies [2,4,5] and is being integrated in some of the standard angiography packages. Its usefulness has also been demonstrated in 4D CT. It has also been applied in other fields including astronomy [3] and is currently being applied to image airflow [6-8].
Two different versions of CIPs have been used – arrival time and percentage time. In arrival time CIPs the hue of the color (red-yellow-green-light blue-blue-purple) encodes the time that a signal arrives at a location in a sequence of images, for example, when contrast arrives at a location in angiography.
In percentage time CIPs the hue encodes the percentage time a signal spends at a location. For example, due to respiration-induced movement, tumor density may be present at a certain location only for part of the respiratory cycle e.g. at end-expiration, while at the more central regions tumor density will be present throughout the respiratory cycle. While both arrival time and percentage time CIPs can be applied to the same sequence of grayscale images, the preferred version depends on the specific application.
2. METHOD
In computers, colors are usually represented by three numbers that correspond to the intensity of the red, green and blue (RGB) components of the color. But an alternative representation is to use three numbers to represent the hue, saturation and brightness (HSB) of the color. The value of the hue usually ranges from 0 to 1 and corresponds to a continuous range of colors – red-yellow-green-light blue-purple. As hue is cyclic, the same hue corresponds to both 0 and 1. The saturation also ranges from 0 to 1. A saturation of 0 means the corresponding color is actually a shade of grey with its brightness controlled by the third number. A saturation of 1 means a pure hue. Thus an intermediate saturation is somewhere between grayscale and a pure hue.
The hue, saturation and color of the color of each pixel of the CIPs image can be calculated from the grayscale images on a pixel by pixel basis. For each pixel the CIPs equations are quite simple.
Brightness = MaxIP (1) Saturation = (MaxIP-MinIP) ⁄MaxIP (2) and
Hue = (index of the image of the maximum value for the pixel)/(images per hue cycle) (3) or
Hue = MeanIP/MaxIP (4)
where MaxIP is the maximum value of the corresponding grayscale pixels, MinIP is the minimum value of the corresponding grayscale pixels and MeanIP is the mean value of the grayscale pixels. Equation (3) is used for arrival time CIPs and equation (4) for percent time CIPs.
While obviously many different variations on the above equations are possible, for the problems considered so far, the above equations have been found to be sufficient.
2.1 Arrival time CIPs in Angiography
Digital subtraction angiography is a common medical procedure where contrast, a substance which shows up on x-rays, is injected into the human body. A rapid sequence of 2D or 3D images is often generated as the contrast flows through the body to map out the flow patterns. Because of technology limitations, images are usually acquired at a slower rate in 3D than in 2D and thus 2D is often the preferred modality.
The angiography data used to demonstrate CIPs is the same as the one used in Cover et al. [2]. A subset of these grayscale images, which were acquired in 2D, is shown in figure 1.
The patient was a 46-year-old man who experienced epilepsy as a result of a Spetzler-Martin Grade II arteriovenous malformation (AVM) with feeding from the middle cerebral artery in the left temporal lobe of the brain. Before the interpolation of the grayscale images the brightness of the grayscale images was reversed. The reversal resulted in the background color changing from white to black and the presence of contrast to cause pixels to become brighter rather than darker.
The reversal was implemented by first finding the maximum of all the pixels in the 29 grayscale images. Each of the pixel values in the 29 grayscale images was then subtracted from the maximum to get the new value for the reversed value of the pixel.
Before calculation of the arrival time CIPs, the 29 greyscale images were interpolated in time to generate a total of 113 images. The interpolation performed was only in time, no spatial interpolation was employed.
Two different CIPs images were calculated from the 113 grayscale images – one without hue cycling and one with. When no hue cycling is applied, each hue in a CIPs image corresponds to only one arrival time. But with hue cycling, hues are used cyclically so they each correspond to multiple arrival times.
The CIPs without hue cycling cycled through the full range of hues over 99 interpolated grayscale images. A value less than 113 was used because several of the early and late grayscale images had little information.
When hue cycling was applied to the 99 images, hue cycling had a period of 22 of the interpolated grayscale images. The number 22 was arrived at by trying a range of values and choosing the one that gave the most informative image. A smaller number than 22 yielded a large number of artifacts. A larger number did not present all the information available via hue cycling.
The hue was calculated from equation (3). The image index is 0 for the first image, 1 for the second image, and 112 for the last image. For example, if a pixel had the maximum value in the first image, with no hue cycling its hue would be red. For the last image the hue would be purple.
To display the CIPs hue-saturation-brightness image it is usually necessary to convert it to an RGB image. While there are a variety of different ways to accomplish this conversion the algorithm used for this paper was the HSB to RGB conversion routine in the Java Programming Language [9]. The saturation can also be amplified by multiplying the saturation by a constant after it has been calculated with the above equation.
For a sequence of images where the first image is completely black, such as DSA, the MinIP is always zero and thus change in intensity is always 100% so the saturation is also 100%. Thus for DSA, equation (2) always yields the value of 1. However, in the 4D CT example presented next, the saturation does vary over the CIPs image.
2.2 Percentage time CIPs in 4D CT
Four-dimensional (4D) CT generates a sequence of 3D CT images over a patient’s respiratory cycle. Because a single respiratory cycle has a much shorter time interval than the acquisition time of a sequence of 3D CT images acquisitions, 4D CTs are acquired in an interleaved fashion over many respiratory cycles and reconstructed post acquisition.
The patient used as an example was planned to undergo radiation treatment for Stage I non–small-cell lung cancer [1]. The 4D CT was acquired with the goal of determining the full range of tumor mobility during respiration. As the primary interest is where the tumor was located most of the time, percentage time CIPs is a better choice than arrival time CIPs.
3. RESULTS
3.1 Arrival time CIPs in Angiography
Comparison of the grayscale images in figure 1 with the arrival time CIPs images in figure 2 shows some of the advantages of CIPs over the grayscale images in interpreting DSA images, particularly in differentiating the feeding arteries and draining veins from the AVM nidus.
Figure 2a shows the CIPs without hue cycling. Examination of figure 2a shows contrast in red entering the image from the bottom, changing through yellow-green-light blue as it moves through the image and finally as blue as it exits the image a few seconds later at the bottom of the image. Thus, each hue corresponds to a unique time since the injection of the contrast.
Figure 2b shows the CIPs with hue cycling. Examination of the color contrast entering and leaving the image shows both to be purple. Thus the same hue corresponds to two different arrival times for contrast.
Examination of the AVM in figure 2a and 2b also shows the differences between applying hue cycling or not. In figure 2a the AVM is mostly yellow with a bit of orange in the feeding vessels and a bit of green in the draining vessels. However, examination of the AVM shows red through green as part of the AVM, presenting more information on timing on the flow of contrast through the AVM.
CIPs both without and with hue cycling have their advantages. If the goal is to outline the AVM for radiotherapy treatment the CIPs without hue cycling is likely the preferred images as the details of the internal flow is of little importance. However, if the internal flow of the AVM is of interest then the CIPs with hue cycling is likely to be more useful.
r
Figure 1. Selected images from a sequence of grayscale images from digital subtraction angiography showing contrast moving through vessels in a brain. The knot of vessels slightly to the right of the center of the image is the arteriovenous malformation (AVM) [2].
The usefulness of CIPs in DSA has been examined in two independent studies [4-5]. The comparison was between CIPs in combination with its grayscale images versus the current practice of just the grayscale images. In both cases the authors found solid results supporting the usefulness of CIPs and were enthusiastic about its application in clinical settings.
Figure 2. Arrival time CIPs image of the DSA grayscale images presented in Figure 1. Image a) without hue cycling and b) with hue cycling applied.
3.2 Percentage time CIPs in 4D CT
For the lung tumor example, the lower part of figure 3 shows a sequence of 10 grayscale images spread, with even spacing, over the patient’s respiratory cycle. Each of the 10 images shows the position of the tumor at a phase of the respiratory cycle. Each image is a small part of a larger 3D image set acquired at one of the 10 phases of the respiratory cycle.
For the percentage time CIPs, the full grayscale images over the 10 phases of the respiratory cycle were combined to generate the CIPs.
In contrasts to the DSA example, there were pixels in each of the grayscale images that had nonzero values. Thus the saturation of many pixels differed from 100%. Indeed, since the patient was lying stationary during the 4D CT, many of
the pixels had the same values in all of the grayscale images to within the noise. Thus, from equation 2, the saturation for many pixels will be close to zero. Thus the pixels will appear as grayscale in the CIPs image when they are constant in value over the grayscale images.
Examination of the CIPs image in figure 3 confirms this point. As the patient’s tissue was stationary in most pixels outside the lungs, most of the pixels outside the lungs in the CIPs look like the component grayscale images. This use of grayscale in CIPs, as well a providing a valuable context for the image information inside the lungs, also indicates the area were the tissue is not moving.
However, it should be kept in mind that there are cases in CIPs where moving tissue will show up as grayscale on a CIPs. CIPs is directly sensitive to change in pixel intensity, not tissue motion. If an object is of uniform intensity over a region, motion over that region will not result in any change in pixel intensity. Thus, the resulting area will appear grayscale in CIPs even though there is tissue motion.
Examination of the tumor in the CIPs image shows it is elongated over its range of movement just as would be expected from a maximum intensity image. The range of hue over the tumor is from green to yellow to orange. From the color scale, the green indicates the tumor spent about half it time in this location. The yellow-orange hue indicates the tumor spent only 10 to 20 percent of its time in this region, and the assessment of the time distribution of tumor location can be important, if for instance, if radiation treatment is restricted to certain periods of the respiratory cycle as in respiration-gated radiotherapy. The additional time-information of CIPs can also been seen in other anatomical structures such as the diaphragm, and to a lesser extent the lung vasculature.
Figure 3. The lower part of the figure shows a sequence of 10 grayscale images of a tumor moving in a lung over the respiratory cycle. The color image above the sequence of grayscale images shows the percentage time CIPs image of the whole chest that summarizes the 10 greyscale images. The moving tumor is clearly visible on the lower half of the lung on the right side of the image, with only a limited percentage of time that the tumor spends in the most caudal position, shown in yellow [1].
4. DISCUSSION AND CONCLUSION
Examples of both arrival time and percentage time CIPs have been presented for medical imaging applications. While much work needs to be done to determine the medical modalities where CIPs may be of use, the encouraging assessments of CIPs performance in angiography applications by independent researchers [4-5] bodes well for future applications.
Hue cycling was implemented by multiplying the hue value in equation (3) by a constant. In a similar way the saturation can be modified by multiplying the saturation value in equation (2) by a constant. Multiplying by a constant greater than the one is valuable when the hue does not standout against a grayscale background. Enhancing the hue can be valuable in cases such as perfusion imaging in MRI. In a similar vein, the brightness can be thresholded to suppress brighter objects allowing fainter objects to standout.
Of course the CIPs images do not completely replace their component grayscale images. The single image CIPs contains less information than the combination of the grayscale images. So while the CIPs images contain a valuable summary of the grayscale image in a single image, the grayscale images should always be available for more detailed examination of particular questions.
While a few examples of applications of CIPs have been discussed in the current paper, many other applications are possible, including perfusion MRI and flow studies in the medical field. Hopefully CIPs will provide an additional tool to help manage the tsunami of image data produced by modern instrumentation.
5. ACKNOWLEDGEMENTS
This work was funded by the VU University Medical Center in Amsterdam.
Disclosure: The authors’ employer, the VU University Medical Center Amsterdam, is pursuing patents on color
intensity projections covering applications in astronomy, medical imaging, and other fields. The first and second authors of this paper have a financial interest in the patents.
REFERENCES
[1] Cover, K. S., Lagerwaard, F. J., and Senan, S., “Color intensity projects: a rapid approach for evaluating four-dimensional CT scans in treatment planning,” Int. J. Radiat. Oncol. Biol. Phys. 64, 954 (2007).
[2] Cover, K. S., Lagerwaard, F. J., van den Berg, R., Buis, D. R. and Slotman, B. J., “Color intensity projection of digital subtracted angiography for the visualization of brain arteriovenous malformations,“ Neurosurgery 60, 511-515 (2007).
[3] Cover, K. S., Lagerwaard, F. J. and Senan, S., “Color Intensity Projections: A Simple Way to Display Changes in Astronomical Images ,” Publications of the Astronomical Society of the Pacific 119, 523–526 (2007).
[4] Strother, C. M., Bender, F., Deuerling-Zheng, Y., Royalty, K., Pulfer, K. A., Baumgart, J., Zellerhoff, M., Aagaard-Kienitz, B., Niemann, D. B. and Lindstrom, M. L,. Parametric color coding of digital subtraction angiography. AJNR Am J Neuroradiol 31, 919–924 (2010).
[5] Lin, C. J., Hung, S. C., Guo, W. Y., Chang, F. C., Luo, C. B., Beilner, J., Kowarschik M., Chu W. F., Chang C. Y., “Monitoring peri-therapeutic cerebral circulation time: a feasibility study using color-coded quantitative DSA in patients with steno-occlusive arterial disease,“ Am J Neuroradiol 33, 1685–1690 (2012).
[6] Merzkirch, W., [Flow visualization] Academic Press, New York (1987).
[7] Settles, G. S., [Schlieren and shadowgraph techniques: Visualizing phenomena in transparent media.] Berlin: Springer-Verlag, Berlin, (2001).
[8] Smits, A. J., Lim, T. T., [Flow visualization: Techniques and examples], Imperial College Press, London (2000). [9] Arnold, K., Gosling, J. and Holmes, D., [Java Programming Language 4th ed.] Prentice-Hall Englewood Cliffs, NJ,
