The effect of gamma and noise on perceived quality of X-ray
images
Citation for published version (APA):
van Overveld, W. M. C. J. (1993). The effect of gamma and noise on perceived quality of X-ray images. (IPO rapport; Vol. 952). Instituut voor Perceptie Onderzoek (IPO).
Document status and date: Published: 26/11/1993 Document Version:
Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.
• The final author version and the galley proof are versions of the publication after peer review.
• The final published version features the final layout of the paper including the volume, issue and page numbers.
Link to publication
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:
www.tue.nl/taverne Take down policy
If you believe that this document breaches copyright please contact us at: openaccess@tue.nl
providing details and we will investigate your claim.
Institute for Perception Research P.O. Box 513 - 5600 MB Eindhoven
Rapport no. 952
The effect of gamma and noise on perceived quality of
X-ray images
W.M.C.J. van Overveld
10/io 93/17 26.11.1993
The effect of gamma and noise on perceived
quality of X-ray images
W.M.CJ. van Overveld
Summary
As part of the IPO I Philips Medical Systems project "Perception of Medical Images", we have studied the influence of simultaneous variation of gamma and noise level on the per-ception of digital X-ray images. Gamma is the exponent of the function describing how digital gray values are transformed into luminance. The noise ·considered here is X-ray quantum noise.
Several levels of noise have been applied to three different angiographic images or "scenes" by means of computer simulation. After this, we have applied different gamma values to each of these images to simulate the effect of displays with different characteris-tics. The resulting images have been presented sequentially on a monitor (soft copy), where subjects had to assess the quality of each image using numerical category scaling. The same images have been printed on film (hard copy) and the films have been presented side by side on a view box. Again, subjects had to rate the quality of the images. Both experts and non-experts - regarding the amount of experience with reading of radiographs - participated in the experiments. Where possible, the results have been treated with Thurstone transformation and averaging per group of subjects in the same way as was done in an earlier experiment (cf. [10]).
The main conclusions of this study are the following.
• The optimum gamma varies per scene, for a fixed noise level.
• The optimum gamma shifts to a slightly lower value when the amount of noise is increased, for all scenes and all subjects.
• There are large differences between the results of individual subjects. These can be partly explained from the difference in experience in reading images (diagnostic qual-ity) and partly from personal preference (appreciation oriented qualqual-ity).
• The quality decreases with noise, for all scenes and all subjects. This effect is stronger for hard copies than for soft copies.
• The noise level is more important in subtracted images than in non-subtracted images, both for soft and hard copies.
• Differences between the perceived quality of soft and hard copies are due to the differ-ence in luminance range, which causes some saturation in dark regions on the soft copy when gamma is high.
1 Introduction
This study is concerned with the perceived quality of vascular X-ray images as displayed by the DSI system (Digital Spotfilm Imaging, developed by Philips Medical Systems). As was found in an earlier study (cf. [8], [9]), the perceived contrast of such images is an important criterion for their quality. We have studied the effect of the contrast parameter gamma in an earlier report [10]. Here we add another p~eter, namely noise. We study the combined effect of these parameters on image quality, to see whether the two are dependent and whether the relative importance of the two varies with the type of user or with image contents.
It is known from different application areas that we can distinguish between appreciation and performance oriented quality (cf. [13]). Appreciation oriented quality deals with aes-thetic aspects of images: are they visually pleasing or not? This type of quality can be regarded for any type of imagery: television, text, slides, medical images .... Performance oriented quality, on the other hand, can only be defined for images with which a task should be performed. One can think of medical images meant for diagnostic tasks, but a block of text (on screen or print) in which a user has to search certain items is another example (cf. [ 14 ]), as is the field of radar applications.
Both types of quality are affected by various physical parameters. As mentioned above, in this study we limit ourselves to the effect of gamma and noise.
In
previous studies (cf. [6], [11], [12]), the effect of contrast for medical images has been noted, although most studies were confined to performance oriented quality. The effect of gamma (or perceived contrast in general) on appreciation oriented quality has been investigated earlier for non-medical images like TV scenes and slides. Examples of such studies can be found in [13], [15], [ 17]. The effect of noise has been noted in several studies, but again these studies were mainly concerned with detection of details ([5], [2]) or they were confined to natural images [4]. The combined effect of different image parameters like noise and contrast has been studied in papers dealing with multi-dimensional scaling like [7]. Again these were restricted to natural images and thus to appreciation oriented quality.In
our study we looked at a combination of appreciation and performance oriented quality. Although the subjects expressed the perceived quality of an image in a single rating, the difference between appreciation and performance could be gathered from the remarks made by the subjects when they were asked which criterions they used for their quality judgment.The following sections describe the images used (Section 2), the theory of noise addition (Section 3), gamma transformations (Section 4), and the set-up and results of two experi-ments (Sections 5 and 6). Some conclusions can be found in Section 7.
Part of this work - to wit, the noise simulation, the preparation of the stimuli and the first part of the soft copy experiment - has been described in the report of the student N. Henault (cf. [3]) who was involved in the first phase of this project.
2 Image material
We used 1024 x 1024 pixel black and white images, of 8 bit grey values. Three angio-graphic images (referred to as "scenes") were used:
• "leg": a non-subtracted view of a leg just below the knee, where the thin blood vessels pass over the bone and over the soft tissue. This gives rise to very different types of contrast, all of which are diagnostically important.
• "kidney": a contrast image of the artery feeding the kidney. The kidney itself is filled with contrast liquid as well. Both the artery and its ramifications are important, but also the contour of the kidney against the background matters.
• "cerebral": a subtracted image of cerebral vessels. Very thin vessels are visible in this image, provided the contrast is high enough. The smooth bright background of this image is quite sensitive to noise.
We used a resolution of 1024 pixels instead of 512 as in [10], because we wanted to study the effect of noise, for which a sufficiently high spatial resolution is necessary. The images are shown in the figures below. We also show the grey value histograms of the interior parts of each image; i.e., the image without the rectangular shutters or the black area sur-rounding the circle. These histograms are used in the gamma transformations described in Section 4.
250
FIGURE 2. Grey value histogram of "leg".
FIGURE 3. Image "kidney".
14000
12000
10000
200
FIGURE 4. Grey value histogram of "kidney".
FIGURE S. Image "cerebral".
200000 160000 120000 80000 40000 0 " ... -0 50 100 150
FIGURE 6. Grey value histogram of "cerebral".
200 250
3 Noise simulation
3.1 Noise sources
The sources of noise in an X-ray chain can be divided into different categories. We distin-guish the following types (cf. [3]):
• X-ray quantum noise. These variations in intensity are due to the quantum nature of the X-rays. The statistical properties of this type of noise can be modelled by a Poisson dis-tribution, which implies that the standard deviation of the noise depends on the inten-sity of the signal. The amount of noise depends on the X-ray dose (number of quanta), the beam energy, and the detection quantum efficiency (the amount of absorption at the image intensifier). In our study, we varied the amount of X-ray noise through (simu-lated) variations of the X-ray dose (see next section for details about the simulation). • image intensifier noise. This results from inhomogeneities in the image intensifier
screen and gives rise to a fixed pattern of noise, or mottle, in different images. This type of noise is only dominant for X-ray doses significantly higher than those used in the system we have considered (DSI). Therefore we decided not to vary this type of noise in our study.
• electronic noise. This arises from the plumbicon camera recording the image and con-sists of both beam shot noise in the TV camera tube and preamplifier noise. This amount of noise depends on the working point of the camera and on the use of the dia-phragm. We did not vary this in our experiments, although we did use a certain amount of camera noise in the simulations because this introduces a high frequency component not present in the quantum noise (see next section). The statistical properties of the camera noise may be modelled by a Gaussian distribution.
• time jitter noise. The fact that certain characteristics of the imaging chain change in time causes a noise source when subtracted images are considered. The difference between two "identical" images (eliminating the fixed pattern mottle) taken at different times can be regarded as noise. We have not studied this type of noise since it is rather unpredictable and difficult to model.
3.2 Simulation program
For our purposes, X-ray quantum noise was considered to be the most important noise source. We decided to vary this amount of noise by simulating the effect of a varying X-ray dose. Details of the program that was used for the simulation can be found in [3]. To keep this report self-contained, we briefly present the outline of the simulation used (see Figure 7).
The inversion of the white compression is needed to change the grey values to the signal at the output of the image intensifier. At this level, two types of noise are added: the Poisson noise that simulates the X-ray photon noise and Gaussian noise simulating the electronic
camera noise. Note that the mean grey value (of the centre 2162 pixels of the image) at the output of the image intensifier can be related to the X-ray dose and thus to the number of photons per pixel; from this, the standard deviation of the Poisson noise is determined as the square root of the number of photons. The photon noise is filtered through the modula-tion transfer funcmodula-tion (MTF) of the optical system, but the electronic noise is not. There-fore the high frequency noise is mainly caused by the camera.
Test images of two phantoms were generated on an actual X-ray system, with varying amounts of noise. Simulated noise was added to the same images, to check if the noise simqlation program gave comparable results to the effect of true X-ray dose variation. After slight adjustment of the parameters in the algorithm (see [3]), the results were good enough, in the sense that a radiologist would assume that the different noise levels in the simulated images could indeed have been caused by variations in the dose.
original image invert white compression Add Add Gaussian Poisson noise noise
'.
- /"!"'\ \....J •l+
noise image 'rApply modulation transfer function of the imaging system
+
+
filtered Gaussian noisy image .,__ _ _ _ _ _ ~ -1"1-1-f"'\~-1----~ noise - \.LJ ~ image ' final imageFIGURE 7. Noise simulation scheme. See text of Section 3.2 for explanation.
3.3 Noise levels used
To get an impression of the relevant noise levels to be used in our experiments, we decided to ask the opinion of radiologists. We presented two radiologists of the St. Antonius hospi-tal in Nieuwegein with a large range of noise levels applied to each of the scenes. We asked them which noise levels were still acceptable to them (w.r.t. diagnostic usefulness) and which were not. Although one of them tolerated more noise than the other (see [3]), we were able to determine an average threshold value for the "aeceptability" of noise (in terms of the number of quanta per pixel) for each of the scenes.
This threshold value was chosen as the upper limit of the range of noise levels applied. The lower limit was the level where one just started to notice a difference with the original image, which was considered to be "noise free" because of the high dose used during the actual acquisition. Note that the limits were different for each of the scenes. Within the intervals defined by these limits, we chose four noise levels to be used in the experiments. These noise levels were logarithmically equidistant (the number of photons increased by a constant factor for each noise level). The levels we used are given in Table 1 below.
Leg Kidney Cerebral
130 ph/pix 260 ph/pix 1030 ph/pix
=45µR =90µR =370µR
65 ph/pix 130 ph/pix 410 ph/pix
=20µR =45µR = 150µR
32 ph/pix 65 ph/pix 163 ph/pix
= lOµR =20µR =55µR
16 ph/pix 32 ph/pix 65 ph/pix
=5µR = lOµR =20µR
TABLE 1. Noise levels applied to scenes "leg", "kidney" and "cerebral", in terms of X-ray dose (µR) and number of photons per pixel.
4 Gamma transformation
4.1 Transformation formulas
The parameter "gamma" has been varied in the same way as in [ 10]. The steps in the trans-formation are the following.
We define
(1) where g is the 8 bit digital grey value of an image pixel (the digitized video level coming out of an image intensifier/fV chain), yd is the gamma value of the display, and L is the luminance of the image pixel as it is displayed on the given display device (either a moni-tor screen or a film hanging on a view box). A given display can be calibrated - using a digital lookup table - such that it has the above characteristics, for a certain "f d· In our experiments, we have chosen 'Yd equal to 1.5, because this was closest to the actual display behaviour of our monitor.
For varying y IP (IP is short for "Image Processor": the Gould-DeAnza image processor we used for the transformations), we applied an 8 bit lookup table transforming the original grey values gin to new values gout as follows:
(2)
where the parameter P depends on the range of gamma values used in the experiment and on the grey value histogram of the original image. The factor P was introduced to keep the average luminance in the images constant while varying gamma. For the expression for P
and for further details of the transformations, the reader is referred to [10].
The values gout are fed to the display device. If we combine equations (1) and (2), we now find
(3)
with 1101 = 'Yip· 'Yd • Here C and C' are constants independent of"( IP • Using this
rela-tionship, we can vary "(tot through the parameter "f IP •
4.2 Gamma values used
As in the gamma experiment described in [10], we initially planned to use the same gamma range for all three scenes. However, during pilot experiments it became clear that a gamma value that was quite suitable for a cerebral image ("(1p
=
4) gave completelyunsatisfactory results for the leg image, and it would not be useful to use such a high gamma for the leg scene. Indeed, detail got lost in the dark regions where the vessel crossed the bone of the leg, when the gamma value was too high (note: a high gamma enhances detail in bright regions at the expense of detail in darker regions, as seen in [10)). For the above reason we arrived at individual ranges of gamma values which gave more or less "acceptable" images. Within each of these ranges, we chose four equally spaced val-ues of gamma. These valval-ues are specified in Table 2. Note that the valval-ues in Table 2 are values for
y
IP. in (3); these should be multiplied by a display gamma of 1.5 to find the totalgamma.
Leg Kidney
Cerebral
0.5 1.0 1.0
1.1 1.6 2.0
1.7 2.2 3.0
2.3 2.8 4.0
5 Soft copy experiment
5.1 Experimental set-up
For each of the original images described in Section 2, we first applied the noise addition with the values given in Section 3. Then we applied the gamma transformation for each of the resulting images, in the same way as described in [10]: given the grey value histogram of the noise corrupted images, the PQ transformation was computed using the histogram of the interior part of the image (discarding the black background). Hence we obtained 16 stimuli for each of the three scenes.
Note that the gamma transformation has been applied to the images after the noise simula-tion had taken place. This order of transformasimula-tions mimics the actual imaging system: first, noise is introduced in the acquisition part and then the effect of gamma takes place in the display part.
In the soft copy experiment, the images were shown on a high resolution colour monitor that was calibrated to have a gamma value of 1.5, in the same way as in [10]. The monitor was set in a room that was lit only by desk lamps directed to the wall behind the monitor, so that no direct reflections were visible on the monitor screen. The amount of light reflected from the back wall was 3 cd/m2. The viewing distance was 1 m, and the radius of the images on the screen was 25 cm so that the viewing angle was 14 degrees. Note that this viewing angle is much larger than that used in [10]: this turned out to be necessary because of the noise variations, which were not visible from the distance of 2 m as used in [ 1 O]. The larger viewing angle is also better in accordance with the practical situation in radiology departments. In fact, the same small viewing distance was used by the radiolo-gists who determined the relevant range of noise levels as described in Section 3.3. As in [ 10], we checked the calibration of the monitor by means of a test image which con-sisted of 64 squares with grey values 3, 7 , ... , 255. This image was processed with the same gamma transformations as used for the medical images. The luminance values of the squares in the resulting images were measured, to check if the characteristic had the form of (3). Graphs of these measurements are shown in appendix A.
The subjects' task was to judge the quality of each stimulus by assigning an integer number to it between 1 and 10. Quality was judged from a diagnostic viewpoint, as far as the subject could judge that, but cosmetic considerations were also taken into account. '1 O' corresponded to the highest quality and '1' to the lowest. Each image was presented for 7 seconds on the monitor and was followed by a uniform grey field that stayed on the screen for about 3 seconds. All 16 stimuli of one scene were shown in one session, and each stimulus appeared 4 times in the session. The order of these 64 images was chosen by a randomizing process and was different for each scene (but constant over the subjects). Before each session, the subject was shown a test sequence of 8 stimuli in which the extremes of the range of applied gammas and noise levels were present. Thus he/she could determine the variation of contrast and noise that occurred in the experiment, and could decide which image should get a "l"and which should get "10". Although the subject
assigned numbers to the stimuli in the test session, these numbers were not used in the analysis.
After the subject had judged all images, he/she was asked which criterions he had used. For instance, did the visibility of a single detail in the image determine the quality? Did the general appearance of noise in the background play a role? The answers to these ques-tions could be used to find the relative importance of appreciation and performance ori-ented quality. Some of the answers given by subjects will be discussed in the next section. Thirteen subjects have taken part in the experiment. Six of them are laymen with respect to medical images. We refer to this group of subjects as the "non-experts". The remaining seven subjects are so-called "technical experts": application engineers of Philips Medical Systems who are used to looking at radiographs from a technical point of view. All sub-jects had normal or corrected-to-normal visual acuity. None of them had experience with category scaling or with image quality experiments in general. No experts (radiologists) participated in this experiment because it was practically impossible to use the calibrated display in hospitals. The hard copy experiment, on the other hand, could be transferred to hospitals so that radiologists could participate more easily in that experiment.
5.2 Results
Using the above procedure, we obtained quality scores on a scale from 1 to 10 for each presentation of each stimulus and for each subject. We performed the following analysis of the data (compare to [10]). First of all, we modified the data per subject per session using Thurstone's law of categorical judgment, case V (cf. [18], [l]), to obtain average responses on a linear scale (interval scale). Figure Bl in the appendix shows the results after Thurstone transformation for each subject. The horizontal axis shows 'Yip and on the vertical axis the transformed value of quality is plotted, for each of the scenes. The param-eter in the curves is the noise level, in X-ray photons per pixel. Standard errors in the mean were usually between 0.2 and 0.3; they are not indicated in the plots.
In the figures we see that there are clear variations in quality judgment, both between sub-jects and between scenes. However, the differences between scenes are much larger than
· those between subjects. For this reason we reduced the amount of data by averaging over the subjects. As in [10], we distinguished between groups of subjects at different levels of expertise: non-experts and technical experts. Per group, the averaging was done in the same way as in [10]: computing z-scores per subject (subtracting the average over the stimuli for that subject and dividing by the standard deviation), computing the mean z-scores over the subjects and transforming these back to the original scale (adding the orig-inal grand mean and multiplying with the overall standard deviation). The resulting aver-age scores per group and per scene are presented in Figure 8 on paver-age 15. The standard error in the mean for the average scores per group of subjects is about 0.3. ·
From the figures, we can draw various conclusions. Firstly, we see that quality increases with the number of photons per pixel (i.e., decreases with the noise level). We also see that
Soft copy leg 6 non-exp
Soft copy leg 7 appl.
o 130 ~h/pix o D65 ph/pix 130 ph/pix
D65p~IX 2 A32 ph/pix
•32p~ix
• 16 p pix ~ 16 ph/pix
1.4 .8 1.2 1.6 2 2.4 1
gamma .4 .8 1.2 1.6 2 2.4
gamma
Soft copy 7 Soft copy
kidney 6 non-exp kidney 7appl.
~ 6 (ij ~ ::> C' (ij ::> C' 0260 ph/pix 0260 ph/pix D 130 ph/pix
c 130 ~h/pix 4 A65 ph/pix
Ja.65 p fo'x ~32 ph/pix
•32 ph pix
1 1.4 1.8 2.2 2.6 3 1 1.4 1.8 2.2 2.6 3
gamma gamma
Soft copy Soft copy
cerebr 6 non-exp cerebr 7appl. o 1030 ~h/pix c410 p~rx o 1030 ph/pix ~7 •65 p pr Ja.163 ~ pi 7 D 410 ph/pix A 163 ph/pix (ij ::> O' 1 1.5 2 2.5 3 gamma 3.5 4 4.5 ~ c;; ::> O' ~65 ph/pix 4.+-.-..---....---..---.--... .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma
FIGURE 8. Quality as a function of gamma, per group of subjects ("appl" = technical
experts) and per scene. The parameter is the number of X-ray photons per pixel.
there is an optimum value of gamma for each scene, which varies not only per scene but also (less significantly) per group of subjects. The observations about gamma are the same as in [1 O]. Differences between scenes can be explained from the remarks made by the subjects: e.g., most technical experts said that the visibility of the vessel crossing the bone was an important quality criterion for scene "leg". As the visibility of this detail decreased with gamma, they chose the lowest gamma as the optimum one. Since noise did not affect the visibility of this vessel, noise has relatively little effect on the quality of this scene. The non-experts did not use such specific details for their judgment; but if the bone of the "leg" scene became too dark (high gamma), they found the image less appealing.
Differences between groups of subjects are visible in scenes "leg" and "cerebral". For "leg", the technical experts preferred a slightly lower gamma (because of the visibility of the dark vessel on the dark bone, as mentioned above) and for "cerebr" they preferred a higher gamma. The high gamma in scene "cerebr" caused the very thin vessels to stand out from the background, which was desirable from a diagnostic point of view. non-experts chose a lower gamma because they did not want the image to be too much "black-and-white" (appreciation oriented quality).
Although the value of the optimum (per group) remains constant when noise is varied, the effects of gamma and noise on quality are not independent. If they were independent, this would imply that all curves in one plot would be parallel to each other.
In
our plots this is not the case. This is especially clear when the data for the lowest two gamma values in each plot are studied. In the curves with the diamond marker (highest noise) the difference between these two points is much less than in the curves with the circular markers (lowest noise). In other words: the images with lowest gamma values are judged to be relatively better, compared to the optimum, when the noise level is increased. In the case of the tech-nical experts and the "leg" image, the optimum gamma even appears to shift from 1.1 for the lowest noise level to 0.6 for the higher noise levels (and probably the "true" optimum for these noise levels has an even lower gamma value, not used in the experiment).At the high end of the gamma axis, the dependency between noise and gamma is less clearly visible. For most of the scenes and subject groups, the curves are more or less par-allel at the right end of the scale. Only for the cerebral scene and the technical experts·, we see that the quality drops faster with increasing gamma when the noise is more apparent. We also see that noise plays a larger role for the subtracted image ("cerebr") than for the non-subtracted ones: the curves in the "cerebr" graphs are further apart than those for the other two scenes. This is in accordance with the remarks of subjects, who were bothered by the noise in the bright background of the subtracted image (especially for high gamma). From the plots in the appendix it can be seen that the above mentioned observations also hold for individual subjects, in general. Still we see large differences between the curves for separate subjects. Not only do we find different optimum gammas for different sub-jects, but we see differences in weighting of the importance of noise and gamma as well. These differences can be partially explained by the subjects' remarks. Some of them found that high noise levels were annoying (especially in "cerebr"), and others did not mention the noise at all, but only looked at the local contrast of relevant details.
6 Hard copy experiment
6.1 Set-up
In this experiment, we used hard copies of the same images as used in the soft copy exper-iment. As in the gamma experiment [ 10], these were printed on an Agfa laser imager that was again calibrated to a grey value to luminance characteristic with a gamma of 1.5. For the non-experts, the images were viewed on a light box with a luminance of 2500 cd/m2.
To simulate an average amount of ambient light that can be found in reading rooms, we used a desk lamp that lit the wall behind the view box. The wall reflected approximately 70 cd/m2, and 1 cd/m2 reflected from the front of the view box as a result of this. We cov-ered all parts of the view box where there were no images hanging, so that these did not contribute to the ambient light. The viewing distance was approximately 50 cm and the radius of the images was 12 cm. Thus the viewing angle was 13 degrees. The viewing cir-cumstances for the other subjects varied a little, depending on the view boxes available at the sites at which the experiment was carried out.
As in the soft copy experiment, the subjects had to judge the quality of the stimuli in sepa-rate sessions. The task was slightly different from that in the soft copy experiment, because images had to be judged simultaneously: all 16 stimuli corresponding to one scene were mounted together each other on a view box, in 4 rows of 4 images, in random order. Subjects had to express the quality in a number from 1 to I 0 like in the soft copy experiment. We used six nonexpert subjects, three of whom had also participated in the soft copy part of the experiment, and the seven technical experts that had done the soft copy experiment. Furthermore, five radiologists participated in this experiment.
6.2 Results
The data of the scaling task could not be submitted to Thurstone scaling, since subjects rated each stimulus only once. We can, however, average the results for a group of sub-jects (experts, technical experts, or non-experts) by z-transformation in the same way as described in Section 5.2. The results of this averaging are shown in Figure 9 on the next pages; the data for all individual subjects are plotted in Figure B2 in the appendix. The standard error in the mean for the average scores per group of subjects is about 0.5. The conclusions that can be drawn from these graphs are similar to those of the first exper-iment. Again the quality decreases with the noise level, the optimum gamma depends on the scene and lower gamma values are relatively better when the noise increases. As before, the noise level plays a larger role for the subtracted image than for the non-sub-tracted images, although here the difference between subnon-sub-tracted and non-subnon-sub-tracted images is less large. In general, noise appears to be more important for -hard copies than for soft copies. We will discuss some of the reasons for this in Section 7.
~ 'iii ::J tT 1 Hard copy leg 6 non-exp 0130 ph/pix c65 ph/p1x •32 phfpix +16 phfpix .8 . 1.2 1.6 2 2.4 gamma o260p~ix c 130 ~ pix 465p~IX +32 p pix 1.4 1.8 2.2 2.6 3 gamma Hard copy cerebr 6 non-exp 1.5 2 2.5 3 3.5 4 4.5 gamma ~ 'iii ::J tT ~ 'iii ::J tT Hard copy leg 7 appl.
o
130 ph/pix D65 ph/pix A 32 ph/pix ~ 16 ph/pix 2'+---... --__, ... __ __, __ ...,.._ .4 .8 1.2 1.6 2 2.4 gamma 8 7 5 0260 ph/pix D 130 ph/pix A 65 ph/pix ~ 32 ph/pix 4 1 1.4 1.8 2.2 2.6 3 gamma 8 7o
1030 ph/pix D 410 ph/pix 4 A 163 ph/pix ~65 hi ix .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma.4 ~.,. § O" Hard copy leg Sexp. 0130 ph/pix
ass
ph/p1x 1t. 32 phfpix • 16 phfpix .8 1 1.2 1.6 gamma Hard CODY kidney 5 exp. 2 0260 ph/pix a 130 phfpix 1t. 65 ph/p1x • 32 phfpix 1.4 1.8 2.2 2.6 gamma Hard copy cerebr 5 exp. o 1030 ph/pix c410 ph/p1x 1t. 163 phfpix • 65 ph/p1x 2.4 3 .5 1 1.5 2 2.5 3 3.5 4 4.5 gammaFIGURE 9. Quality as a function of gamma, per group of subjects ("appl"
=
technical experts) and per scene. The parameter is the number of X-ray photons per pixel.The fact that we find no significant differences between groups of subjects is mainly due to the large individual differences between subjects. In the appendix, it can be seen that the individual differences are larger than in the soft copy case; the fact that each image is judged just once instead of four times explains part of the differences. After averaging the results of the individual subjects within groups, the differences disappear so that we can-not draw conclusions about the perceived quality for non-experts as compared to technical or medical experts.
Some information could be obtained from the remarks made by the subjects. As in the soft copy experiment, the non-experts more or less scanned the whole image to see if several details (both blood vessels and bony detail) were visible, but the general impression of "greyness" or "noisiness" was also mentioned quite often. Technical experts mainly looked at details: the thin vessels over the bone and soft tissue in the "leg" scene, various aspects of the "kidney" scene (thick and thin vessels, the periphery of the kidney tissue, the spine, ... ), and the very thin vessels in the scene "cerebr". The radiologists looked at similar details, although there were some differences: in the "leg", they looked mainly at the vessel crossing the soft tissue because that was the problem area for this patient; in the "kidney", they were largely interested in the thin vessels. Most of the experts mentioned that a large amount of noise lowered the diagnostic quality of the images (mainly in "kid-ney" and "cerebr"), because the relevant details were less clearly visible. Surprisingly enough, the different quality criterions of the groups of subjects are not reflected in the graphs of their quality ratings.
7 Conclusions
In this final section we compare the results of the two experiments and make some general concluding remarks. Note that it is not a priori clear that we may compare the results of the two experiments: we used "sequential scaling" (one image at a time) in the soft copy experiment and "parallel scaling" (all stimuli of one scene together) in the hard copy experiment. In [16], however, it has been verified that the two approaches give comparable results.
First, we look at the effect of noise. Both studies show that the perceived quality increases with the number of photons per pixel. This effect is more important for hard copies than for soft copies. The difference is probably due to the fact that all images of one scene can be directly compared on a hard copy, so that small differences in noise are noted more eas-ily than in the case where one image at a time is shown on a monitor. A difference in MTF between the hard and soft copy displays may also explain the fact that noise is more clearly visible on a hard copy. In fact, this difference has been found before (cf. [8],[9]). The subtracted image "cerebr" is more severely degraded by noise addition than the two non-subtracted images. This degradation of "cerebr" is mainly due to the appreciation ori-ented component of quality, since the noise is mostly visible in the background where there are no diagnostic details. Only for the highest gamma value in combination with the highest noise level, some of the smallest vessels are masked by the noise, so that perform-ance oriented quality comes into play.
In general, noise mostly affects the appreciation oriented quality; not only in the back-ground of the "cerebr" scene but also in the other two scenes. This follows from the remarks made by the expert subjects, who admitted that the relevant details were still visi-ble in a rather noisy background, but they did not "like" the appearance of the noise. As mentioned above, there are a few exceptions to this: at the maximum noise level in "cer-ebr" and "kidney", it becomes difficult to see some of the diagnostic details.
As for the influence of gamma: we have seen that there is an optimum gamma value for each scene and each subject. The differences between subjects are rath~r large, but the dif-ferences between scenes are even larger (note that we could not even use the same range of gamma values for all scenes in our experiments).
Our findings show that perceived quality as a function of gamma behaves in a very differ-ent way for soft and hard copy images. This is because of saturation in the dark regions of the image. As seen in the appendix, the fact that the dynamic range of the soft copy is much smaller than that of the hard copy gives rise to a much more severe saturation at low grey values. This is quite noticeable for high gamma values: in an image with a uniform grey value distribution such as the test image, grey values less than 100 will all be mapped to 0 if the gamma value is sufficiently high. If the image contains important information in parts where all grey values are less than 100, this information will disappear and the qual-ity will drop. As can be seen in the histograms of the three scenes (Figures 2, 4, 6), the kid-ney and leg images indeed contain information in the low grey values. Although we did not apply the maximum gamma (y
1p= 4.0) for these images for exactly these reasons
still some saturation took place for the highest gamma values applied. For the leg scene, the effect on quality was strongest because there the information in the low grey values was more important than for the kidney scene.
Gamma influences the appreciation oriented quality, as mentioned by some of the subjects. If gamma is too low, the subjects judge the image as "too fiat" or "gray". Subjects also mention that the image looks blurred when the gamma is too low (note that the perceived sharpness of the stimuli is affected by varying the gamma only - the spatial resolution is unchanged!). If gamma is too high, subjects may be bothered by the appearance of very bright or very dark regions in the image, which makes the image "tiresome" to look at -and even more so if this is combined with a high noise level.
Gamma also influences the performance oriented quality; much more than noise does, at least in the range of gamma and noise values applied by us. The effect of gamma on per-formance oriented quality is obvious in the few cases where information is lost on the soft copy due to a gamma value that is too high. But also when no saturation occurs, the (expert) subjects are concerned with the visibility of details that have very low contrast with their surroundings. Only if these details are visible - which exclusively depends on the gamma value, as long as the noise is not too severe - the performance oriented quality is high.
Finally, we have noted a small but systematic interaction between the effects of gamma and noise. For all scenes and subjects, the optimum gamma is slightly influenced by the noise level. At the highest noise levels, there is tendency towards lower gamma values.
This is consistent with the remarks made by the subjects, who complained about the noise only at high gamma values. Both performance and appreciation oriented quality are affected in this case.
Unfortunately, our results are obscured by the large inter-observer variations. Part of the variability could be lowered by running the hard copy experiment more than once with each subject, so that the accuracy of the hard copy results could be compared to that of the soft copy experiment where we had four replications. However, all of the expert subjects and many of the technical experts were quite determined about which image they liked best; they would have ranked the stimuli very similarly to the first time if the experiment were repeated. The non-experts seemed to be less certain of their judgments, in general. Yet they show a fairly small standard error in the mean for the soft copy experiment which suggests that the hard copy experiment, too, could be repeated without getting very differ-ent results. We can only conclude that the personal preference or "taste" of subjects for the appearance of contrast and noise in X-ray images plays a role that cannot be neglected when trying to design an "optimum" display device for such images.
References
[ 1] M. C. Boschman, "THURCATD: Een programma voor de verwerking van resultaten van categorieschalingsexperimenten: Berekening van parameters in bet Thurstone model - conditie D", IPO manual no. 108, 1991 (in Dutch).
[2] A. E. Burgess and R. F. Wagner, "Detection of bars and discs in quantum noise". SPIE, Vol. 173, pp. 34-40, 1979.
[3] N. Henault, "Noise influence on the perceived quality of digital X-ray images", report of Universite de Technologie de Compiegne, France, Biomedical Engineering, 1993. [4] T. S. Huang, "The subjective effect of two-dimensional pictorial noise", IEEE
Trans-actions on Information Theory, pp. 43-53, January 1965.
[5] V. Klymenko and R. E. Johnson, "Visual increment and decrement threshold curves as a function of luminance range and noise in simulated computed tomographic scans", Investigative radiology. Vol. 27, pp. 598-604, 1992.
[6] H. MacMahon, C. E. Metz et. al., "Digital chest radiography: effect on diagnostic accuracy of hard copy, conventional video, and reversed gray scale video display for-mats", Radiology vol. 168 no. 3, pp. 669-673, 1988.
[7] H. Marmolin and S. Nyberg, "Multidimensional scaling of image quality", FOA report C 30039-H9, Swedish national defence research institute, Stockholm, 1975. [8] W. M. C. J. van Overveld, "A study on perceived quality of digital X-ray images
based on interviews with radiologists in The Netherlands, Switzerland and Great Brit-ain", IPO report no. 861, 1992.
[9] W. M. C. J. van Overveld, "A study on X-ray image quality based on interviews with radiologists in the USA", IPO report no. 921, 1993.
[ IO]W. M. C..J. van Overveld, "The effect of gamma on subjective quality and contrast for X-ray images", IPO report no. 907, 1993.
[ll]S. M. Pizer, R. E. Johnston, D. C. Rogers and D. V. Beard, "Effective presentation of medical images on an electronic display station", RadioGraphics, vol. 7, no. 6, pp. 1267-1274, 1987.
[12]S. M. Pizer, J.B. Zimmerman and E. V. Staab, "Adaptive grey level assignment in CT scan display", Journal of computer assisted tomography, vol. 8, no. 2, pp. 300-305, 1984.
[13]J. A. J. Roufs, "Perceptual image quality: concept and measurement'!, Philips Journal of Research, vol. 478, no. l, pp. 35-62, 1992.
[14]J. A. J. Roufs and M. C. Boschman, "Visual comfort and performance", in Vision and Visual Dysfunction, gen. ed. J. R. Cronley-Dillon, vol. 15: The Man-machine Inter-face, ed. J. A. J. Roufs, pp. 24-40. 1991.
[15]J. A. J. Roufs and A. M. J. Goossens, "Perceived quality and contrast as a function of gamma", IPO Annual Progress Report 23, pp. 65-70, 1988.
[16]S. Spiertz and F. Rademakers, "Hard-copy experiment: vergelijken van schalen en ordenen en van hard- en soft copy", IPO report no. 903, 1993 (in Dutch).
[ 17]A. A. A. M. van Tongeren, "De perceptieve beeldkwaliteit van complexe scenes als functie van de luminantie-overdrachtsexponent gamma", IPO report no. 710, 1989 (in Dutch).
[18]W. S. Torgerson, Theory and methods of scaling, John Wiley and Sons, New York, 1958.
Appendix A: Effect of display calibration
The figures below show the characteristics of the monitor and hard copy unit after gamma correction, for 'Y tot values of 1.5 and 6.0 ('YIP
=
1.0 and 4.0). The measurements were madeon a test pattern of squares with increasing grey values.
1000
-
C\I~
100 ~g
as c·e
10 ::I ...JSoft copy characteristics
gamma= 1.5 gamma=6.0
..
..
....
-·
..
~-...:. ... ·--·-·,f".•.
.··
..
1+---....---...
50 100 1 0 200 2 0 grey valueFIGURE Al. Luminance transfer curves of the monitor after gamma correction.
1000 -1000
~
~B
100 c as c·e
::I ..J 10 1 0Hard copy characteristics
gamma= 1.5 gamma=6.0 .'
l
·---··-·
....
·
•',
.
,.
•' • .l,,
,.
•' 1••',.
•' •' ,.•'...
..
···
···"
...
~ 50 100 150 200 250 grey valueFIGURE A2. Luminance transfer curves of the hard copy unit after gamma correction.
Except for the horizontal parts of the curves, the measured values do correspond to the relationship given in Eq.(3) with a
y
tot value of 1.5 and 6.0, respectively. The horizontalparts are due to the restricted dynamic range of the display system. It can be seen that the soft copy suffers more from this restriction than the hard copy does: more clipping takes place in the soft copy curve. This accounts for the differences found between judgments of the soft and hard copies as noted in the conclusion (Section 7).
Appendix B: Results of individual subjects.
The following pages show the graphs of all subjects that participated in the two experi-ments. Figure B 1 shows the results of the soft copy experiment, and Figure B2 shows the graphs of the hard copy experiment. The figures are grouped as follows: first the graph_s of all non-experts are shown, then the graphs of the technical experts, and finally (for the hard copy experiment) the graphs of the expert subjects are given. Within each group of subjects, we first plot all individual results for scene "leg", then for scene "kidney" and then for "cerebr".
No error bars are plotted. For the soft copy experiment, where we used four replications, we found a standard error in the mean of around 0.2 to 0.3 in all cases. In the hard copy experiment we did not have any replications so that no error bars can be given.
The average results of the subjects within a group (non-experts, technical experts, and experts) can be found in Sections 5 and 6, for soft and hard copies, respectively.
1 1 Soft copy leg IVO 0130 ph/pix a65 ph/p1x •32 phfpix 1 • 16 hfoix .4 .8 1.2 1.6 gamma .8 Soft copy leg gvl 0130 ~h/pix ass p ro1x •32p~ix •16p pix 1.2 gamma o 130 ph/pix a65 ph/p1x &32 phfpix • 16 phfpix .8 1.2 1.6 1.6 gamma 2 2 2.4 Soft copy leg hs 0130 ph/pix ass ph/p1x •32 phfpix •16 phfpix .4 .8 1.2 1.6 gamma o 130 phlpix ass ph/p1x .6. 32 phfpix • 16 ph/pix .4 .8 1.2 gamma Soft copy leg ti< 0130 ph/pix D65 ph/plX &32 phfpix • 16 phfpix .8 1.2 gamma 1.6 1.6
FIGURE Bl. Soft copy results. A: non-experts (scene "leg").
2 2.4
2 2.4
Soft copy kidney IVO 0260 ph/pix D130 phfpiX • 65 ph/p1x • 32 phfpix 1 1.4 1.8 2.2 gamma Soft copy kidney gvl 0260 ph~ix D130 ~pix .l65 p~IX •32 p pix 1 1.4 0260 ph/pix D 130 phfpix .6 65 ph/p1x o 32 ph/pix 1.8 2.2 gamma Soft copy kidney nh 2.6 2.6 3 3 1+----.---.--..---.----...---~:...-. 1 1.4 1.8 2.2 2.6 3 gamma 8 7 2 1 7 0260 ph/pix D130 ph/pix A65 ph/p1x •32 ph/pix Soft copy kidney hs 1 1.4 1.8 2.2 2.6 0260 ph/pix c 130 ~hlpix .6 65 p PIX o 32 ph/pix 1 1.4 0260 ph/pix D130 ph/pix A65 ph/pJX •32 phfpix 1 1.4 gamma Soft copy kidney he 1.8 2.2 2.6 gamma Soft copy kidney · rk 1.8 gamma 2.2 2.6
FIGURE Bl. Soft copy results. A: non-experts (scene "kidney").
3
3
3
Soft copy cerebr 1vo 2.5 3 3.5 4 4.5 gamma Soft copy cerebr gvl o 1030 ph/pix D410 ph/plX • 163 phfpix •65 ph/p1x 1 1.5 2 2.5 3 3.5 4 4.5 gamma 1 o 1030 ph/pix D410 ph/plX • 163 ph/pix •65 ph/p1x 1 .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma o 1030 ph/pix a 41 O ph/p1x • 163 phfpix •65 ph/p1x Soft copy cerebr hs .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma o 1030 ph/pix D410 ph/p1X 4165 ~h/pix 7 • 65 p pix Soft copy cerebr he 1 1.5 2 2.5 3 3.5 4 4.5 gamma o 1030 ph/pix D410 ph/prx • 163 ph/pix •65 ph/p1x Soft copy cerebr rk 1 1.5 2 2.5 3 3.5 4 4.5 gamma
o 130 ph/pix c65 ph/p1x •32 ph/j)ix •16 phfpix Soft copy leg be 1.2 1.6 gamma Soft copy leg rve .4 .8 1.2 1.6 gamma o 130 ph/pix D65 ph/p1x 1 • 32 ph/pix • 16 ph/pix .4 .8 1.2 gamma 1.6 2 2 2.4 .4 .8 o 130 ph/pix 2 C65 ph/pix 1 A 32 ph/pix ~ 16 ph/pix .4 .8 0130 ph/pix D65 ph/plX •32 ph/pix • 16 phfpix 1.2 1.6 gamma Soft copy leg ha 1.2 1.6 gamma 2.4 1.4 .8 1.2 1.6 gamma
FIGURE Bl. Soft copy results. B: technical experts (scene "leg").
2 2.4
2 2.4
2 2.4
~ a; ::I CT ~ a; ::I CT 8 7 4 Soft copy leg pg o 130 ph/pix D65 ph/plX A 32 ph/pix • 16 ph/pix .8 0260 ph/pix D130 ~h/piX A65 p PIX •32 ph/pix 1 1.4 0260 ph/pix D 130 ph/pix A65 ph/pix ~32 ph/pix 1 1.4 1.2 1.6 gamma 1.8 2.2 gamma Soft copy kidney 1.8 2.2 gamma 2 2.4 2.6 3 ha 2.6 3 ~ a; ::I CT 7 ~ a; ::I CT 7 0260 ph/pix a 130 phlpix A 65 ph/plX • 32 ph/pix 1 1.4 0260 p~ix D130 ~ pix A65p~IX •32 p pix 1 1.4 Soft copy kidney be 1.8 2.2 2.6 gamma Soft copy kidney Ive 1.8 2.2 2.6 gamma Soft copy kidney evr 0260 ph/pix D130 ~h/pix •65p pix • 32 ph/pix 1 1.4 1.8 2.2 2.6 gamma
FIGURE Bl. Soft copy results. B: technical experts (scene "leg", "kidney").
3
3
7 7 0260 ph/pix D130 ph/pix A65 ph/p1x • 32 ph/pix Soft copy kidney rte 1 1.4 1.8 2.2 2.6 gamma o 1030 ph/pix C410 ph/pix A 163 ph/pix • 65 ph/pix Soft copy cerebr be 3 .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma o 1030 ph/pix D410 ph/plX A 163 ph/pix •65 ph/p1x 1.5 2 2.5 3 3.5 4 4.5 gamma 7 7 0260 ph/pix a 130 ph/pix A 65 ph/pix • 32 ph/pix 1 1.4 1.8 2.2 2.6 gamma Soft copy cerebr mvo 3 ~6 a; ::i CT 5 o 1030 ph/pix D410 ph/pix ~ 163 ph/pix +65 ph/pix .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma
FIGURE Bl. Soft copy results. B: technical experts (scene "kidney","cerebr").
1 3 3.5 4 4.5 gamma Soft copy cerebr pg ~~---o o 1030 ph/pix D410 ph/pix A 163 ph/pix o 65 ph/pix 1 1.5 2 2.5 3 3.5 4 4.5 gamma 1 o 1030 ph/pix 4 D 410 ph/pix A 163 ph/pix 055 ph/pix .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma
~ a; :J C'" 1 .8 1 .8 1 .8 Hard cppy leg IVO 0130 ~h/pix D65p~IX A32p~iX •16p pix 1.2 1.6 2 2.4 gamma Hard COPY. leg gvl 1.2 1.6 gamma Hard copy leg as 1.2 1.6 gamma 0130 ~h/pix a65 p fo'x •32 phroix •16 ph pix 2 2.4 0130 ph/pix D65 ph/plX A 32 ph/pix • 16 phfpix 2 2.4 ~ a; :J C'" .8 1 ~ a; :J C'" .4 .8 1 .8 Hard copy leg tis 1.2 1.6 gamma Hard copb leg a 1.2 1.6 gamma Hard cop~ leg r 1.2 1.6 gamma
FIGURE B2. Hard copy results. A: non-experts (scene "leg").
0130 ~hlpix D65 pgro1x A32 p pix •16 ph/pix 2 2.4 o 130 ph/pix D65 ph/pix A32 ph/pix • 16 ph/pix 2 2.4 0130 ph/pix D65 ph/plX A32 phfpix • 16 phfpix 2 2.4 35
~
ca
:;, CT 1 1 1 1 Hard COP.Y kidney ivo . - A - - - - o 0260 p~ix e1130 ~ pix A65p~1X •32 p pix 1.4 1.8 2.2 2.6 3 gamma Hard copy kidney gvl o260p~ix ~ca
:;, CT ~ca
:;, CT a 130 ~ pix 665 p folX · •32 ph pix 1.4 1.8 2.2 gamma Hard copy kidney as 0260 ph/pix e1 130 ph/pix A65 ph/plX • 32 ph/pix 2.6 1.4 1.8 2.2 2.6 gamma 3 3 Hard copy kidney hs 0260 ph/pix Cl 130 ~h/pix A65 p /pix •32 ph/pix 1 1 1.4 1.8 2.2 2.6 3 gamma Hard copy kidney ab 1 0260 ph/pi 4 c 130 ph/pix A 65 ph/pix • 32 ph/pix 1 1.4 1.8 2.2 2.6 3 gamma Hard copy kidney rh o260p~ix a130 ~pix A65p~IX +32 p pix 1 1.4 1.8 2.2 2.6 3 gamma~ (ij ::I CT ~.,. as ::::> CT o 1030 ph/pix a 41 O ph/p1x Hard copy cerebr 1vo • 163 ph/pix • 63 ph/p1x 1 1.5 2 2.5 3 3.5 4 4.5 gamma Hard copy cerebr gvl o 1030 ~h/pix D410 p tolX • 163 ~pix •65 p PIX 1 1.5 2 2.5 3 3.5 4 4.5 gamma Hard copy cerebr as o 1030 ~h/pix a 41
o
pgro1x • 163 ~pix •63 p PIX .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma 1 ~ (ij ::I CT ~ (ij ::I CT 7 o 1030 phlpix a410 ph/p1x A 163 ph/pix •65 ph/p1x 1 1.5 2 Hard copy cerebr fls 2.5 3 3.5 gamma Hard copy cerebr ab o 1030 ph/pix C410 ph/pix A 163 ph/pix ~65 ph/pix .5 1 1.5 2 2.5 3 3.5 gamma Hard copy cerebr rh o 1030 ~h/pix a 41 O pgro1x • 163 ~pix •65 p PIX 1 1.5 2 2.5 3 3.5 gammaFIGURE 82. Hard copy results. A: non-experts (scene "cerebr").
4 4.5
4 4.5
4 4.5
.8 Hard copy leg be 1.2 1.6 gamma Hard copy leg Ive 2 2.4 1 Hard copy leg mvo 0130 ph/pix D65 ph/pix £32 ph/pix t 16 ph/pix 4,.,.__.. ... __..._. __ ..___..._..,._ ... __,,.._..., .4 .8 1.2 1.6 2 2.4 gamma Hard copy leg ha 1+-___________ __,,_._.. ... __ .,._ __ _... .8 1.2 1.6 2 2.4 .4 .8 1.2 1.6 2 2.4 gamma Hard copy leg evr o 130 ph/pix D65 ph/pix A 32 ph/pix o 16 ph/pix
1-.----.----... ---__ ._. __ ,._ __ __,
.4 .8 1.2 1.6 2 2.4 gamma 1 1 gamma 0130 ph/pix D65 ph/pix A32 ph/pix o 16 ph/pix .8 1.2 ~.6 gammaFIGURE 82. Hard copy results. B: technical experts (scene "leg").
Hard copy leg pg .8 1.2 gamma 0130 ph/pix D65 ph/pix A32 ph/pix • 16 ph/pix 1.6 2 2.4 0260 ph/pix D130 ph/piX £65 ph/pix +32 ph/pix 1 1.4 1.8 2.2 2.6 3 0260 ph/pix 7 D 130 ph/pix A 65 ph/pix gamma Hard copy kidney ha ~ 0 32 ph/pi a; :J er 1 1.4 1.8 2.2 2.6 gamma 3 1 1.4 Hard copy kidney be 1.8 gamma 0260 ph/pix D 130 ph/pix A 65 ph/pix • 32 ph/pix 2.2 2.6 3 Hard copy kidney Ive 0260 ph/pix D130 phfpiX •65 ph/p1x •32 phfpix 1 1.4 1.8 2.2 gamma Hard copy kidney evr 2.6 3 0260 ph/pix D 130 ph/pix A65 ph/pix • 32 ph/pix 1 1.4 1.8 . 2.2 2.6 3 gamma
FIGURE B2. Hard copy results. B: technical experts (scene "leg","kidney").
1 Hard copy kidney rte 1 0260 ph/pix D 130 ph/pix A65 ph/pix •32 ph/pix 1.4 1.8 gamma 2.2 Hard copy o 1030 ph/pix c 410 ph/pix A 163 ph/pix • 65 ph/pix cerebr be 2.6 3 .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma 6 o 1030 ph/pix D410 ph/plX 6163 phfpiX •65 ph/p1x 3 Hard copy cerebr lve .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma 8 7 ~ ~6 CT 5 4 1 1 .5 1 Hard copy kidney pg 1.4 1.8 2.2 gamma Hard copy 0260 ph/pix D 130 ph/pix A 65 ph/pix • 32 ph/pix 2.6 3 cerebr mvo o 1030 ph/pix D410 ph/pix '163 ph/pix +65 ph/pix 1.5 2 2.5 3 3.5 4 gamma Hard copy cerebr .ha 4.5 o 1030 ph/pix D410 ph/pix A 163 ph/pix ~ 65 ph/pix .5 1 1.5 2 2.5· 3 3.5 4 4.5 gamma
1 Hard copy cerebr evr o 1030 ph/pix D410 ph/pix A 163 ph/pix 065 ph/pix 1 1.5 2 2.5 3 3.5 4 4.5 gamma Hard copy cerebr p o 1030 ph/pix D410 ph/pix A 163 ph/pix o 65 ph/pix .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma o 1030 ph/pix D410 ph/prx A 163 ph/pi ~65 ph/pr Hard copy cerebr rte .5 1 1.5 2 2.5 3 3.5 4 4.5 gamma
FIGURE B2. Hard copy results. B: technical experts (scene "cerebr").
1 1 Hard copy leg jav o 130 ph/pix
ass
ph/p1x .A 32 phfpix • 1S phfpix .8 1.2 1.S 2 2.4 gamma Hard COP.Y leg c:Jljp 0130 pt:i/pix a SS ph/_p1x • 32 phf_pix • 1S phfpix .8 1.2 1.S gamma Hard copy leg wa 2 2.4 0130 ph/pixass
ph/p1x •32 phfpix • 1S phfpix .8 1.2 1.S 2 2.4 gamma 1 1 Hard COP.Y leg to .8 1.2 1.6 .8 gamma Hard copy leg zw 1.2 1.6 gammaFIGURE B2. Hard copy results. C: experts (scene "leg").
o 130 ph/pix a SS ph/p1x •32 phfpix •16 phfpix 2 2.4 o 130 ph/pix a6S ph/p1x .A 32 phfpix • 1S phfpix 2 2.4
1 Hard copy kidney Jdv 0260 ph/pix D 130 phfpiX • 65 ph/prx •32 phfpix 1 1.4 1.8 2.2 2.6 0260 ph/pix D 130 pt:lfpiX •65 ph/prx •32 phfpix gamma Hard copy kidney dbp 1 1.4 1.8 2.2 2.6 gamma Hard copy kidney wa 0260 ph/pi a 130 phfpix .6. 65 ph/prx • 32 phfpix 3 1 1.4 1.8 2.2 2.6 3 gamma Hard copy kidney to 0260 ph/pix D130 phfpiX • 65 ph/prx •32 phfpix 1 1.4 1.8 2.2 2.6 gamma Hard copy kidney zw 0260 ph/pix D 130 phfpix .6. 65 ph/prx •32 phfpix 1 1.4 1.8 2.2 gamma 2.6
FIGURE 82. Hard copy results. C: experts (scene "kidney").
3
3
1 1 Hard copy cerebr Jdv o 1030 ph/pix D410 ph/plX A 163 phfpiX • 65 ph/p1x