Evaluation of a new satisfaction questionnaire about the use of
imaging: VIPA
Name: Rick Pinkster
Student number: S2005417
Course: Master thesis
Study: Health Psychology & Technology
Date: 24-06-2020
First internal supervisor: Peter ten Klooster Second internal supervisor: Erik Taal
External supervisors: Anne Meesters and Joep Kraeima
Organization: UMCG
EC’s: 25 EC
Table of contents
Abstract 4
Introduction 5
History of imaging 5
Patient satisfaction 6
Patient satisfaction with imaging 6
Using 3D images 7
Questionnaire about patient satisfaction 8
Validating the preliminary questionnaire 9
Demographic aspects related to patient satisfaction 11
Goal of the study 11
Method 13
Subjects and design 13
Materials 14
Data analysis 15
Item quality 15
Structural validity 15
Internal consistency 16
Concurrent validity 16
Associations of the questionnaire 16
Content validity 17
Results 18
Observations of filling in the questionnaire 19
Item analysis 20
Structural validity 21
Internal consistency 23
Correlation between the two factors 24
Concurrent validity 24
Associations of the questionnaire 24
Content validity 25
Discussion 27
References 32
Appendix A: Vragenlijst tevredenheid 2D afbeeldingen (eerste versie) 38
Appendix B: Vragenlijst tevredenheid 3D afbeeldingen (eerste versie) 41
Appendix C: Inter-item correlation 44
Appendix D: Response distribution of the different answer categories 45
Appendix E: Rotated factor loading after first PCA (based on eigen value >1) 46
Appendix F: Answers open text option 47
Appendix G: Final version of VIPA and instruction 50
Abstract
The purpose of this study was to test the preliminary Vragenlijst Invenatarisatie Patiënttevredenheid Afbeeldingen (VIPA) for its psychometric properties and to explore associations of the questionnaire. The ultimate aim of VIPA is to measure differences in patient satisfaction between the use of 2D images and the use of 3D images. VIPA was already tested for its content validity and face validity. During this quantitative field study, VIPA was analyzed with item analysis, structural validity via principal components analysis (PCA), internal consistency and potential associations of the questionnaire were examined. At the end, answers of the open text option were analyzed to explore if VIPA measures all subjects related to patient satisfaction about imaging.
106 trauma patients with bone fractures filled in VIPA. These respondents were sampled via convenience sampling. The preliminary VIPA had 22 items. Nine items were deleted during this psychometric field study, so the final version of VIPA has 13 items remaining. PCA indicated a factor structure of two different factors. These factors were interpreted as ‘the importance of seeing images’ (with 9 items) and ‘clearness of the image’
(with 4 items). Both factors showed an adequate Cronbach’s α with values of 0.84 and 0.75.
There were no indicators for severe floor or ceiling effects. The correlation of Spearman’s ρ between the two factors was not sufficient to calculate a total score (r = 0.34), so the factors have to be scored independently. The two factors were both moderately correlated with overall satisfaction of imaging, and ‘importance of seeing images’ had a weak negative correlation with level of education. The coding of the open text option suggested adequate content validity and only lead to one new subject: the frequency of seeing images. However, this subject is not necessarily related to patient satisfaction.
VIPA is the first questionnaire to measure patients’ satisfaction with the use of 2D images or 3D images. Based on the outcomes, it is suggested that the final VIPA is a valid and reliable instrument to measure patients’ satisfaction with the use of images that can be used for further research. VIPA can now be used to measure difference in satisfaction among groups of patients about the use of 2D and 3D images. VIPA could be further validated by performing confirmatory factor analysis with another sample to compare if the factor structure and internal consistency would be the same as presented in this study. There was little information in literature about the correlates of patient satisfaction with imaging, so therefore, this topic needs to be further explored.
Introduction
History of imaging
Two dimensional (2D) images are frequently used in hospitals to provide information to doctors and patients about a disease or injury (Kimmel, Adalsteinsson, Sapiro, & Sethian, 1996). The three most used methods for diagnostic 2D imaging are x-rays, Magnetic Resonance Imaging (MRI) and Computerized Tomography (CT). Images of x-rays were used for the first time in 1895, images of CT-scans are in use since 1972 and images of MRI-scans are in use since 1982. These images can be used to inform doctors on the location of an injury or to detect a tumor (Apelian, 2012; Als-Nielsen & McMorrow, 2011). Besides that, these 2D images can also be shown to patients to explain their injury or disease to them, so patients would better understand their injury or disease (Carlin, Smith & Henwoord, 2014).
During the past twenty years, developments in technology regarding three dimensional (3D) scans have enabled computer programs to provide images of better quality. These 3D images are reconstructed from digital data of 2D images, which are mostly based on images of CT-scans and sometimes based on images provided by MRI-scans. Compared to 2D images, 3D images in medical care have the advantages to provide more detailed information and better visualization. It is also possible to rotate the image, to highlight specific parts on the images and to display the information in one image instead of several segmented images (Kimmel et al., 1996; Morris & Van Wijhe, 2010). However, 3D images are more expensive to create, more complex and need more specialized equipment compared to 2D images (Rengier et al., 2010). Figure 1 and figure 2 provide examples of a 2D image and a 3D image of a pelvis.
Figure 1: 2D example of a pelvis via x-ray Figure 2: 3D example of a pelvis
Patient satisfaction
Although 3D images have a lot of advantages, there is no information at the moment about the opinions of patients about the use of 3D images as far as known. The opinions of patients could be gathered to measure patients’ satisfaction about a certain topic. Information about patient satisfaction is relevant for hospitals, so they can increasingly improve treatments to the wishes of patients. There are several advantages for hospitals to have satisfied patients, for example satisfied patients find it easier to follow medical instructions and they need on average fewer medical visits (Nathorst-Böös, Munck, Eckerlund & Ekfeldt-Sandberg, 2001).
Research also found that higher patient satisfaction is related to better compliance and less early dropouts from treatments (Wartman, Morlock, Malitz & Palm, 1983). Besides the advantages of meeting the wishes of patients, it is nowadays more important for doctors to involve patients in making choices related to their treatment. The involvement of patients in making choices about the treatment is called shared decision making. This method combines the expertise of the doctor with the wishes of the patients (Stiggelbout et al., 2012). Showing 2D and 3D images to patients could support the process of shared decision making, because patients receive more information and could make better decisions about their treatment (Canbazoglu, Salman, Yildirim, Merdenyan, & Ince, 2016).
Patient satisfaction with imaging
In general, patients are satisfied with the use of images to explain their pain or fracture. British research about radiography showed that patients who saw images were more satisfied than patients who saw no images, even if the health outcomes of the patients who saw images were worse (Kendrick, Fielding, Bentley, Kerslake, Miller & Pringle, 2001).
Another study compared the use of MRI-scans to plain radiology (Jarvik et al., 2003). This study showed that patients who saw MRI-scans were more reassured and more satisfied compared to patients who only received plain radiology. The patients who saw MRI-scans were still more satisfied, even when their health outcomes were worse (Jarvik et al., 2003). It is expected that patients are more satisfied with imaging because they do often not know where their pain comes from or what kind of fracture they have. Seeing images helps patients to understand the situation better, and they could find it comforting to know what is going on (Rhodes, Mcphillips-Tangum, Markham & Klenk, 1999).
Using 3D images
University Medical Centre Groningen (UMCG) recently started working with 3D images which are mostly based on CT-scans and sometimes on MRI-scans. Despite the costs and the complex process to create 3D images, it is assumed that these images are easier to understand for patients than 2D images, for the reasons such as being able to highlight certain parts in 3D images and to rotate the image. Currently, there is no information from patients that confirms this expectation, because the use of 3D images in hospitals to explain fractures to patients is relatively new. Studies about this subject are more focused on patient satisfaction with 3D prints instead of 3D images on screen (Bernard et al., 2016; Bizzotto et al., 2016). So far, only one study specifically researched the satisfaction of 3D images compared to the satisfaction of 2D images (Phelps, Wellings, Griffiths, Hutchinson & Kunar, 2017). This study concluded that 3D images were better understood, and participants found it easier to recall information when they saw 3D images. However, this study was a laboratory study among healthy people, so the researchers stated that these results can not directly be generalized to patients in clinical settings (Phelps, Wellings, Griffiths, Hutchinson & Kunar, 2017). For this reason, research regarding patient satisfaction with the use of 3D images within a clinical setting is needed.
Patient satisfaction is a commonly used indicator for gaining information about patients, because it provides in-depth information about patients’ opinions and why they have these opinions (Naidu, 2009). Therefore, patient satisfaction does not only provide information about the patients’ opinions regarding a certain treatment, but also it provides information about which different aspects of a treatment are rated positive or negative (Al- Abri & Al-Balushi, 2014). It is important to maximize patient satisfaction, because higher patient satisfaction leads to higher compliance to the doctors’ advises, better adherence to the treatment and patients are more likely to change their behaviour into healthy behaviour (Sitzia
& Wood, 1997). To gain insight in the satisfaction of patients, a quantitative measurement is needed to measure different aspects of a treatment (Danielsen et al., 2010). Existing questionnaires measure general satisfaction about treatments, so it would be better to design a more specific questionnaire if the patient satisfaction of a specific part of the treatment needs to be known (Al-Abri & Al-Balushi, 2014; Ford, Bach & Fottler, 1997). If general patient questionnaires are used to measure specific parts of a treatment, the reliability and validity of the measurement decrease because it is uncertain if the specific parts are measured in a right way (Elaine, Gail & Richard, 2002).
Questionnaire about patient satisfaction
Recently, a new preliminary questionnaire was developed to measure the level of satisfaction of patients which were shown either 2D images or 3D images, called the
‘Vragenlijst Inventarisatie Patiënttevredenheid Afbeeldingen (VIPA)’, which could be translated as ‘questionnaire inventory patient satisfaction about imaging’. VIPA was developed after extensive literature review and interviews with patients who had all different kind of fractures, for example arm, leg and pelvis fractures. VIPA is already tested for its content validity among patients via the Three-Step-Test-Interviewing model (TSTI-model) as described by Hak (2004) and it has been tested for its face validity with medical specialists and (social) psychologists. Results from VIPA could show if the use of 3D images leads to a higher patient satisfaction compared to the use of 2D images. VIPA provides scores about patient satisfaction and those scores can be compared to analyze if there is a significant difference in patient satisfaction between the use of 2D and 3D images (Pinkster, 2019).
The preliminary VIPA measures patient satisfaction based on thirteen different aspects (Pinkster, 2019), which are displayed in table 1. The first aspect is the expectations of the patients, because patients are more satisfied if a treatment meets their expectations (Bjertnaes, Sjetne & Iversen, 2012). The second aspect is the amount of provided information to a patient. Providing more information is related to higher patient satisfaction (Kelvin et al., 2016). The third aspect is the level of anxiety of the patients. Higher levels of anxiety are related to low satisfaction, while low levels of anxiety are related to high satisfaction (Phelps, Wellings, Griffiths, Hutchinson & Kunar, 2017). The fourth aspect is the level of trust in the treatment. If patients have trust in (parts of) a treatment and think it contributes to their recovery, they report higher satisfaction (Phelps, Wellings, Griffiths, Hutchinson & Kunar, 2017). The fifth aspect is the patient’s satisfaction about the treatment (Phelps, Wellings, Griffiths, Hutchinson & Kunar, 2017). The sixth aspect is specifically related to the use of 2D or 3D images and if patients understand the information that is provided by the image. The better they understand the provided information, the higher the patient satisfaction (Bernard et al., 2016; Bizzaotto et al., 2016). The seventh aspect is how the provided information contributes to the decision-making process (Brédart, Bouleuc & Dolbeault, 2005; Lee, Back, Block & Stewart, 2002). The last six aspects of the VIPA are based on earlier research within the UMCG. These aspects are: How 2D or 3D images stimulates remembering information, how clear the provided information is for patients, showing progress in the recovery process of the injury, how seeing 2D or 3D images contributes to adhering during treatment and how
seeing 2D or 3D images is related to the recovery process (Pinkster, 2019). The preliminary VIPA had two different versions: One version measured patient satisfaction with the use of 2D images and the other version was about satisfaction with the use of 3D images. The questions of both versions are the same, but the type of image (2D or 3D) is specified in each different version.
Validating the preliminary questionnaire
Validating a questionnaire is an important process, because it increases the probability that a questionnaire accurately measures what it aims to do (Kowalski, Crocker & Faulkner, 1997). Although VIPA is already qualitatively tested for its content validity and its face validity, it still needed to be quantitatively tested for its psychometric properties. To analyze the correlation between all items, the results of a questionnaire could be tested via inter-item correlation. This test indicates which items have a high correlation with each other and which items have a low correlation with the other items. Another test is to analyze the variance of the scores, to examine if items suffer from floor or ceiling effects. These effects could occur if questions are too steering, too easy, too difficult or not relevant (Fries, Rose & Krishnan, 2011). Floor and ceiling effects need to be avoided, because it indicates low content validity for that question and low responsiveness (Terwee et al., 2007). Structural validity could be tested via factor analysis in order to identify which factors are underlying in the questionnaire (Terwee et al., 2007; Reio & Shuck, 2015). The various items that are related to a factor could be tested for its internal consistency, to indicate the degree of relatedness among items and to indicate internal reliability (Tsang, Royce & Terkawi, 2017). Concurrent validity is the degree to which the values of the questionnaire are consistent with another measurement (Hagströmer, Oja & Sjöstrom, 2006). VIPA has a rating scale about general satisfaction with imaging, and this could be used to analyze how the scores of VIPA correlate with general satisfaction about imaging. Although content validity was already tested during a previous study, it could be tested again by analyzing the answers of the open text option, to indicate if VIPA contains already all subjects related to patient satisfaction or if some subjects were still missing. Finally, the topic of construct validity is important when a questionnaire is tested for its psychometric properties. Construct validity contains the consistency between a questionnaire and physiological, demographical or clinical variables (Hagströmer, Oja &
Sjostrom, 2006).
Table 1: Overview of elements related to patient satisfaction and created questions
Aspect: Found with
the method:
Items (in Dutch):
Expectations Literature -14. Voorafgaand aan mijn behandeling had ik verwacht 2D/3D afbeeldingen te gaan zien tijdens een consult.
Information seeking / providing information
Literature / interviews / observations
-2. De 2D/3D afbeelding gaf veel informatie over mijn letsel.
-5. De 2D/3D afbeelding van mijn letsel riep veel vragen op.
-7. De 2D/3D afbeelding was nodig om de uitleg van de arts over mijn letsel te begrijpen.
Anxiety Literature / interviews
-3. Ik werd angstig toen ik de 2D/3D afbeelding voor het eerst zag.
-15. Ik vond het confronterend om de 2D/3D afbeelding van mijn letsel te zien.
-21. De 2D/3D afbeelding was geruststellend voor mij om te zien tijdens consulten.
Understanding Literature / interviews
-6. Door het zien van de 2D/3D afbeelding begreep ik mijn letsel beter.
-9. Ik begreep de uitleg van de arts over de 2D/3D afbeelding.
-19. De 2D/3D afbeelding maakte de uitleg over mijn letsel beter te begrijpen.
Perceived accuracy
Literature -20. Ik vind dat een 2D/3D afbeelding een betrouwbare weergave gaf van mijn letsel.
Trust Literature -13. Ik heb vertrouwen dat een 2D/3D afbeelding bijdraagt aan een goede diagnose van mijn letsel.
Satisfaction Literature -4. Ik vond het prettig dat er 2D/3D afbeeldingen zijn gebruikt om mijn letsel uit te leggen.
-17. Het zien van de 2D/3D afbeelding van mijn letsel was heel belangrijk.
-23: Raadt u het gebruik van 2D/3D afbeeldingen aan tijdens de behandeling van andere patiënten? Zou u uw antwoord willen toelichten?
Decision making Literature / observations
-8. Door het zien van de 2D/3D afbeelding kon ik samen met mijn arts een weloverwogen keuze maken over mijn
vervolgbehandeling.
Remembering information
Observations / interviews
-18. Toen ik thuis kwam kon ik de informatie over mijn letsel herinneren doordat ik een 2D/3D afbeelding had gezien tijdens mijn consult.
Clearness of information
Observations / interviews
-1. De 2D/3D afbeelding gaf duidelijke informatie over mijn letsel.
-16. Ik vond het moeilijk om op de 2D/3D afbeelding te zien waar mijn letsel precies zit.
Showing progress Observations / interviews
-10. Door de 2D/3D afbeelding kon ik goed de voortgang van herstel van het letsel zien.
Intention to adhere during treatment
Staff UMCG -12. De 2D/3D afbeelding motiveerde mij om mij te houden aan de adviezen van de arts.
-22. De 2D/3D afbeelding motiveerde mij om te werken aan mijn herstel.
Relation to recovery
Staff UMCG -11. Het zien van de 2D/3D afbeelding was van belang voor mijn revalidatieproces.
Demographic aspects related to patient satisfaction
No correlations were found in literature between demographic aspects and patient satisfaction with imaging. Therefore, a search in literature was done to find associations between demographic aspects and patient satisfaction in general. Literature provides conflicting information about the correlation between patient satisfaction and age. In general, it is mentioned that agehas the strongest influence on patient satisfaction (Rahmqvist, 2001) and that older people score higher on patient satisfaction questionnaires than younger people (Cohen, 1996). Also, research of Rogers et al. (2013) regarding trauma patients found that age is the strongest predictor of patient satisfaction. However, research of Van Vliet-Koppert et al.
(2011) about patient satisfaction related to the treatment of toe fractures did not find a correlation between satisfaction and age. Another demographic which could influence patient satisfaction is the type of injury. However, some literature mentions there is a relation between type of injury and patient satisfaction (Rahmqvist, 2001), while other literature argues there is not a relationship between these two aspects (Hseih & Kagle, 1991). The influence of gender on patient satisfaction is unclear. Some studies mention there is no influence of gender on patient satisfaction, but other research found small differences in satisfaction between males and females (Doering, 1983; Hopton, Howie & Porter, 1993). In cases where research found a relation between gender and patient satisfaction, males were mostly a little more satisfied than females (Rahmqvist, 2001). Research about the relation between level of education and patient satisfaction showed that lower education is related to higher satisfaction (Hall & Dornan, 1990). Various studies are sometimes conflicting about the expected associations between patient satisfaction and demographics. This could be explained because some studies are about patient satisfaction in general and some studies are about patient satisfaction related to specific topics. Unfortunately, at the moment, there are no studies about predicted associations between demographics factors and patient satisfaction with the use of 2D and 3D images. Based on the information about general patient satisfaction, it is expected to find in this study higher levels of satisfaction about the use of 2D or 3D images related to older age and to lower education.
Goals of the study
The aim of this study was to further validate and refine the preliminary VIPA. This was done by examining the item quality, factorial structure, and internal consistency of VIPA.
As stated in the previous paragraph, no information could be found in literature about predicted associations between demographic variables and patient satisfaction about the use of
2D and 3D images. Therefore, associations with patient satisfaction about the use of imaging were exploratory analyzed.
Method
Subjects and design
Trauma patients with bone fractures see 2D and 3D images during a consultation with a doctor or medical specialist. The patients are shown images of their fractures, so they better understand the progress of their recovery. In this cross-sectional study, patients were asked to fill in VIPA once. Three medical specialists of the hospital consulted patients and asked these patients to participate in this study and to answer the questions of VIPA. Two of these medical specialists were clinical nurse specialists and one of them was a technical physician.
During the first sixty-five consultations, the researcher was also present to observe the explanation of the images. After the consultation, the medical specialist or the researcher answered on the VIPA how many injuries and which kind of injuries were shown to the respondent. Thereafter, the researcher went to a separate room with the respondents and the respondents filled in the rest of VIPA. The researcher was in the same room as the respondents, so it could be observed if patients experienced any difficulties with VIPA. The data was collected from the 6th of January 2020 till 13th of March 2020. The questions of VIPA were answered on paper and the answers were transferred to the digital program REDCap. This program stored the data on the secured network of the UMCG.
Patients with bone fractures were the target group of this study. The kind of fractures differed widely from, for example, pelvis fracture to thumb fracture. It was also possible that patients had multiple fractures. Only patients whom were shown 2D or 3D images during their consult were included, because VIPA is about the satisfaction of patients about the use of 2D or 3D images. The respondents’ age had to be 18 years or older. Exclusion criteria for this study were for patients to be younger than 18, if patients did not understand Dutch texts and if patients stayed in a mental health clinic.
Patients were selected via convenience sampling. Every consecutive patient was asked after their consultation to participate in this study (Migchelbrink, 2010). Patients whom where shown 2D images of their fractures were asked to fill in the version of VIPA about 2D images and patients whom where shown 3D images of their fractures were asked to fill in the version of VIPA 3D images. The treatments of patients were not changed for this study. 3D images are mostly shown to patients in combination with 2D images, so patients whom where shown
both types of images during their consultation were asked to fill in the version of VIPA about 3D images and the patients whom where shown only 2D images filled in the version of VIPA about 2D images. The preliminary version of VIPA about 2D images can be found in Appendix A and the preliminary version of VIPA about 3D images can be found in Appendix B.
The minimum required sample size differed for the different tests that were done during this study. For examining the ceiling and floor effects, it is mentioned in literature to have a sample size of at least 50 respondents (Terwee et al., 2007). Literature suggests very different numbers for the sample size in the case of factor analysis. Some literature argues a certain number of respondents for each item of the questionnaire. However, these numbers vary widely from 3 respondents for every item to 20 respondents for every item. The number of 5 respondents for every item is mostly used as a rule of thumb (Gorsuch, 1983; Mundfrom, Shaw & Lu Ke, 2005). There is also literature that does not calculate the sample size based on the number of items, but states that at least a sample of 50 respondents is needed for factor analysis. However, this is considered as a very weak sample size (Winter, Dodou & Wieringa, 2009). The preliminary VIPA consisted of 22 Likert-scale items. To meet the rule of thumb, it was aimed to recruit at least a sample size of 110 respondents. If this number of respondents was not possible within the time range of this study, a minimum number of 50 respondents was needed. Eventually, 106 respondents were included for this study.
Materials
The preliminary VIPA had six questions about demographics, 22 Likert-type items scaled from 1 till 5, one 1-10 global rating scale, one yes/no question and an open option for respondents to explain their answers or to add information. The demographic questions are about gender, age, highest level of education, number of injuries, type of fracture and at which moment the VIPA was filled in. The 22 Likert-type items are based on the 13 aspects which are mentioned in the introduction. The answering scales for these items range from totally not agree (1) to totally agree (5). Four items had to be recoded, so that higher scores indicated higher satisfaction. The 1-10 global rating scale is about the general satisfaction about the use of the 2D or 3D images, where 1 indicates ‘very unsatisfied’ and 10 ‘very satisfied’. The last question is about if respondents would recommend the use of 2D or 3D images in the treatment of other patients. This question can be answered with yes or no and it is possible for
respondents to explain their answer or to add additional information. VIPA was filled in on paper.
Data analysis
All data were digital stored in REDCap and all analyses were executed via SPSS version 24. Six stepwise analyses were done to develop and validate the preliminary VIPA.
Item quality
The first analysis was individual item analysis. This was done by analyzing the inter- item correlations and by analyzing every item for floor and ceiling effects in the response distribution. The inter-item correlation was used to analyze how the different items correlate with each other. Items with a correlation higher than 0.7 (or -0.7) were considered potentially redundant, because these values would indicate a too strong correlation between two items (Streiner, Norman & Cairney, 2015). If a pair of items had this high correlation, it was chosen to keep only one of them in the VIPA based on balancing item formulation and content coverage. Thereafter, the data of every item was analyzed for floor and ceiling effects by examining the variance of all answers of respondents at a certain item. Floor and ceiling effects indicate that many respondents filled in the highest answer possibility or the lowest answer possibility, indicating that such an item cannot discriminate well between people or measure worsened or improved satisfaction over time (Field, 2009). Most patient satisfaction questionnaire reports high satisfaction, possibly because patients are glad to even receive care (Friedberg, Safran & Schneider, 2012). Therefore, it was chosen to use cutoff of 80% to decide if an item was suitable or not for the questionnaire. So, if 80% of the respondents gave the highest or lowest possible answer, the variance of this item was considered too low, and this item was excluded from VIPA and further analyses.
Structural validity
The second analysis was Exploratory Factor Analysis (EFA), which examined the factor structure of VIPA. EFA is aimed at discovering the empirical factors underlying the items and is needed for indicating how VIPA can be scored as either a total scale score or subscale scores. Kaiser-Meyer-Olkin (KMO) was used to indicate how suitable the data was for factor analysis. KMO had to be at least 0.5 or higher to perform a factor analysis (Field, 2009). EFA was executed by a series of Principal Component Analyses (PCAs) with varimax rotation. The rule of an eigenvalue more than 1 was used for the first PCA. Afterwards, a
scree plot was used to indicate the optimal number of factors (Abdi, 2003; Terwee et al., 2007; Reio & Shuck, 2015). Items with a factor loading lower than 0.40 were considered for removal (Clark & Watson, 1995). Items with a high negative correlation or which correlated more than 0.40 on multiple factors were also considered to be left out of VIPA (Field, 2009).
The items which were removed as a result of the PCAs were not used for further analyses.
Internal consistency
The third analysis was to test the internal consistency of the total scale or subscales resulting from the PCA using Cronbach’s alpha (α). A high Cronbach’s α (close to 1) indicates a strong internal consistency and a low Cronbach’s α (close to 0) indicates a weak consistency. A Cronbach’s α of 0.7 or higher indicates an adequate internal consistency for group-level analysis of the scale scores (Tsang, Royce & Terkawi, 2017). During the process of analyzing, it was also examined how Cronbach’s α would increase or decrease if an item would be left out. The goal was to have a Cronbach’s α of at least 0.7 for every factor in the final instrument (Field, 2009).
Concurrent validity
To measure concurrent validity, the scores of the factors were tested for its correlation with the answers of the question about overall satisfaction with imaging. At the end of the questionnaire, the question ‘How satisfied are you with the use of imaging in general’ is asked and patients can answer this question with a number from 1 till 10. This question was tested for possible correlations with the factors via Spearman’s ρ, because there were indicators that some data were not normally distributed. A correlation of 0.1 to 0.3 was considered as weak, a correlation of 0.4 to 0.6 was considered as moderate, and a correlation of 0.7 to 0.9 was considered as strong (Dancey & Reidy, 2007).
Associations of the questionnaire
Since it was not possible to find robust correlates in literature, the aim of this analysis was to indicate how the factors found during PCA are associated with the demographical and clinical characteristics of the patients. Demographical and clinical variables examined were age, location of the fracture, different medical specialists, level of education, and number of weeks after injury. First, (sub-) scales were computed based on the factors found during PCA.
For every factor, the scores of the different items related to the factor were summed and divided by the number of items related to the factor. There were indicators that some data
were not normal distributed, so therefore, the demographics were examined with the non- parametric test Spearman’s ρ for analyzing for correlations with the factors found during PCA. Associations with categorical variables were examined with one-way analysis of variance (ANOVA). Because the literature review had not resulted in sufficient information to formulate robust hypotheses, a medical specialist was asked about her expectations with respect to possible associations between the variables and the factors. These expectations were compared to the observed results of the association tests.
Content validity
The last question of VIPA is ‘would you recommend the use of imaging during the treatment of other patients?’ This question could be answered with yes or no, and patients had the option to explain their answer or to provide additional information. This question provided data, which was qualitatively analyzed via ‘inductive coding’. This means that the answers of the respondents were analyzed to identify key subjects and to see how often every subject was related to an answer. For this analysis, the iterative way of open coding was used, which means that the different codes are based on the answers of the respondents (Babbie, 2016).
Since no literature was available to find existing coding schemes, the coding scheme had to be based on the information from the respondents. The codes were used to analyze if any important subjects were missing in the questionnaire.
Results
108 patients were recruited to fill in VIPA. From these patients, two respondents were excluded because they could not independently read Dutch texts. Eventually, 106 respondents were included for analysis. From these respondents, 61 respondents (57.5%) were male and 45 respondents (42.5%) were female. 100 respondents were shown only 2D images and six respondents were shown 3D images. From the 100 respondents whom were shown 2D images, two respondents were shown an image of a MRI-scan, seven respondents were shown an image of a CT-scan and 96 respondents were shown a x-ray image. These numbers added up are more than 100, because some respondents were shown a x-ray image as well as a MRI- scan or a CT-scan. From the respondents who saw 3D images, four respondents were shown also a x-ray image. The other two respondents saw no 2D image besides the 3D image.
The age of the respondents ranged from 18 years old to 93 years old, with a mean age of 51.4 (SD = 20.7) years. The level of education ranged from primary education to university. Most of the respondents saw only one fracture on an image, but five respondents saw multiple fractures. The different types of fractures were divided in the three subcategories torso, arms and legs. 61 respondents had a fracture located in their torso, 28 respondents had a fracture located in their arms and 17 respondents had a fracture located in their legs. The moment of completing VIPA differed between 1 week after their injury till 110 weeks after the injury with a mean of 22.0 weeks after the injury (SD = 26.6). Three different medical specialists explained the images to the patients during this study. 58 respondents had their image explained by medical specialist A, 35 respondents had their image explained by medical specialist B and 13 respondents had their image explained by medical specialist C.
An overview of the respondents’ characteristics is displayed in table 2.
Table 2: Characteristics of the respondents
Baseline characteristics Number of
respondents
Percentage of respondents Gender
Male 61 57.5%
Female 45 42.5%
Highest level of education
Primary education 10 9.4%
Pre-vocational education 16 15.1%
Vocational education 35 33.0%
Higher general continued education 4 3.8%
Preparatory scientific education 1 0.9%
University of applied sciences 27 25.5%
Research university 13 12.3%
Type of image shown
2D 100 94.3%
3D 6 5.7%
Type of 2D image shown
MRI 2 1.9%
CT 7 6.6%
X-ray 96 90.6%
No 2D image 1 0.9%
Location of the fracture
Torso 61 57.5%
Arms 28 26.4%
Legs 17 16.0%
Medical specialist who explained the image
A 58 54.7%
B 35 33.0%
C 13 12.3%
Observations of filling in the questionnaire
During the completion of VIPA, it was noticed that some respondents had difficulties with item 5 (“The image raised a lot of questions”) and item 14 (“Before the start of my treatment, I expected to see images during consultations”). Five respondents asked the researcher about the meaning of item 5 and seven respondents had questions about item 14.
Item 5 was created with the assumption that the visualization of an image is unclear when it raises a lot of questions. However, based on the thoughts patients shared, some patients had a lot of questions because they received new information from the image, not because they found the image unclear. Item 14 also raised a lot of questions by patients. Some patients asked what was meant with the question, because they were notified before their consult that they would see an image.
Item analysis
No item responses were missing, so every respondent filled in VIPA completely. First, item analysis was used to analyze the inter-item correlations to assess potential item redundancy. An overview of the individual inter-item correlations can be found in Appendix C. Some items showed a high correlation with each other. Item 1 (“The image provided clear information about my injury”) had an inter-item correlation of 0.73 with item 2 (“The image provided a lot of information about my injury”). This high inter-item correlation indicated that the items are very similar to each other and suggested content redundancy. Therefore, item 2 was excluded from further analysis, because it was expected that the formulation ‘clear information’ is more to the point than ‘a lot of information’.
Item 3 (“I was afraid when I saw the image for the first time”) and item 15 (“It was confronting to see the image of my injury”) also had a high inter-item correlation of 0.71.
Again, it was chosen to exclude one of these items based on this high inter-item correlation.
The major differences between the two items are that item 3 used the word ‘afraid’ and item 15 used the word ‘confronting’, and item 3 named more specifically which moment the item is about with the addition ‘when I saw the image for the first time’. It is assumed that the word ‘afraid’ is better to understand than the word ‘confronting’. It was also favored to indicate a specific moment when patients could get afraid or could be confronted, because some patients see different images at different moments. Based on this information, it was chosen to exclude item 15 from VIPA.
The inter-item correlation additionally showed that despite recoding, the negatively formulated items had the highest number of negative correlations with the other items. Item 3 had with sixteen other items a negative correlation (ranging from -0.01 till -0.30), item 5 had with eleven other items a negative correlation (ranging from -0.01 till -0.28), item 15 had with thirteen other items a negative correlation (ranging from -0.04 till -0.24) and item 16 had with six other items a negative correlation (ranging from -0.02 till -0.19). Despite this high number of negative correlations, these items were kept for further analysis.
Furthermore, item response frequencies on every item were examined. Frequencies showed that none of the items scored 80% or more on the lowest or highest response, which indicates there are no severe floor or ceiling effects. The table in Appendix D presents
response distribution for every item. Concluding, based on the item analysis, only item 2 and item 15 were excluded during item analysis and not used in further analyses.
Structural validity
Principal components analysis (PCA) with varimax rotation was executed to indicate the structure of VIPA and to analyze which items showed poor factor loadings, so these items could be excluded from VIPA. Based on the rule of an eigenvalue ≥ 1, five different factors were found with an explained variance of 60.84% during the first PCA. These results can be found in the table displayed in Appendix E. However, the scree plot (see figure 3) showed two dominant factors in the structure of VIPA. Therefore, PCA was executed again with a fixed number of two factors.
Figure 3: Scree plot
Additionally, fixed two-factor PCAs were executed iteratively by removing poorly loading items. Items were removed if they did not meet the following conditions: Items needed to have a positive factor loading of 0.40 or higher, items needed to load on one factor only with 0.40 or higher, and the difference in factor loading for one item needed to be more than 0.10. At first, PCA was executed with all items (except for items 2 and 15). Based on this solution, it was chosen to exclude item 3 (“I was afraid when I saw the image for the first
0 1 2 3 4 5 6
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Eigenvalue
Components
time”), because it had a factor loading of -0.42 related to factor 1 and a factor loading of 0.28 related to factor 2. PCA was repeated and this led to the exclusion of item 5 (“The image of my injury raised a lot of questions”). Item 5 had a factor loading -0.31 related to factor 1 and 0.35 related to factor 2. The third PCA led to the exclusion of item 11 (“Seeing the image was important for my recovery”). Although it had a factor loading of 0.66 on factor 1, it was decided to exclude this item because of its’ high negative factor loading of -0.41 related to factor 2. The next PCA resulted in the exclusion of item 4 (“I was satisfied with the use of images to explain my injury”). This item had a factor loading of 0.21 related to factor 1 and a factor loading of 0.37 on factor 2. The fifth time PCA was executed, item 10 (“The image showed me the progress of my recovery”) was excluded. Item 10 had a factor loading of 0.39 related to factor 1 and a factor loading of 0.49 related to factor 2. It was chosen to exclude item 10, because the factor loadings were close to each other and the item loaded relatively strongly on both factors. The last time PCA was executed, it resulted in the exclusion of item 6 (“Seeing the image helped me to understand my injury better”). This item had a factor loading of 0.42 related to factor 1 and a factor loading of 0.32 related to factor 2. It was chosen to exclude this item, because the factor loadings were very close to each other.
One more item was deleted during the testing of internal consistency in the next stage.
This was item 16 (“It was difficult to see the exact location of my fracture on the image”) and this item was removed to increase the internal consistency. PCA was repeated and this led to the final factor structure, which is presented in table 3 with the eigenvalue and explained variance. The total explained variance of the final factor solution was 49.51%. The value of Kaiser-Meyer-Olkin (KMO) was 0.79, which indicated that the data was suitable for executing PCA. The final factor solution presented a factor structure with two different factors. The first factor was interpreted as ‘the importance of seeing images’ and the second factor was interpreted as ‘clearness of the image’.
Table 3: Final factor structure (factor loadings ≥ 0.40 are presented in bold)
Items Importance of
seeing images
Clearness of the image 1. De afbeelding gaf duidelijke informatie over mijn letsel. -0.01 0.67 7. De afbeelding was nodig om de uitleg van de arts over mijn letsel te
begrijpen.
0.70 0.05
8. Door het zien van de afbeelding kon ik samen met mijn arts een weloverwogen keuze maken over mijn vervolgbehandeling.
0.61 0.18
9. Ik begreep de uitleg van de arts over de afbeelding. 0.13 0.87 12. De afbeelding motiveerde mij om mij te houden aan de adviezen
van de arts.
0.76 0.12
13. Ik heb vertrouwen dat een afbeelding bijdraagt aan een goede diagnose van mijn letsel.
0.15 0.71
14. Voorafgaand aan mijn behandeling had ik verwacht afbeeldingen te gaan zien tijdens een consult.
0.55 0.00
17. Het zien van de afbeelding van mijn letsel was heel belangrijk. 0.77 0.06 18. Toen ik thuis kwam kon ik de informatie over mijn letsel
herinneren doordat ik een afbeelding had gezien tijdens mijn consult.
0.55 0.02
19. De afbeelding maakte de uitleg over mijn letsel beter te begrijpen. 0.60 0.35 20. Ik vind dat een afbeelding een betrouwbare weergave gaf van mijn
letsel.
0.15 0.75
21. De afbeelding was geruststellend voor mij om te zien tijdens consulten.
0.62 0.14
22. De afbeelding motiveerde mij om te werken aan mijn herstel. 0.73 0.13
Eigenvalue 4.469 1.968
Explained variance 30.49% 19.02%
Internal consistency
The internal consistency was analyzed to indicate how the different items correlate within the same factor. The factor structure resulting from the PCAs led to a high Cronbach’s α values of 0.84 for factor 1 and a nearly acceptable value of 0.67 for factor 2. It was possible to increase this value if item 16 (“It was difficult to see the exact location of my fracture on the image”) would be deleted. It was chosen to delete this item, so Cronbach’s α for factor 2 would increase to 0.75. After the exclusion of item 16, Cronbach’s α was calculated again for factor 2. Summarized, the internal consistency of the final factor structure led to a Cronbach’s α of 0.84 for factor 1 (with nine items) and 0.75 for factor 2 (with four items).
Correlation between the two factors
The correlation of Spearman’sρ between both factors was 0.34, which is indicated as a weak correlation. Therefore, it is assumed that the two factors measure relatively independent subjects, so the factors were used separately for further analyses and no total VIPA score was computed for analyzing the correlates of the questionnaire.
Concurrent validity
To measure concurrent validity, the global rating scale about general satisfaction with imaging was tested for its correlation with the two factors via Spearman’s ρ. The importance of seeing images had a correlation of 0.42 with the global rating scale. This indicated a moderate correlation. The clearness of images had a correlation of 0.40 with the global rating scale, which also indicated a moderate correlation.
Associations of the questionnaire
It was analyzed how the two factors were associated with sociodemographic and clinical patient-related variables. The variables gender, location of the fracture and the different medical specialists were analyzed with one-way ANOVA. The variables level of education, overall satisfaction, number of weeks after injury and age were analyzed via Spearman’s ρ for possible correlations with the factors. The priori expectations about a positive association by a medical specialist were compared with the outcomes of these analyzes.
The results of the one-way ANOVA tests were all not significant, which indicated that none of the scores of the variables differed for the two different factors. This means there were no associations found between the variables and the two factors. This corresponded almost completely with the expectations of the medical specialist. However, the medical specialist did expect a positive association between clearness of the image and the location of the fracture, but this association was not found. This association was expected, because some fractures are located on more difficult locations than other fractures, but the tests did not indicate differences in clearness of the image for different locations.
The results of Spearman’sρ indicated one significant negative correlation between the importance of seeing images and level of education. This correlation was not expected.
Although the medical specialist expected positive correlations between all the other variables and the two factors, there were no other significant correlations found during the analyses with Spearman’s ρ. Based on literature, it was expected to find positive correlations between age and the two factors, but these correlations were also not found. An overview of the outcomes of Spearman’s ρ compared to the expectations of the medical specialist can be found in table 4.
Table 4: Expectations compared to the outcomes of Spearman’s ρ
Variables Expectations from practice Outcomes
Spearman’s ρ (r)
Importance of seeing images and level of education
No association expected -0.29*
Clearness of the image and level of education
It is expected that patients with a higher level of education do better understand the images than patients with lower level of education
-0.05
Importance of seeing images and time after injury
In the first weeks after the injury, it is expected that patients are more focused on processing the trauma or accident, than seeing images. Therefore, it is expected that patients find it less important to see images during the first period after their injury
0.17
Clearness of the image and time after injury
It is expected for patients to understand the image better if the time after the injury increases, because it is more likely for them that they have seen previous images during their treatment
0.05
Importance of seeing images and age
It is expected that younger people find it more important to see images than elderly, because elderly think most of the time that they would not understand the image
0.16
Clearness of the image and age
It is expected that elderly would experience more
difficulties in the understanding of the image than younger people
-0.05
* Correlation is significant
Content validity
Content validity is tested during this study by analyzing the answers of the open text option of the multiple-choice question. The multiple-choice question was: Would you recommend the use of imaging during the treatments of other patients? 104 respondents answered with the option ‘yes’ and only two respondents answered with ‘no’. The
respondents who answered with ‘no’ both explained their answers. One respondent explained that he would not recommend the use of imaging for other patients, because he prefers to see 3D images. This respondent filled in the version of VIPA about satisfaction with 2D images.
Another respondent explained he would not recommend the use of images for other patients, because it could differ for every patient if it is beneficial to show images.
49 respondents explained their answer or used this as a possibility to share other feedback. All the answers of this question can be found in Appendix F. The information from these answers was coded and the coding scheme is displayed in table 5. Seven codes were used to categorize the answers of the respondents. Six codes were related to topics which were already covered in VIPA. However, one new subject was found. This is the topic ‘the frequency of seeing images’ and it was coded twice.
Table 5: Coding Scheme
Code Number of
times used
Quotes of respondents (Translated from Dutch to English)
Visual addition 24 “It provides visualization next to the explanation of the doctor”
“You can see what is going on”
Clearness 25 “Provides insight”
“The explanation was clear”
Comforting 3 “Visual images are more comforting than only verbal explanation”
Want to see images more often
2 “I would have preferred to see images sooner, for example already during the second consultation”
Progress in recovery 7 “You can see the progress of recovery, something I really appreciate”
Satisfaction 25 “Great! Carry on”
Informative 8 “It was very informative to and I am interested in it”
Answers with no coding
3
Discussion
To date, no validated questionnaires are available to measure patient satisfaction about the use of 2D and 3D images during consultations with a medical specialist in the hospital.
The aim of this study was to further develop and field test the preliminary VIPA for its psychometric properties via item analysis, exploratory factor analysis (EFA), internal consistency and searching for associations of the questionnaire. Based on the outcomes, it is suggested that the resulting VIPA is a valid and reliable instrument to measure patients’
satisfaction with images that can be used for further research such as measuring differences in satisfaction about the use of 2D and 3D images among groups of patients. The final version of VIPA with an instruction can be found in appendix G.
The preliminary version of VIPA had 22 items and was developed using extensive literature review, interviews and observations. Nine items were deleted in the current field study based on stepwise psychometric analysis, so there are now thirteen items remaining in the final instrument. Two items were deleted during item analysis, six items were deleted during EFA and one item was deleted to improve the internal consistency of its respective subscale. Although nine items were deleted during this study, only one of the thirteen aspects from table 1 is not included anymore in the final version of VIPA. This is the aspect ‘relation to recovery’. This aspect was only related to item 11 (“Seeing the image was important for my recovery”), but the content of item 22 (“The image motivated to work on my recovery”) also seems related to the aspect ‘relation to recovery and this item is still included. Because the other twelve aspects are still included in VIPA and the aspect ‘relation to recovery’ seemed to be covered by item 22, it is not expected that the exclusion of items has led to a decrease in content validity. Further on, content validity was tested during this study by analyzing the answers of the open-ended question about advising other patients to see images during their treatment. Seven key topics were found during the coding process. Six subjects of these topics were already covered by items in the current VIPA. The subjects ‘visual addition’,
‘clearness’, ‘comforting’, ‘satisfaction’ and ‘informative’ were added to VIPA based on literature. The subject ‘progress in recovery’ was added to the preliminary VIPA based on interviews and observations (Pinkster, 2019). However, two patients mentioned one topic which was not covered in VIPA. This is the code ‘want to see images more often’. Although this subject was mentioned by respondents, it is not necessarily related to the aim of VIPA.
The aim of VIPA is to measure patient satisfaction about the actual use of 2D images, while
the topic ‘wants to see images more often’ is more about how frequently the medical specialists use images.
Item analysis did not indicate severe floor or ceiling effects. EFA resulted in a final factor structure of two different factors, which were interpreted as ‘the importance of seeing images’ with nine related items and ‘clearness of the image’ with four related items. Both factors showed adequate internal consistency. The inter-factor correlation was low with a weak correlation of 0.34. This indicated that the factors measure different subjects and it would not be justified to measure one total score (Taylor, 1989). Despite their weak correlation, both factors almost correlated equally with the overall satisfaction about imaging with 0.40 and 0.42. Therefore, it is suggested there is a relatedness between the factors. Still, the two subscales should be used independently to score VIPA.
Although no information is available in literature about a possible correlation between the two factors and patient satisfaction with imaging, the factor ‘clearness of the image’
seemed very similar to the aspect ‘clearness of information’, which was mentioned in table 1.
This aspect was found during the development of the preliminary VIPA and was added to the questionnaire based on interviews and observations with trauma patients. However, there was no aspect associated to ‘the importance of seeing images’. It would be possible that this subject is very broad and could be connected to multiple aspects of table 1, for example to expectations or understanding, because patients could find it important that their expectations are met or patients could find it more important to see images when images increase the understanding of their injury. Therefore, it is advised to further research qualitatively the factor ‘the importance of seeing images’, so it would be clear which subjects it measures and how this is associated to patient satisfaction about imaging. The aspects of table 1 that are related to the factor ‘the importance of seeing images’ could be further researched by gathering information from patients and medical specialists via interviews how these aspects could be related to ‘the importance of see images’.
EFA showed a two-factor structure with an explained variance of 49.52%. This is just below the advised explained variance of 50% (Pett, Lackey & Sullivan, 2003). It means that more than half of the variance in item responses is not explained by the two factors. A part of this unexplained variance could be influenced by personality traits (Hendriks, Smets, Vrielink, Van Es & De Haes, 2006). Therefore, it would be possible that some of the variance
in item responses is influenced by certain personality traits of patients. For that reason, it is advised to combine VIPA with a questionnaire about personality traits during further research, so it could be analyzed if a certain personality trait has a strong influence on the score of VIPA.
It was notable that all negatively formulated items had a low inter-item correlation or a negative inter-item correlation with the other items. As a result, these reversed items were all deleted from the final questionnaire based on the psychometric analyses. The items all together measure the topic patient satisfaction with 2D and 3D images, so it was expected for the items to have positive correlations with each other after recoding. Some items in the preliminary VIPA were reversed to avoid response style bias, which is the tendency for respondents to answer the items without paying enough attention to the content. This leads to answers which do not necessarily reflect the real opinions of respondents (Rorer, 1965).
Although item reversion was used to avoid response style bias, item reversion could on the other hand initiate response style bias. For example, when respondents do not read carefully, they could miss the subtle reversion and think they have to answer with agree to indicate satisfaction (Van Sonderen, Sanderman & Coyne, 2013). Research also found that respondents experience more difficulties in understanding a question when items are reversed (Swain, Weathers & Niedrich, 2008). Therefore, reversed items do not necessarily reflect the opinions of respondents and could actually be a threat for the validity of the questionnaire (Van Sonderen, Sanderman & Coyne, 2013). This could explain why the reversed items in the questionnaire had a low or negative inter-item correlation with the other items, but had adequate inter-item correlation with each other. Since the final VIPA consists of positively worded items only, likelihood of response bias or error is reduced.
It was difficult to test the preliminary VIPA for its construct validity, because no literature was available to state robust hypotheses about associations of patient satisfaction with imaging. Therefore, hypotheses were formulated based on patient satisfaction in general and based on the experiences of one medical specialist. From literature, it was assumed that age would have the strongest influence on patient satisfaction and that it would have a positive correlation with the two factors (Rahmqvist, 2001). This was also expected by the medical specialist. However, no significant correlation was found between age and the two factors. Further on, research of Hall & Dornan (1990) stated that lower education is related to higher levels of general patient satisfaction, so a negative correlation between the two factors
and levels of education was expected. The medical specialist did only expect a positive correlation between the factor ‘clearness of the image’ and levels of education, and did not expect a correlation between ‘the importance of seeing images’ and levels of education. The results indicated only a weak negative correlation between ‘the importance of seeing images’
and levels of education. In total, 12 tests for associations were executed between patient related variables and the two factors. Only one outcome matched the expectation of the medical specialist. Therefore, it is advised to further explore the construct validity of VIPA.
Research showed that health outcomes of patient are not associated with the experiences of patients (Manary, Boulding, Staelin & Glickman, 2013). Therefore, it is assumed that construct validity could be further tested to compare patients’ health outcomes with their scores on VIPA, and the hypothesis would be to find no association.
This study had some strengths. The first strength of this study was the sample size. On forehand, the aim was to include between 50 and 110 respondents. With 106 respondents, the number of 110 was almost reached. This number of respondents was especially important for executing robust PCA. Another strength of this study is the number of different medical specialists who explained the images to patients. It was chosen to use a fixed number of different medical specialists, because every medical specialist has its own way of discussing images with patients. Therefore, it was chosen to have a maximum of three medical specialist, so it could be analyzed if the different explanations could lead to differences in satisfaction.
The analyses showed no significant different scores of satisfactions between the three medical specialists. This indicated that the potential different explanations of these three medical specialists did not lead to significant differences in satisfaction. This was important, because the aim of VIPA is to measure patient satisfaction about imaging and not how satisfied patients are with the explanations of the different medical specialists. The last strong point was the high participations rates among patients. Almost every patient who was asked to participate, agreed to fill in VIPA. In this way it was avoided that the sample was selective or biased. The only way how this sample still could be selective, is if the medical specialists who participated in the study saw patients with specific injuries or fractures.
This study also had some limitations. First, it was not possible to compare VIPA with other satisfaction questionnaires about the use of 3D images, because VIPA is the first questionnaire to measure patient satisfaction about the use of 3D images as far as known.
Also, literature about patient satisfaction related to imaging was scarce. Therefore, it was not
possible to state robust assumptions based on literature to test the questionnaire for possible associations, so the hypotheses are based on experiences of only one medical specialist.
Second, the researcher was present to observe if respondents experienced any difficulties with completing VIPA, but this could also lead to socially desirable answers (Krumpal, 2013). It was chosen to be present during the completing of VIPA, because the presence of a researcher leads to higher response rates and respondents answer more often all the questions than when no researcher is present (Webster, 1997). It is unclear if and how the presence of the researcher influenced the answers of the respondents. Therefore, it is advised to perform follow up research without a researcher or medical specialist in the same room as the respondents, so the scores of satisfaction could be compared between both samples to analyze if the researcher’s presence has influence on the answers. Third, because the main focus of this study was to test VIPA for its psychometric properties, there was no intra-coder reliability assessment for the answers of the open text option. Therefore, it is possible that other persons could have come to different codes based on the answers of the open text option. Fourth, a low number of patients have seen 3D images during this study, which made it not possible to analyze differences in satisfaction between 2D and 3D images. Therefore, further research needs to indicate if VIPA can measure differences in satisfaction between patients who saw 2D images and patients who saw 3D images.
This study tested VIPA for its psychometric properties and the resulting questionnaire could be used in research now for measuring satisfaction among groups of patients about the use of 2D and 3D images. It is advised to perform Confirmatory Factor Analysis in future research, so the stability of this factor structure in another sample could be analyzed. In this study, VIPA was tested among trauma patients with bone fractures, but it could be further tested among other patient groups. It could also be tested if VIPA can measure differences in satisfaction between different types of 2D images, so for example between MRI, CT and x- ray. Although VIPA is tested for its psychometric properties and is ready to use in research, VIPA could be further validated for example for its reproducibility. In that case, VIPA could be filled in on several moments by the same patients and the scores of these different moments could be compared with each other via test-retest reliability (Terwee et al., 2007).
