University of Groningen The Measurement and Prediction of Physical Functioning after Trauma de Graaf, Max Willem

(1)

The Measurement and Prediction of Physical Functioning after Trauma

de Graaf, Max Willem

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.

Document Version

Publisher's PDF, also known as Version of record

Publication date: 2019

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):

de Graaf, M. W. (2019). The Measurement and Prediction of Physical Functioning after Trauma. Rijksuniversiteit Groningen.

Copyright

Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.

(2)

The Measurement and

Prediction of Physical

Functioning after Trauma

(3)

978-94-034-1500-0 (eBook)

Lay-out: M.W. de Graaf.

Cover:

J.M. de Graaf and M.W. de Graaf.

The cover picture is a two dimensional plot of a Clifford

Attractor, a (chaotic) strange attractor function created with

Visions of Chaos.

Printed by: Ipskamp Printing, Enschede.

© Copyright M.W. de Graaf, 2019.

All rights reserved. No part of this book may be reproduced or

trans-mitted in any form by any means, electronical or mechanical, including

photocopy, recording or any information storage or retrieval system,

without prior written permission of the copyright owner.

(4)

The Measurement and

Prediction of Physical

Functioning after Trauma

Proefschrift

ter verkrijging van de graad van doctor aan de

Rijksuniversiteit Groningen

op gezag van de

rector magnificus prof. dr. E. Sterken

en volgens besluit van het College voor Promoties.

De openbare verdediging zal plaatsvinden op

Maandag 27 mei 2019 om 16.15 uur

(5)

Prof. dr. E. Heineman

Copromotores

Dr. M. El Moumni

Dr. I.H.F. Reininga

Dr. K.W. Wendt

Beoordelingscommissie

Prof. dr. C.K. van der Sluis

Prof. dr. M.J.H. Verhofstad

Prof. dr. H.C.W. de Vet

(6)

B.T. de Jong, MSc

N.F. Osinga, MSc

(7)

(8)

(9)

Chapter 1

11 General Introduction

Chapter 2

33 Structural Validity of the Short

Musculoske-letal Function Assessment in Injured Patients

Chapter 3

61 The Short Musculoskeletal Function

Assess-ment: A Study of the Reliability, Construct

Vali-dity and Responsiveness in Patients Sustaining

Trauma

Chapter 4

87 Short Musculoskeletal Function Assessment:

Normative Data of the Dutch Population

Chapter 5

105 Pre-Injury Health Status of Injured Patients: a

Prospective Comparison with the Dutch

Popu-lation

Chapter 6

131 Minimal Important Change in Physical

Func-tioning in Trauma Patients: a Study using the

Short Musculoskeletal Function Assessment

(10)

The Development and Internal Validation of

a Model to Predict Functional Recovery after

Trauma

Chapter 8

177 General Discussion

Chapter 9

199 English Summary

Chapter 10

207 Nederlandse Samenvatting

Chapter 11

215 Dankwoord

Chapter 12

221 Over de auteur

(11)

(12)

Chapter 1

(13)

General Introduction

Measurement instruments play a key role in making diagnoses, prognoses and decisions in modern medicine. The field of trauma surgery is no exception. Whether one deals with a hemodynamically unstable patient or the recovery of physical functioning after a fractured elbow, measurements form the basis of a good understanding of the problem at hand.

In this thesis, many different aspects of a measurement instrument will be addressed. The focus lies specifically on evaluating the quality of a measure-ment instrumeasure-ment with regard to its capability to measure physical functioning in patients with a broad range of traumatic injuries. Specific characteristics of this population, such as heterogeneity of patient characteristics, types of injuries and injury severity, pose additional challenges to making and inter-preting measurements that are correct for the entire population.

Before the matter of the evaluation of measurement instruments is discus-sed in greater detail, it should be noted what measurement instruments are used for. They serve the process of understanding problems. It is therefore relevant to be aware that there are various kinds of problems (e.g. contexts of problems) that may be identified in the first place. Specific contexts of problems require a unique approach. Knowledge of these contexts may help place the findings of this thesis in a broader context. To this end, some back-ground on the various contexts of problems will first be provided.

Contexts of problems

Complex problems are fundamentally different from complicated problems, and require a fundamentally different approach in order to solve them, is what Kurtz and Snowden said when they developed the Cynefin framework in 1999.1, 2_{Cynefin is a conceptual framework that aids in}

under-standing problems and subsequent decision-making, by identifying contexts that require a different approach. In the framework four groups of contexts can be identified, ranked by the “amount” of uncertainty involved: simple/obvi-ous, complicated, complex and chaotic contexts.1_{Kurtz and Snowden pointed}

out that each context had its unique aspects, and that problems encountered within each of the different contexts would require a unique approach (Figure 1).1, 2_{The different contexts will be explained below.}

(14)

These contexts are generally accepted as “knowledge”. An example of a simple/ known context is the case of a 30-year-old female patient that fell from her bike and hit a sharp object with her ankle. This yielded a cut wound of 3 cm near the ankle. Cause and effect are clear. The solution of eliminating the defect logically follows, for example by using sutures.

Complicated/knowable contexts express a higher degree of uncertainty

than simple/known contexts. For a complicated problem there is a wide vari-ety of possible causes that can create an effect.1, 2_{Similarly to simple/known}

contexts, the set of causes are directly related to the effect and do not change over time or between individuals.2_{This is usually the domain of “experts”,}

as all possible causes are often only known by a select group of people.2_An

example of a complicated problem is a 20-year-old football player with an unstable ankle fracture he got after a tackle. Kurtz and Snowden pointed out that complicated problems can be “solved” by analyzing the problem and

reducing the possible causes.1-3_{The expert involved is a trauma surgeon that}

has the specific knowledge required for such problems. As there are many available treatments for ankle fractures, the fracture components and presence of accompanying injuries such as syndesmosis injuries have to be analyzed. A thorough understanding of the fracture components and the cause of the instability enables the surgeon to reduce the possible causes of the instability and choose a suitable treatment, for example by using plate osteosynthesis.

Problems become complex in the hypothetical case that the cause of the instability in the fracture cannot be deduced from observing the fracture components. Similarly to a complicated context, in a complex context a vari-ety of causes can create a specific effect.2_{However, in a complex context the}

number of causes and the strength of their relationship to the effect are not clear.4_{Often (a part of) the causes are non-observable latent factors that may}

vary from individual to individual.4_{An example of a complex context is the}

consideration of what treatment is most beneficial to the quality of life of a 59-year-old female with a nonunion after a right-sided ankle fracture. She has pain in the ankle and problems with walking and performing her activities of daily living. She also has obesity, diabetes mellitus type 2, and a status after a

(15)

causality, so that a sufficiently informed decision can be made.2, 4_{In relation}

to our example, the fracture and previous treatment itself should be evalu-ated, but also the extent of her problems with walking and performing daily

activities, management of chronic health conditions, coping style, expected recovery, occupation, personal life goals and possibly many other factors should be evaluated in an exploratory manner. In chaotic contexts there is no perceivable relation between cause and effect.1_{For the sake of this}

introduc-tion, the chaotic context will not be further discussed.

In healthcare, whether it is a general practice in a small village or a major university medical center, complex and complicated problems are both abun-dant.3 As described previously, complex problems require a different approach than complicated problems. In complex contexts, analytic reduction of the

(16)

possible causes, rather than exploration of the possible causes, is problematic.4, 5_{Inappropriate reductionism may lead to oversimplification of the problem}

and inappropriate conclusions.4, 5_{Referring to the complex example of the}

patient with a nonunion of a fractured ankle, reduction of the causative factors involved by concluding the bone has not healed, so therefore an additional surgical intervention is required, oversimplifies the problem and does not do justice to other factors involved. It can be concluded that healthcare providers should be aware of the complexity of medical contexts, and embrace rather than reduce and ignore its presence.3, 4

Biomedical and biopsychosocial model

In the past age of medicine, the biomedical model has been the predo-minant approach to solve clinical problems.3, 6_{The underlying hypothesis of}

the biomedical model is that all disease is a result of cellular abnormalities.6

This approach has led to a very detailed understanding of human anatomy and (patho)physiology, including the discovery of the human genome and the invention of antimicrobial agents.3, 6-8_{Though the exceptionally detailed}

understanding of these concepts is clearly relevant for many diseases, it does not appear to be the panacea to all medical problems. The biomedical model does not take multifactorial diseases or psychiatric diseases into account.6, 9

William Osler (1848-1919) already noted that “It is much more important to know what sort of a patient has the disease, than to know what disease the patient has”.10_{He meant that many more factors influence a patient’s health}

and accompanying medical decisions besides the observable disease. By realizing the complexity of medical problems, Osler was ahead of his time. In the 20th_{century criticism of the biomedical model increased, leading to the}

emergence of the biopsychosocial model,9_{a novel model that described that}

disease may be caused or influenced by biological, psychological and social factors, matching the determinants of health of the World Health Organiza-tion’s definition of health.11

How should we define health?

(17)

causes that affected health. However, in recent decades this definition has increasingly been criticized for being negative and reductionistic.6, 12, 13_The

definition implies that any person that is not in a state of complete physical, mental and social well-being is “unhealthy”.12

In 2011, Huber et al. proposed changing the definition to “the ability to adapt and self-manage”.12_{This definition was clearly broader than that of the}

WHO, and explicitly did not mention factors that would determine health, as these factors and their importance may vary per individual. One could say that Huber’s definition recognized and embraced the complexity of health.

Huber et al. suggested that the measurement of health status may be guided by conceptual frames that operationalize important aspects of health and relate to the ability to adapt and self-manage, such as the several classi-fication systems of the WHO.12_{The International Classification of}

Functio-ning, Disability and Health (ICF) is the classification system of the WHO that describes and relates health and functioning related problems.14_{In the ICF}

classification, three interconnected aspects of functioning and its problems are identified: 1) Body Function and Structure, 2) Activities and 3) Participation (Figure 2).14_{These aspects are related to contextual factors (environmental,}

personal). The domain Body Function and Structure refers to the anatomic and physiological functions of the human body (such as joint, muscular, respiratory function). The domain Activity refers to the execution of tasks or actions by an individual, such as doing household chores or sports. The Parti-cipation domain refers to involvement in life situations, such as taking part in a discussion or playing a game with others. The ICF classification describes the domains that should be assessed to evaluate functioning, but does not describe how the domains should be measured.15

Measurement in healthcare contexts

Measurement of health outcomes is essential in scientific research and clinical practice.16_{Taking into account that many medical problems are}

complex, measurements should honor the complexity of the domain of inte-rest. Traditionally, outcome of treatment has been assessed solely by means of measurements such as vital signs, range of motion of an affected joint, bone healing assessed by radiographic imaging, laboratory tests and morta-lity. The problem with these traditional outcome measures is that they are simple outcomes that explicitly relate to the ICF Body Functions and Structure

(18)

domain. However, they generally do not reflect the complex domains of Acti-vity or Participation. Whether there are problems in the ActiActi-vity and Participa-tion domains depends largely on a patient’s individual situaParticipa-tion. For example, a fractured ankle may pose much greater problems within these domains for a professional football player than for someone with an office job. Due to this individual dependence, patients themselves may provide the most accurate information to evaluate the Activity and Participation domains. Due to the shift in focus of what health is, the traditional clinical ways of measuring health and the effects of treatment are increasingly accompanied by patient-repor-ted outcome measures (PROMs), which are better suipatient-repor-ted for probing complex domains.17

PROMs are measurement instruments that are directly reported by a patient (i.e. the responses are not interpreted by a physician or anyone else).18

Though a PROM may be any type of measurement instrument, most are in the form of a self-reported or interviewer-administered questionnaire. PROMs provide unique information that cannot be obtained with traditional measu-res of health outcome. The key purpose of a PROM is to capture patients’

Figure 2: The International Classification of Functioning Disability and Health model.14

(19)

PROMs provide information that is useful in individual patient care, for example by assessing patients’ individual functional outcome after trauma. One can evaluate whether a patient recovers as expected or has reached a normal level of functioning. PROMs provide valuable information at other levels too, like when comparing treatment outcomes in research, or as quali-ty-control instruments for benchmarking performance of individual health-care providers and institutions.17

PROMs are increasingly used to capture perspectives of patients in research and clinical practice. To date, use of PROMs is a requirement for clinical trials funded by the US Food and Drug Administration (FDA).18_{In addition, since}

April 2009 the routine collection of PROMs is required by the National Health Services (NHS) in the United Kingdom to measure and improve clinical quality of specific elective surgical procedures such as hip and knee replacement and inguinal hernia repair.17, 19

Quality of health measurement instruments

Health measurement instruments, including PROMs, provide information from which clinical or organizational decisions are made.16, 20_These

instru-ments should therefore provide reliable and valid measureinstru-ments,16_otherwise

biased or inaccurate results are obtained which may lead to false conclusions and incorrect decisions. Since the constructs assessed with PROMs are often subjective and not directly observable, it is very important that the quality of the instrument is warranted.16_{To determine whether a PROM is of high}

quality, its clinimetric measurement properties have to be evaluated.16, 20_These

properties can be divided into several aspects: validity, reliability, internal consistency and responsiveness.21, 22_{Definitions of these aspects are provided}

in Table 1. It is important that the studies that evaluate these measurement properties are of high methodological quality, so that the conclusions regar-ding the instrument’s clinimetric properties are solid.16_{The COSMIN checklist}

(COnsensus-based Standards for the selection of health status Measurement INstruments) was recently developed; it contains quality criteria for studies that evaluate these measurement properties.22-24_{The authors of the COSMIN}

checklist have also provided guidance for researchers on how measurement properties should be evaluated.25

(20)

Trauma and injury-related disability

A specific group of patients that suffers from disability and problems with physical functioning are those who have sustained physical trauma. Trauma is the largest cause of death in adults younger than 45 and is frequently consi-dered an important cause of disability.26_{Due to a reduction in trauma-related}

mortality and an increased lifespan, the number of people living with inju-ry-related disabilities is growing.27_{In the Netherlands, the annual number of}

injured patients treated at emergency departments approaches 1.1 million.26

A Dutch study showed that injured patients who were hospitalized for more than three days still experienced many problems two years after the injury: 60% still had pain, more than 50% had problems with mobility and daily activities, and more than 25% suffered from depression or anxiety.28_{Many patients are}

not able to return to work due to their disabilities following injury, which has significant consequences for both individuals and society. The financial bill for society is high: total costs are estimated at 5.30 billion euros, of which 1.55 billion is direct medical care.29_{Insight into the determinants of both short-}

and long-term disability in a broad population of trauma patients can be used to prioritize the development of preventive policies and improve trauma care. In trauma, clinical outcome and treatment effectiveness have often been assessed solely by means of traditional outcome measures (i.e. range of motion, vital signs, injury severity score, mortality).30, 31_{These measures have}

shown to correlate poorly with patients’ view of their functioning and are poor determinants of long-term disability.32_{The reason that traditional outcome}

measures correlate poorly with functional outcome is that these outcomes are solely representative of the Body Functions and Structure domain of the ICF model. Physical functioning as perceived by the patient is not measured with the traditional outcomes.31_{They evaluate even less to which extent patients}

are bothered by their injuries and how the disability relates to their functio-ning and participation.

To date, limitations in physical, mental and social functioning after trauma have been scarcely studied. This is partly due to a lack of well-evaluated measu-rement instruments.33, 34_{While a large number of PROMs have been designed}

(21)

Developed instruments range from ones that evaluate very specific constructs (such as knee function) to instruments that evaluate very gene-ral constructs (such as genegene-ral health status). The choice of an appropriate measurement instrument should depend not only on the construct of interest but also on the population that is investigated.41_{A measurement instrument}

should be suitable for the specific target population.41_{The population of}

trauma patients is heterogeneous, since it exhibits a broad spectrum of inju-red body regions and injury severities. Hence when a broad range of injuinju-red patients is evaluated it is important that the instrument is applicable to the entire population. A PROM that can be used to evaluate physical functioning and that may be applied among a broad range of injured patients is the Short Musculoskeletal Function Assessment (SMFA).42

Short Musculoskeletal Function Assessment

In 1999, Swiontkowski et al. developed the SMFA as an instrument to assess physical functioning in patients with a broad range of musculoskeletal condi-tions.42_{The SMFA has been designed as an instrument that is not too specific,}

nor too general, and is therefore considered suitable for heterogeneic samples such as trauma patients.42, 43_{The SMFA contains 46 items and was developed}

as a shorter alternative to the 101-item-long Musculoskeletal Function Assess-ment.42_{The SMFA contains items that assess the physical functioning of the}

entire human body, problems with daily activities and psychological aspects of functioning. The SMFA has recently been translated into Dutch (SMFA-NL, Appendix 1).44

Treatment of trauma patients and recovery of physical functioning after trauma are complex contexts. Recalling from Kurtz and Snowden, in such contexts the causative factors involved have to be probed, in order to be able to make sufficiently informed (treatment) decisions.2, 4_{The SMFA may be eligible}

to probe various domains of physical functioning, in order to gain insight into the (recovery of) physical functioning of trauma patients. However, the clini-metric properties of the SMFA-NL have not been evaluated in a population consisting of patients with a broad range of acute traumatic injuries. Before any PROM can be used in another “new” patient population, the measurement properties of the instrument in this “new” population have to be evaluated.25

Hence an analysis of its clinimetric properties is needed before the SMFA-NL can routinely be used in patients with acute injuries.

(22)

Due to the nature of PROMs, the scores derived from the instrument may not be readily interpretable.45_{For example, it may not be directly clear which}

score represents a “healthy” or “normal” state. Scores of a PROM do not readily show which change in score may be considered meaningful either.45_{This poses}

an additional challenge for clinicians: interpretation of the data derived from PROMs. An appropriate interpretability of a (change in) score is a prerequisite for using a PROM in clinical practice. In order to make the scores interpreta-ble, additional information is necessary to place single scores and changes in scores within a clinical context.25

Aim and outline of this thesis

The aim of this thesis was to evaluate the quality of the SMFA-NL questi-onnaire with regard to its ability to assess physical functioning in patients that sustained a broad range of acute traumatic injuries. The thesis is divided into two parts. The aim of the first part was to evaluate the clinimetric properties (reliability, internal consistency, validity and responsiveness) of the SMFA-NL. The aim of the second part was to investigate the interpretability and prog-nostic performance of the SMFA-NL with regard to (recovery of) physical functioning after trauma.

Part I – Clinimetric properties of the SMFA-NL

The clinimetric properties of the SMFA-NL have not yet been assessed in patients with a broad range of acute traumatic injuries. Establishing these measurement properties is essential, as it explains the extent to which the SMFA-NL yields reliable and valid measurements of physical functioning in trauma patients. To evaluate whether the SMFA-NL can be used to assess functional status of trauma patients, various measurement properties will be evaluated according to the COSMIN criteria. In Chapter 2, the structural

vali-dity and internal consistency of the SMFA-NL were evaluated. In this chapter the number of latent constructs that may be measured using the SMFA-NL and the configuration of items that represent these constructs were evaluated using confirmatory factor analysis. In Chapter 3, the test-retest reliability,

(23)

that has good clinimetric properties is not necessarily a well-interpretable instrument. The aim of Chapter 4 was to create a benchmark for what is a

normal level of physical functioning by obtaining normative data of the Dutch population. In Chapter 5, pre-injury physical functioning of trauma patients

was evaluated and compared to the level of physical functioning of the Dutch population.

When trauma patients recover from their injuries, the scores on the subsca-les of the SMFA-NL may change over time. Which change in score may be regarded as clinically important is unclear though. In Chapter 6 the smallest

change in SMFA-NL score that is considered important to patients was evalu-ated for each subscale.

Measurement and interpretability of (change in) physical functioning are important to patients and physicians to justify the clinical use of an instru-ment. In Chapter 7 it was investigated whether recovery of physical

func-tioning after trauma can be predicted accurately. A prognostic model was developed with which functional recovery after trauma can be predicted using the SMFA-NL.

Chapter 8 provides a general discussion of the studies presented in this

thesis. Several theoretical and practical implications of the research and suggestions for future research are given.

(24)

References

1. Kurtz CF and Snowden DJ. The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Syst J 2003; 42: 462-483.

2. Snowden D and E Boone,Mary. A Leader’s Framework for Decision Making, 2007, p.68. 3. Sturmberg JP and Martin CM. Complexity and health – yesterday’s traditions, tomor-row’s future. J Eval Clin Pract 2009; 15: 543-548.

4. Kempermann G. Cynefin as reference framework to facilitate insight and decision-ma-king in complex contexts of biomedical research. Frontiers in neuroscience 2017; 11: 634.

5. Martin CM and Sturmberg JP. Perturbing ongoing conversations about systems and complexity in health services and systems. J Eval Clin Pract 2009; 15: 549-552.

6. Wade DT and Halligan PW. Do biomedical models of illness make for good healthcare systems?. BMJ 2004; 329: 1398-1401.

7. Watson JD and Crick FH. Molecular structure of nucleic acids. Nature 1953; 171: 737-738. 8. Tan SY and Tatsumura Y. Alexander Fleming (1881-1955): Discoverer of penicillin. Singapore Med J 2015; 56: 366-367.

Table 1: Definitions of clinimetric properties.21

Clinimetric property Definition

Validity The degree to which a PROM measures the construct* that is

in-tended to be measured.

Construct validity The degree to which the scores of a PROM are consistent with hy-potheses (for example with regard to other PROMs or differences between relevant groups), based on the assumption that the PROM validly measures the construct* that is intended to be measured. Structural Validity The degree to which the scores of a PROM are an adequate

reflecti-on of the dimensireflecti-onality of the creflecti-onstruct that is measured

Reliability The degree to which the PROM is free of measurement error.

Test-retest reliability The extent to which scores for patients who have not changed, are the same for repeated measurements.

Internal consistency The degree of interrelatedness among the items of a PROM, or its subscales.

Responsiveness The ability of a PROM to detect change over time in the construct* that is measured

Interpretability** The degree to which one can assign qualitative meaning to a PROM’s quantitative score or change in score.

*Construct: Construct refers to the theoretical concept that is intended to be measured, e.g. physical functioning for the SMFA-NL. ** Interpretability is not a formal measurement property, but regarded an important aspect of a measurement instrument.

(25)

11. Grad FP. The preamble of the constitution of the World Health Organization. Bull World Health Organ 2002; 80: 981-981.

12. Huber M, Knottnerus JA, Green L, et al. How should we define health?. BMJ 2011; 343: d4163.

13. Jadad AR and O’Grady L. How should health be defined?. BMJ: British Medical Jour-nal (Online) 2008; 337.

14. World Health Organization. International Classification of Functioning Disability and Health: ICF, Geneva, Switzerland 2001.

15. Boonen A, Stucki G, Maksymowych W, et al. The OMERACT-ICF Reference Group: integrating the ICF into the OMERACT process: opportunities and challenges. J Rheumatol 2009; 36: 2057-2060.

16. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010; 19: 539-549.

17. Devlin N and Appleby J. Getting the most out of PROMs: putting health outcomes at the heart of NHS decision-making. Monographs 2010.

18. US Department of Health and Human Services FDA Center for Drug Evaluation and Research, US Department of Health and Human Services FDA Center for Biologics Evaluation and Research and US Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health and Quality of Life Outcomes 2006; 4: 1-20.

19. Devlin NJ, Parkin D and Browne J. Patient-reported outcome measures in the NHS: new methods for analysing and reporting EQ-5D data. Health Econ 2010; 19: 886-905.

20. Mokkink LB, Terwee CB, Knol DL, et al. Protocol of the COSMIN study: COnsen-sus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol 2006; 6: 2.

21. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN study reached internatio-nal consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010; 63: 737-745.

22. Mokkink LB, Terwee CB, Knol DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol 2010; 10: 22-2288-10-22.

23. Terwee CB, Mokkink LB, Knol DL, et al. Rating the methodological quality in syste-matic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res 2012; 21: 651-657.

24. Prinsen CAC, Mokkink LB, Bouter LM, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018; 27: 1147-1157.

25. De Vet HCW, Terwee CB, Mokkink LB, et al. Measurement in medicine: Cambridge University Press, 2011.

26. Polinder S, Haagsma JA, Lyons RA, et al. Measuring the population burden of fatal and nonfatal injury. Epidemiol Rev 2012; 34: 17-31.

(26)

27. Van Beeck EF, Larsen CF, Lyons RA, et al. Guidelines for the conduction of follow-up studies measuring injury-related disability. J Trauma 2007; 62: 534-550.

28. Meerding WJ, Looman CW, Essink-Bot ML, et al. Distribution and determinants of health and work status in a comprehensive population of injury patients. J Trauma 2004; 56: 150-161.

29. Meerding WJ, Mulder S and van Beeck EF. Incidence and costs of injuries in The Netherlands. Eur J Public Health 2006; 16: 272-278.

30. Browner BD, Jupiter JB, Krettek C, et al. Skeletal Trauma E-Book: Elsevier Health Sciences, 2014.

31. Oltman R, Neises G, Scheible D, et al. ICF components of corresponding outcome measures in flexor tendon rehabilitation–a systematic review. BMC musculoskeletal disor-ders 2008; 9: 139.

32. Polinder S, Haagsma JA, Belt E, et al. A systematic review of studies measuring health-related quality of life of general injury populations. BMC Public Health 2010; 10: 783-2458-10-783.

33. Gabbe BJ, Williamson OD, Cameron PA, et al. Choosing outcome assessment instru-ments for trauma registries. Acad Emerg Med 2005; 12: 751-758.

34. Gruen R, Gabbe B, Stelfox H, et al. Indicators of the quality of trauma care and the performance of trauma systems. Br J Surg 2012; 99: 97-104.

35. van de Water ATM, Shields N and Taylor NF. Outcome measures in the management of proximal humeral fractures: a systematic review of their use and psychometric properties. Journal of Shoulder and Elbow Surgery 2011; 20: 333-343.

36. Schmidt S, Ferrer M, González M, et al. Evaluation of shoulder-specific patient-re-ported outcome measures: a systematic and standardized comparison of available evidence. Journal of Shoulder and Elbow Surgery 2014; 23: 434-444.

37. Dodd A, Osterhoff G, Guy P, et al. Assessment of functional outcomes of surgically managed acetabular fractures: a systematic review. Bone Joint J 2016; 98-B: 690-695.

38. Eechaute C, Vaes P, Van Aerschot L, et al. The clinimetric qualities of patient-assessed instruments for measuring chronic ankle instability: a systematic review. BMC musculos-keletal disorders 2007; 8: 6.

39. Jia Y, Huang H and Gagnier JJ. A systematic review of measurement properties of patient-reported outcome measures for use in patients with foot or ankle diseases. Quality of Life Research 2017; 26: 1969-2010.

40. Ng R, Broughton N and Williams C. Measuring Recovery After Ankle Fractures: A Systematic Review of the Psychometric Properties of Scoring Systems. The Journal of Foot and Ankle Surgery 2018; 57: 149-154.

(27)

43. Bouffard J, Bertrand-Charette M and Roy J. Psychometric properties of the Muscu-loskeletal Function Assessment and the Short MuscuMuscu-loskeletal Function Assessment: A systematic review. Clinical Rehabilitation 2015.

44. Reininga IH, el Moumni M, Bulstra SK, et al. Cross-cultural adaptation of the Dutch Short Musculoskeletal Function Assessment questionnaire (SMFA-NL): internal consis-tency, validity, repeatability and responsiveness. Injury 2012; 43: 726-733.

45. de Vet HC, Terwee CB, Ostelo RW, et al. Minimal changes in health status question-naires: distinction between minimally detectable change and minimally important change. Health and Quality of Life Outcomes 2006; 4: 1-5.

(28)

1 Hoeveel moeite heeft u met het in of uit een lage stoel komen?

Geen moeite Geringe moeite Matige moeite Veel moeite Onmogelijk om te doen

    

2 Hoeveel moeite heeft u met het openen van medicijnflesjes of –potjes?

    

3 Hoeveel moeite heeft u met het doen van uw dagelijkse boodschappen of anderszins winkelen?

    

4 Hoeveel moeite heeft u met traplopen?

    

5 Hoeveel moeite heeft u met het maken van een stevige vuist?

    

6 Hoeveel moeite heeft u met het in of uit de douche of bad stappen?

    

7 Hoeveel moeite heeft u met het makkelijk in slaap vallen?

    

8 Hoeveel moeite heeft u met bukken of knielen?

    

9 Hoeveel moeite heeft u met het gebruik van knopen, drukknopen, haakjes of ritsen?

    

10 Hoeveel moeite heeft u met het knippen van uw eigen vingernagels?

    

We zijn geïnteresseerd in hoe u deze week omgaat met de gevolgen van uw letsel(s) of aandoening(en). We willen graag weten of u hierdoor problemen ondervindt in uw dagelijkse bezigheden.

Graag elke vraag beantwoorden door het best passende antwoord aan te kruisen. Beantwoord alstublieft alle vragen, ook de vragen die ogenschijnlijk niet van toepassing zijn op uw letsel(s) of aandoening(en).

De volgende vragen hebben betrekking op hoeveel moeite u deze week heeft met dagelijkse activiteiten als gevolg van uw letsel(s) of aandoening(en).

Appendix

Appendix 1: The Dutch version of the Short Musculoskeletal Function

(29)

12 Hoeveel moeite heeft u met lopen?

    

13 Hoeveel moeite heeft u met het in beweging komen nadat u heeft gezeten of gelegen? Geen moeite Geringe moeite Matige moeite Veel moeite Onmogelijk om te doen

    

14 Hoeveel moeite heeft u met het zelfstandig de deur uit gaan?

    

15 Hoeveel moeite heeft u met autorijden?

    

16 Hoeveel moeite heeft u met het zelfstandig naar het toilet gaan?

    

17 Hoeveel moeite heeft u met het gebruiken van knoppen of hendels (bijvoorbeeld het openen van _{deuren of het open draaien van autoramen)?}

    

18 Hoeveel moeite heeft u met schrijven of typen?

    

19 Hoeveel moeite heeft u met het draaien om uw as?

    

20 Hoeveel moeite heeft u met het uitvoeren van uw gebruikelijke lichamelijke recreatieve _{activiteiten, zoals fietsen, hardlopen of wandelen?}

    

21 Hoeveel moeite heeft u met het uitvoeren van uw gebruikelijke vrijetijdsbesteding, zoals hobby’s, _{handwerken, tuinieren, kaarten of uitgaan met vrienden?}

    

22 Hoeveel moeite heeft u met seksuele activiteiten?

    

23 Hoeveel moeite heeft u met het verrichten van lichte huishoudelijke activiteiten of_{tuinwerkzaamheden, zoals afstoffen, afwassen of planten water geven?}

(30)

24 Hoeveel moeite heeft u met het verrichten van zware huishoudelijke activiteiten of_{tuinwerkzaamheden, zoals vloeren dweilen, stofzuigen of grasmaaien?}

    

25 Hoeveel moeite heeft u met het uitvoeren van uw dagelijkse werk, zoals een betaalde baan, _{huishouden of vrijwilligerswerk?}

    

26 Hoe vaak loopt u mank?

Nooit Zelden Soms Meestal Altijd

    

27 Hoe vaak vermijdt u gebruik van uw pijnlijke ledematen of rug?

    

28 Hoe vaak zit uw knie op slot of gaat u door uw knie?

    

29 Hoe vaak heeft u concentratieproblemen?

    

30 Hoe vaak beïnvloedt het te veel doen op een dag uw bezigheden van de volgende dag?

    

31 Hoe vaak gedraagt u zich geïrriteerd tegenover mensen om u heen (bijvoorbeeld mensen afsnauwen, kortaf reageren of snel bekritiseren)?

    

32 Hoe vaak bent u moe?

    

33 Hoe vaak voelt u zich lichamelijk beperkt?

    

34 Hoe vaak voelt u zich boos of gefrustreerd vanwege uw letsel(s) of aandoening(en)?

    

De volgende vragen informeren naar hoe vaak u deze week problemen ervaart, die veroorzaakt worden door uw letsel(s) of aandoening(en).

(31)

35 In welke mate wordt u gehinderd door problemen bij het gebruik van uw armen, handen of benen?

Geen hinder Geringe hinder Matige hinder Veel hinder Extreem veel hinder

    

36 In welke mate wordt u gehinderd door rugproblemen?

    

37 In welke mate wordt u gehinderd door problemen tijdens werkzaamheden rondom uw huis? Geen hinder Geringe hinder Matige hinder Veel hinder Extreem veel hinder

    

38 In welke mate wordt u gehinderd door problemen met douchen of in bad gaan, aankleden, naar het toilet gaan of andere persoonlijke verzorging?

    

39 In welke mate wordt u gehinderd door problemen met slapen en rusten?

    

40 In welke mate wordt u gehinderd door problemen bij vrijetijdsbesteding en recreatieve _{activiteiten?}

    

41 In welke mate wordt u gehinderd door problemen met uw vrienden, familie of andere belangrijke _{mensen in uw leven?}

    

42 In welke mate wordt u gehinderd door problemen met nadenken, concentreren of onthouden? Geen hinder Geringe hinder Matige hinder Veel hinder Extreem veel hinder

    

43 In welke mate wordt u gehinderd door problemen met aanpassen aan of omgaan met uw _{letsel(s) of aandoening(en)?}

    

44 In welke mate wordt u gehinderd door problemen met het doen van uw dagelijkse werk? Geen hinder Geringe hinder Matige hinder Veel hinder Extreem veel hinder

    

45 In welke mate wordt u gehinderd door problemen met het afhankelijk voelen van anderen? Geen hinder Geringe hinder Matige hinder Veel hinder Extreem veel hinder

    

46 In welke mate wordt u gehinderd door problemen met stijfheid en pijn?

    

De volgende vragen hebben betrekking op in welke mate u deze week gehinderd wordt door problemen vanwege uw letsel(s) of aandoening(en).

(32)

(33)

(34)

Structural Validity of the Short

Musculoskeletal Function

Assessment in Injured Patients

M.W. de Graaf

I.H.F. Reininga

K.W. Wendt

E. Heineman

M. El Moumni

Chapter 2

(35)

Abstract

Background: The Short Musculoskeletal Function Assessment (SMFA) is a

widely used patient reported outcome measure, originally having two elements of outcome: the function index and the bother index. In multiple studies it has been argued that the SMFA should be scored using three, four or six subsca-les instead. Hence there is inconsistency about the number of the underlying dimensions of the SMFA.

Objective: The aim of this study was to evaluate the structural validity of the

various proposed subscale configurations of the SMFA in a broad range of Dutch injured patients.

Design: A prospective cohort study.

Methods: Injured patients were asked to fill in the SMFA-NL at 5 to 8 weeks

post-injury. The structural validity of six different factor structures that have been proposed in other studies was evaluated using confirmatory factor analy-ses. Internal consistency was analyzed using Cronbach’s alpha.

Results: A total of 491 patients participated (response rate: 74%). The

four-fac-tor structure of Reininga et al. showed an acceptable fit (RMSEA = 0.070, CFI = 0.973, TLI = 0.971). Other models, including the original 2-index structure, showed insufficient structural validity in Dutch injured patients. The four-fac-tor structure showed sufficient discriminant validity and good internal consis-tency (Cronbach’s alpha ≥ 0.83).

Limitations: It is unclear whether conclusions are generalizable across

diffe-rent countries, elderly and non-injured patients.

Conclusion: In a broad range of injured patients, the SMFA-NL may be best

scored and interpreted using a four-factor structure. Other factor structures showed insufficient structural validity.

(36)

Introduction

Injuries are a large contributor to the international burden of disease.1-3

In the treatment of injured patients, traditional outcome measures such as x-ray recordings and range of motion do not accurately reflect the patients’ perspective on their functioning.4,5_{Patient Reported Outcome Measures}

(PROMs) have increasingly gained attention: PROMs have been incorporated in clinical trial guidelines6_{and regular care procedures that require PROMs}

as quality control.7

When a heterogeneous group of injured patients is evaluated, a general musculoskeletal outcome measure may be used. In 1999, Swiontkowski et al. developed the Short Musculoskeletal Function Assessment (SMFA) as an outcome measure to evaluate physical function of patients with a broad range of musculoskeletal disorders, including injured patients.8_{The SMFA}

was originally designed to evaluate two latent constructs: patients’ physical status and how bothered they are by functional problems due to the muscu-loskeletal conditions. Hence it was originally divided into two basic elements of outcome: the function index and the bother index. Later, the SMFA has been translated and cross-culturally adapted into multiple languages.9-16_In

some cross-cultural validation studies it was argued that the SMFA may be interpreted by three14,15_{, four}10,11_{or six}9_{subscales instead of the original two}_.8

The validity of the different configurations of subscales (i.e. structural validity) has rarely been studied and resulted in inconsistency about the number and nature of the latent constructs that are evaluated with the SMFA.

Structural validity is an important aspect of validity that concerns the vali-dity of a factor structure of a PROM.17_{The factor structure defines the number}

of latent constructs (i.e. number of subscales) that may be evaluated and the configuration of items that represent these constructs (Figure 1). Therefore structural validity guides how a PROM should be scored and interpreted.

The aim of this study was (1) to investigate the structural validity and inter-nal consistency of the various proposed subscale configurations of the SMFA in Dutch patients with a broad range of acute injuries and (2) to identify the

(37)

Figure 1: Model 4, the four-factor structure of the SMFA-NL

In circles S-1 to S-4: the latent constructs that are measured as subscales: Lower Extremity Dysfunction, Upper Extremity Dysfunction, Problems with Daily Activities and Mental and Emotional Problems. Square boxes 1, 2, …, 46 represent SMFA item 1, 2,…, 46 belonging to the specific latent construct. Curved lines represent correlations between subscales, straight lines show which item loads on which construct i.e. subscale.

(38)

Methods

Patients

Patients were recruited at the Trauma Department of the University Medi-cal Center Groningen (The Netherlands), a level-1 trauma center. Patients that presented with one or more acute injuries and required a follow-up treatment for at least five weeks at the trauma surgery outpatient clinic were prompted for inclusion. Exclusion criteria were: age under 18 or above 67 years, not able to read or write Dutch, severe neurological deficits, severe traumatic brain injury, pathologic fractures and severe psychiatric conditions. Patients recei-ved the standard Dutch translation of the SMFA11_{(SMFA-NL) questionnaire}

5-8 weeks after the injury, in which patients reported their functioning of the past week. Patients had either been treated surgically or conservatively. Patients received the questionnaire on paper or electronically; non-responders were reminded once.

The methods employed in this study have been reviewed by the local Institutional Review Board, and waived further need for approval. Patients consented with the participation in this study. The study was carried out in compliance with the principles outlined in the Declaration of Helsinki on ethical principles for medical research involving human subjects.

Questionnaire and theoretical framework

The SMFA was developed as a shorter alternative to the 101-item Musculos-keletal Function Assessment (MFA), in order to enhance clinical usability.8,18

Both questionnaires rest on the same theoretical framework. The question-naires were developed to assess physical functioning of patients with a broad range of musculoskeletal conditions. The SMFA was designed as an instrument that was not too general, nor overly specific. Items that were often overlooked were incorporated, such as coping, adaptation and acceptance. Four primary categories were used: upper extremity, daily activities, mobility and mental and emotional functioning. Together these categories made-up the function index. The bother index was added to assess the extent to which patients are bothered due to their conditions.

(39)

and cross-culturally adapted into Chinese, Danish, Dutch, German, Korean,

Portuguese, Spanish and Swedish.9-16

Evaluation of structural validity

Structural validity has been defined as: the degree to which scores of a PROM are an adequate reflection of the dimensionality (i.e. the expected number of subscales) of the construct that is measured.19_{A construct can}

be regarded as the “hidden variable” that cannot be measured directly, but can be measured through multiple other measurements. For example the construct “lower extremity function” cannot be measured directly, but it can be measured by multiple items of a PROM that evaluate the aspects of lower extremity function.

Factor analysis is a frequently used technique to evaluate a set of latent constructs underlying the items of a PROM.20_{There are two main types of}

factor analysis. The first type, exploratory factor analysis (EFA) may be used when there is no clear idea of how many constructs are represented by a PROM and which items represent the specific constructs.21_{In some of the different}

cross-cultural validation studies of the SMFA, EFAs were used to explore the factor structure of the SMFA.10,11,14,15,22_{Different factor structures were}

repor-ted, which caused unclarity about the number of subscales and which items represent these subscales. An EFA provides limited information regarding the structural validity of the found factor structure, nor can it be used to compare the structural validity of different factor structures.

The second type of factor analysis, confirmatory factor analysis (CFA), over-comes these limitations. In a CFA, explicit relationships between the items in the questionnaire and the constructs that may be evaluated are pre-specified, e.g. the factor structures of the SMFA that were reported in earlier studies. CFA tests how well the data fits the pre-specified factor structure. When the pre-specified factor structure yields an improper ‘goodness of fit’ with the data, the model is rejected.21_{For example, a PROM in which a single score is used, it}

is critical to demonstrate a good fitting one-factor structure. In this study, CFA was used to confirm and validate the different factor structures of the SMFA.

Models

The path diagrams of the analyzed factor structures are shown in Figure 1 and Appendix A. To aid the interpretation of the factor structures, a list of

(40)

Model 1: The original two-index factor structure is the most widely used

method of interpreting the SMFA.23,24_{The function index consists of 34 items}

and the bother index consists of 12 items. Although construct validity, test-re-test reliability and responsiveness have been evaluated, structural validity of the original two-index structure has not been evaluated.

Model 2: In the Mexican cross-cultural validation study, Guevara et al.15

conducted a principal component analysis and reported a three-factor solu-tion. The obtained factors were: upper extremity function, lower extremity function, and daily activities. In their analysis items 14, 16, 29, 31 and 38 were dropped.

Model 3: In the Brazilian cross-cultural validation study, Taylor et al.14

conducted a principal component analysis and found that a different three-fac-tor model fitted best. Subscales were named: upper extremity dysfunction, lower extremity dysfunction, and bother. In their analysis items 7, 15, 23, 30, 32, 35, 37, 45 were dropped.

Model 4: In the Dutch cross-cultural validation, Reininga et al.11_conducted

a principal component analysis and proposed a four-factor structure, contai-ning all items of the SMFA. Subscales were named: upper extremity dysfunc-tion, lower extremity dysfuncdysfunc-tion, problems with daily activities, and mental and emotional problems.

Model 5: In the Chinese cross-cultural validation, Wang et al.9_{reported a}

model that consisted of six subscales. Subscales were: daily activities, mobi-lity, arm and hand function, emotional status, sexual activity & driving a car, and difficulties with falling asleep. Item 36 was excluded from the final model. Although Model 5 is overidentified (df = 933), the subscales difficulties falling asleep, and sexual activity and driving are defined by only one and two items respectively. This low number of items per subscale creates susceptibility to empirical underidentification, e.g. preventing the analysis from obtaining a valid and unique set of factor loadings.21

Model 6: In the Danish cross-cultural validation, Lindahl et al.10

(41)

Statistical analysis

Sample size

It has been recommended to include at least 7 patients per item when the structural validity of a PROM is investigated.25_{Our aim was to include at least}

460 patients (10 patients per item of the SMFA).

Confirmatory Factor Analysis

The confirmatory factor analyses were performed using the R package Lavaan version 0.5-18.26,27_{All models were evaluated conform the Correlated}

Factors Model: each item was restricted to load on one factor and covariance was expressed between factors.21,28_{Factor loadings, error variance and factor}

covariance were freely estimated. The weighted least squares means and

variances adjusted (WLSMV) estimator was used. The WLSMV estimator is robust to non-normality and is recommended when categorical indicators are used.29_{Missing data were handled pairwise. Completely standardized factor}

loadings were calculated.

The model-implied and population variance-covariance matrices of each model were compared using chi-squared tests.28_{The chi-squared test is a}

global test of model fit, however it is considered to be overly strict and sample size sensitive.21,28_{To evaluate model fit, other goodness of fit indices were}

examined: the Root-mean-square error of approximation (RMSEA), compa-rative fit index (CFI) and, Tucker-Lewis index (TLI). Cut-off values that indi-cated an acceptable fit were guided by Hu and Bentler30_{and Steiger}31_{: RMSEA}

≤ 0.07, CFI ≥ 0.95, and TLI ≥ 0.95. A model fit that did not meet all thresholds was considered an unacceptable fit. In addition to fit indices, we evaluated the magnitude, direction and significance of factor loadings of all models. Factors were considered to show sufficient discriminant validity when between-factor correlations ≤ 0.85.21_{There are no strict guidelines for factor loadings, although}

factor loadings ≥ 0.4 were considered salient.21 Internal consistency

Internal consistency refers to the degree of interrelatedness among the items on a scale. Cronbach’s alpha was calculated for each subscale of the evaluated models to evaluate internal consistency. It is widely accepted that Cronbach’s alpha should be ≥ 0.70.32

(42)

General characteristics N (%) Gender (n=491) Male 276 (56) Female 215 (44) Age groups (n=491) 18-24 82 (17) 25-34 72 (15) 35-44 87 (18) 45-54 104 (21) 55-67 146 (30) Marital status (n=464) Single 191 (41) With partner 273 (59) Educational level (n=462) Elementary school 10 (2) High school 150 (32) College 136 (28)

Bachelors degree or higher 160 (36)

Other 6 (1)

Chronic health conditions (n=452)

None 247 (55) One 115 (25) Two 54 (12) Three or more 36 (8) Injuries (N=491) Fracture Upper extremity 164 (33) Lower extremity 145 (30)

Pelvis and sacrum 25 (5)

Spine 27 (6)

Other 0 (0)

Luxation and rupture 40 (8)

Sprain and Contusion 49 (10)

Head injury 3 (1)

Wounds and soft tissue 16 (3)

Organ injury (incl. pneumothorax) 12 (2)

Other 10 (2)

(43)

injury types are presented in Table 1. A total of 164 (33%) patients had an upper extremity fracture and 145 (30%) patients had a lower extremity fracture. Most patients reported they had no chronic health conditions (Table 1). Item 15 and 22, which respectively regarded driving a car and sexual activity, were missing in 2.9% and 2.6%. All other items were missing in less than 2%.

Confirmatory factor analyses

Model fit

All analyses succeeded without errors, except for Model 3 and 5. In the first run, the estimation of the factor loadings of items 16 and 38 in Model 3 and item 7, 11 and 33 in Model 5, yielded a negative error variance and completely standardized factor loading with a value greater than 1.0. This is a theoretically improper solution, known as a Heywood case. The factor loadings of these items were sequentially constrained to 1.0 and models were re-analyzed.33_Both

models yielded a proper solution.

Model 4 was the only model that showed an acceptable fit (RMSEA = 0.070, CFI = 0.973, TLI = 0.971). The fit indices of Models 1, 2, 3, 5 and 6 did not meet the pre-specified thresholds for an acceptable fit. (Table 2).

Factor loadings

The factor loadings of all evaluated models are shown in Appendix C to H. Most factor loadings of Model 4 were higher than 0.80. All factor loadings were > 0.4, statistically significant and positive. The covariance between the individual factors of Model 4 was smaller than 0.85. Indicating there was

Models χ² Df P-value RMSEA

≤0.07 RMSEA 90% CI ≥0.95CFI ≥0.95TLI

Model 1 8833.800 988 < 0.001 0.127 0.125 0.130 0.909 0.905 Model 2 7230.257 776 < 0.001 0.130 0.128 0.133 0.920 0.915 Model 3* 5367.140 664 < 0.001 0.120 0.117 0.123 0.928 0.923 Model 4 3351.996 983 < 0.001 0.070 0.068 0.073 0.973 0.971 Model 5* 5201.451 933 < 0.001 0.097 0.094 0.099 0.951 0.948 Model 6 6080.009 983 < 0.001 0.103 0.100 0.105 0.941 0.938

Bold: models that showed an acceptable fit (RMSEA ≤ 0.07, CFI ≥ 0.95, and TLI ≥ 0.95). * Constrained one or more error variances to zero.

(44)

The majority of the factor loadings of Model 1 ranged between 0.6 and 0.9. The function index contained four items that had factor loadings smaller than 0.4 (item 5 and 28). Factor loadings of Model 2 mainly ranged between 0.5 and 0.9. One item showed a factor loading < 0.4 (item 21). Model 3 showed factor loadings ranging between 0.7 and 0.8 and one loading was smaller than 0.4 (item 28). Model 5 showed factor loadings generally between 0.7 and 0.9. All factor loadings were ≥ 0.4. Model 6 showed factor loadings that mainly ranged from 0.7 to 0.9. The factor loadings of this model were all ≥ 0.4.

Internal Consistency

Cronbach’s alpha values are shown in Table 3. Cronbach’s alpha was ≥ 0.83 for all subscales of Model 4. Model 1,2,3 and 6 showed sufficient internal consistency on all subscales. Model 5 showed insufficient internal consistency of subscale 5 (sexuality and driving, Cronbach’s alpha = 0.68). The subscale “difficulties with falling asleep” of Model 5 was not calculable since it contained only one item (item 7).

Discussion

It is important that measurements taken with a PROM, are based on a valid underlying factor structure. Since its introduction, the original two-index

Table 3: Internal consistency.

Models Subscale 1 2 3 4 5 6 Model 1 0.96 0.93 Model 2 0.94 0.94 0.93 Model 3 0.96 0.88 0.85 Model 4 0.95 0.96 0.97 0.87 Model 5 0.96 0.91 0.90 0.81 0.68 n/a Model 6 0.96 0.96 0.90 0.95

Cronbach’s alpha calculated per subscale. Subscale numbers (n) are the same as in the path models and factor loading tables (Appendix A and C to H). Bold: models that showed an acceptable fit (RMSEA ≤ 0.07, CFI ≥ 0.95, and TLI ≥ 0.95). N/a: not applicable.

(45)

range of acute injuries. A model with four subscales11_{provided evidence of}

structural validity of the SMFA-NL questionnaire.

The four-factor model of Reininga et al.11_{(Model 4) showed an}

accep-table goodness of fit with generally good to excellent factor loadings. The subscales showed sufficient discriminant validity, indicating that all evaluated constructs are sufficiently different from each other. Internal consistency was sufficient, although the three of the four subscales (upper and lower extremity

dysfunction and problems with daily activities) showed Cronbach’s alpha values

> 0.95, which may indicate that there are redundant items in these scales. However removal of items was beyond the scope of this study.

The original two-index (Function Index and Bother Index) model has been investigated and used extensively in clinical settings and research.23,24

Howe-ver, in this study it showed an unacceptable goodness of fit. The two-index model was originally derived in a similar sample of patients, of which most sustained an injury.8_{Upon the development of the SMFA, the Function and}

Bother Index were considered to reflect conceptually different constructs, of which the Function Index was a more objective measure of physical function and the Bother Index more subjective.8_{For instance, patients could report}

that their knee locked just ‘some of the time’, whilst being extremely bothered by it. The distinction of function and botheredness was based on theoretical grounds, but was not verified with a factor analysis. This may have been the cause of the insufficient structural validity. The findings of this study suggest that the SMFA-NL does not measure these constructs separately. The various translation studies of the SMFA that performed an EFA did not find the two-in-dex structure either.

Models 2 (Guevara et al.15_{) and 6 (Lindahl et al.}10_{) showed an unacceptable}

and were therefore considered to show insufficient structural validity. The model of Lindahl et al.10_{was derived in a sample that consisted half of acutely}

injured patients and half of rehabilitation patients with various musculoske-letal conditions, which may have contributed to the unacceptable fit of the model.

Models 3 and 5 (Taylor et al.14_{and Wang et al.}9_{) did not converge due to}

multicollinearity and empirical underidentification respectively. Although constraining the error variance to zero may be regarded a ‘quick fix’, the underlying problems should be addressed. This was beyond the scope of this

(46)

study. Both models showed insufficient structural validity. The study sample of Taylor et al.14_{was similar to the present study. The insufficient fit of the model}

may have been caused by the omission of several items of the questionnaire or cross-cultural differences. The model of Wang et al. was derived in a study sample that contained only a minor fraction of injured patients.34_{Aside from}

the non-convergence, the internal consistency and clinical relevance of the sexuality & driving, and difficulties falling asleep subscales may be a concern for the model of Wang et al.

Van Son et al.35_{have performed an exploratory factor analysis in Dutch}

patients and proposed two three-factor structures separate for upper and lower extremity fractures. In that study, double-barrel items were split. This chan-ges the items of the questionnaire and makes comparison with other studies difficult. The models could therefore not be evaluated in the present study.

A clinical implication of the present study is that it showed that the SMFA-NL may be used best to evaluate four latent constructs using the subsca-les upper extremity dysfunction, lower extremity dysfunction, problems with

daily activities and mental and emotional problems. To enable use of these

subscales in injured patients in a clinical setting or in applied research, additi-onal clinimetric measurement properties such as construct validity, test-retest reliability and responsiveness of the subscales should be evaluated.17

A limitation of this study is its generalizability. The study sample consi-sted of patients of the working-age population that suffered an acute injury. Therefore, it is not clear whether this factor structure can be applied in patients with other musculoskeletal conditions or in elderly. The present study was performed in a Dutch population using the SMFA-NL questionnaire. It is not clear whether the four-factor solution is valid for other countries. Factor struc-tures that showed an unacceptable fit have all been conducted in non-Dutch patients. These models may show sufficient structural validity when evaluated in the original country. We encourage further international evaluation of the structural validity of the SMFA.