University of Groningen Hand eczema Oosterhaven, Jart

(1)

University of Groningen

Hand eczema

Oosterhaven, Jart

DOI:

10.33612/diss.98242014

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.

Document Version

Publisher's PDF, also known as Version of record

Publication date: 2019

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):

Oosterhaven, J. (2019). Hand eczema: impact, treatment and outcome measures. https://doi.org/10.33612/diss.98242014

Copyright

Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.

(2)

Chapter 8

Guideline for translation and

national validation of the Quality Of

Life in Hand Eczema Questionnaire

JAF Oosterhaven, MLA Schuttelaar,

C Apfelbacher, TL Diepgen, RF Ofenloch.

(3)

ABSTRACT

There is a need for well-developed and validated questionnaires to measure patient reported outcomes. The Quality of Life in Hand Eczema Questionnaire (QOLHEQ) is such a validated instrument measuring disease-specific health-related quality of life in hand eczema patients. A re-validation of measurement properties is required before an instrument is used in a new population. With the objective of arriving at a guideline for translation and national validation of the QOLHEQ, we have developed the design of a reference study on how to adequately assess measurement properties of the QOLHEQ, based on interdisciplinary discussions and current standards. We present a step-by-step guideline to assess translation (including cross-cultural adaptation), scale structure, validity, reproducibility, responsiveness and interpretability. We describe which outcomes should be reported for each measurement property, and give advice on how to calculate these. It is also specified which sample size is needed, how to deal with missing data, and which cut off values should be applied for the measurement properties assessed during the validation process. In conclusion, this guideline, presenting a reference validation study for the QOLHEQ, creates the possibility to harmonize the national validation of the various language versions of the QOLHEQ.

(4)

8

INTRODUCTION

There is a need for well-developed and validated questionnaires to measure patient reported outcomes such as health-related quality of life (HRQOL).1_{The Quality of Life in Hand Eczema}

Questionnaire (QOLHEQ) is a disease specific instrument that measures the construct of HRQOL in hand eczema patients. It was developed in an English-language version by a group of international experts in collaboration with patient focus groups for use in clinical trials and to monitor individual patients. The questionnaire consists of 30 questions in four subscales covering impairment because of (a) symptoms, (b) emotions, (c) functioning and (d) treatment/ prevention. In a large German hand eczema population, the QOLHEQ was found to be a valid and reliable instrument for assessing HRQOL.2_{However, in principle, measurement properties}

need to be assessed each time that an instrument is used for measurements in a population that could differ from this original population, to provide evidence that it really measures what it is supposed to measure (in case of the QOLHEQ, the construct of HRQOL). This goes hand-in-hand with cross-cultural adaptation/validation and, in most situations, with translation. Such validation studies are often carried out in very different ways or sometimes not carried out at all, possibly because of insufficient understanding of the procedures involved. This leads to situations in which non-validated measurements from different language versions or different populations are used and compared. We present this guideline to offer a standardized way to translate and validate the QOLHEQ for use in a population that is different from the population in the original validation study. This approach can also be applied to the validation of other measurement instruments, especially those that measure HRQOL.

Readers should recognize that clinimetrics (the measurement of clinical phenomena) is a challenging field with its own (sometimes ambiguous) terminology and statistical and methodological difficulties. This article is intended to give an overview of the requirements for translation and national validation of (HRQOL) questionnaires, using the QOLHEQ as an example. It can, however, only partly substitute for a deeper study into clinimetrics (e.g.

Streiner, Norman and Cairney)3_{, which we deem to be necessary before carrying out the}

validation study described below.

This article is composed of two parts. First, we describe the practical procedure, and second, we suggest an approach for the analyses of measurement properties. We will basically follow the guidelines developed by the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) group, which mainly deal with taxonomy, definitions, and the theory of health-related patient-reported outcomes, and the International Quality of Life Assessment project approach.4–6

TRANSLATION AND CROSS-CULTURAL ADAPTATION

The first step in adapting a questionnaire for use in another population is, in most cases, translation into the target language. We recommend carrying out the translation with the method suggested by Wild et al., condensed into six steps (Figure 1). Every step needs to be documented carefully.7_{Templates for this can be found in the online supplement (Supplement}

S1).

In the first stage, the original English version of the QOLHEQ is translated by two translators (T1 and T2) with the target language as their mother language. One translator should have a degree of expertise on the subject of hand eczema and should be aware of the definition of HRQOL, as well as the multidimensional construct of the QOLHEQ, which is shown in Figure 2 as described in its validation publication.2_{The other translator (T2) should}

(5)

Figure 1 Overview of translation process. Adapted from: Beaton et al.8

In the second stage, the two translated versions are integrated into one (version T-12) by the two translators, the researcher who guides the process, and a health professional with expertise in hand eczema and HRQOL.

In the third stage, we recommend a ‘light’ version of a back-translation of the T-12 version. Only items that were translated differently and cannot be resolved easily and/ or provoke discussion need to be translated back (BT1 and BT2) by two language-expert translators with English as their first language. They should be blinded to the original version of the QOLHEQ, and should preferably not have a medical background. They should not be aware of the definitions of the construct and/or individual items.

In the fourth stage, all versions (T1, T2, T-12, BT1 items and BT2 items) are reviewed by an expert committee, minimally consisting of a methodologist, a health professional (expert on hand eczema and HRQOL), the researcher and translators. Contact with the developers of the QOLHEQ should be sought in case of doubt about whether items have maintained their intended meaning. This will result in a pre-final language version of the QOLHEQ.

In the fifth stage, the pre-final version is tested in a cognitive interview pilot study9

with a minimum of 15 hand eczema patients with varying disease severities, to assess content validity: comprehensibility, relevance, completeness, and cultural difficulties/differences. Patients complete the QOLHEQ and need to be interviewed about each question and their should not be given the definition of the construct and/or individual items, in order to remain ‘naive’.

(6)

8

given response, with the following questions in mind:

• Do patients understand and interpret the items and responses as they were intended by the developers and expert committee?

• Are all items relevant for the patient group? • Do patients miss any essential items?

• Are there cultural incompatibilities in the questionnaire?

It should be noted that the development group of the QOLHEQ consisted of experts from Sweden, Japan, Germany, Finland, Denmark and Australia. Considering this, we do not expect significant cultural incompatibilities or missing essential items in the questionnaire for these countries. However, if necessary, the wording of the QOLHEQ items can be improved in order to ensure cross-cultural equivalence at this stage. This is done by the researcher, and should be carefully documented. Final proofreading by all members of the expert committee is strongly advised.

The sixth stage is the final step in the translation process, in which the original developer and, optionally, the expert committee review all written reports one last time. In this phase, we recommend that all documentation should be sent to the developers for a process audit. After approval, the translated QOLHEQ is ready to be tested in the target population

(7)

to assess the following measurement properties: scale structure, validity, reproducibility, and responsiveness. Interpretability (the degree to which one can assign qualitative meaning to a score or change in score) is not a measurement property, but an important characteristic of any measurement instrument, and should therefore be assessed in each population as well.

DESIGN OF THE VALIDATION STUDY

To assess the measurement properties of the QOLHEQ, a longitudinal study design is needed. Table 1 shows an overview of the required steps.

SAMPLE SIZE AND PATIENT CHARACTERISTICS

As there is no best practice on how to calculate a sample size for validation studies, we suggest an item/subject ratio of 1:10, which leads to a sample size of ≥300.10_{This is sufficient}

to detect small effects (effect size d ≥0.20) in hypotheses testing with a p-value of <0.05 and a power of 90% (calculated with G*POWER 3.1.9).11_{It is strongly advised to strive to include}

a heterogeneous population, that is, a population that includes all stages of disease and/or treatment. The following characteristics of the study population need to be described: age, sex, and the scores (mean ± standard deviation [SD]) of the instruments mentioned below, both for the total group and stratified by gender and age groups.

The following inclusion criterion applies:

• Adult patients (≥ 18 years) with hand eczema of at least 1 week in duration, as diagnosed by a physician. (Note that patients with concomitant diseases can be included, since the QOLHEQ specifically addresses the hand eczema.)

Table 1 Overview of the validation study T₀ inclusion of subjects

Single-score validity TReproducibility1 after 1-3 days TChange score validity (Responsiveness)2 after 4-6 weeks

Questionnaires to be completed by

subjects Questionnaires to be completed by subjects Questionnaires to be completed by subjects

• Demographics • QOLHEQ

• Score hand eczema severity  Suggested: Photoguide • Skin specific HRQOL

questionnaire(s)

 Suggested: DLQI, Skindex • Generic HRQOL questionnaire(s)

 Suggested: EQ-5D, Short Form

• Anchor questions (overall and subscales)

• Global Rating of Change scale

• QOLHEQ • Anchor questions

(overall and subscales)  Only in changed

patients

• Global Rating of Change scale • QOLHEQ

• Repeat skin specific HRQOL questionnaire(s)

• Repeat generic HRQOL questionnaire(s)

• Anchor questions (overall and subscales)

 Only in changed patients

Assessed by physician Assessed by physician

• Hand eczema severity assessment

 Suggested: Photoguide, HECSI

• Repeat hand eczema severity assessment

DLQI, Dermatology Life Quality Index; EQ-5D, Quality of Life Questionnaire of the Euro-QOL Group; HECSI, Hand Eczema Severity Index; HRQOL, health-related quality of life; QOLHEQ, Quality Of Life in Hand Eczema Questionnaire

(8)

8

Exclusion criteria comprise:

• Other dermatological hand disease (e.g. psoriasis).

• The patient is not able to complete the questionnaire by himself or herself. REFERENCE INSTRUMENTS TO BE TESTED

To be able to assess all relevant measurement properties, patients need to follow the schedule of assessments shown in Table 1. All eligible patients can be included in the baseline analyses, but, to assess reproducibility, stable patients are needed when the QOLHEQ is completed a second time, whereas, for responsiveness, patients are needed who improved or deteriorated relative to baseline. To identify these stable and changed patients, we recommend using a seven-point rating scale (Global Rating of Change [GRC] scale) at T₁ and T₂. The question ‘Overall, has there been any change in how your hand eczema bothers you since the last time you completed the QOLHEQ?’ should be answered with the following response categories: much improvement, moderate improvement, minor improvement, no change, minor deterioration, moderate deterioration, much deterioration.12

To be able to draw conclusions on the measurement properties of the QOLHEQ, it needs to be compared to other existing (preferably validated) instruments. This needs to be done on the basis of clear hypotheses about whether or not, and how strongly, the QOLHEQ correlates with these other instruments (see Hypothesis testing). Below, we describe the three domains that need to be taken into account, and recommended instruments to measure them.

1) Skin-specific HRQOL. We recommend using two instruments for this: the Dermatology Life Quality Index (DLQI), comprising 10 items scored on a four-point scale13_{; and the}

Skindex 17 (or 29), consisting of 17 (or 29) items on a three-point scale.14,15

2) Generic HRQOL. We recommend the EQ-5D, consisting of five items scored on a five-point scale, and a visual analog scale.16_{A possible alternative is the Short Form 36 (or}

12), consisting of 36 (or 12) items covering eight health concepts.17,18

3) Severity. A recommended example of an instrument with which to measure this is the Photoguide, developed by Coenraads et al., in which the hand eczema severity is scored by the treating physician on a five-point scale (clear, almost clear, moderate, severe, and very severe).19_{One could alternatively (or additionally) use the Hand}

Eczema Severity Index (HECSI), which is a continuous scale from 0-360 points that takes morphological signs and extent into account.20,21_{If no translated and validated}

instrument is available, the Photoguide is expected to be the most suitable instrument to use, because of its graphical nature.

Regardless of the chosen reference instrument, it should be described in detail, along with citations to publications reporting on its measurement properties, if available.

CODING OF THE QOLHEQ AND HANDLING OF MISSING ITEMS

The standard scoring structure of the QOLHEQ is 0 (never), 1 (rarely), 2 (sometimes), 3 (often), and 4 (always).2_{In the original German validation study, some items of the QOLHEQ had a}

deviating scoring structure. In this study, it was found for three out of the 30 items, the response categories ‘rarely’ and ‘sometimes’ were disordered (causing misfits in the item response theory [IRT] analysis of subscales), meaning either that patients are not able to differentiate between those categories for those items, or that it makes no difference in terms of HRQOL to choose ‘rarely’ or ‘sometimes’ for those items. This concerns items 4 (wearing gloves), 10 (anxious about the future) and 28 (dryness). All items, but specifically these three, need to be analyzed carefully for such problems in a national validation study by using IRT methods. If disordered thresholds

(9)

Table 2 Framework/checklist for the reporting of measurement properties of the QOLHEQ.

What to do? What to report?

General

Sample characteristics Adequately describe study population - Sample size

- Age, sex

- Baseline scores (mean ± SD) of all measurement instruments used Reference

instruments Adequately describe reference instruments - Description of the reference instruments used

- (Validated) measurement properties of the reference instruments with citations

Missing values Report on missing values for each individual

analysis of a measurement property

- Number of included patients per analysis, combined with the percentage of missing values and a description of how they were handled

Translation

Structured translation 6-step method by Beaton et al. Description of process and resulting

adaptations Cross-cultural

adaptation (step 5 in translation process)

- Cognitive interview pilot study - n = 15

- Content validity (comprehensibility, relevance, completeness and cultural difficulties / differences)

Description of process and resulting adaptations, if applicable

Validity – Face validity

Face validity Critically recheck wording of the items. If

“translation” was performed properly, face validity is considered adequately assessed.

- Refer to the original development process of the QOLHEQ

CFA, confirmatory factor analysis; CTT: classical test theory; ICC: intra-class correlation coefficient; IRT, item response theory; MIC: minimal important change; PSI: person separation index; HRQOL: health-related quality of life; SEM: standard error of measurement; SDC: smallest detectable change.

are found, the scoring of these items needs to be adapted accordingly. For example, in the case of disordered thresholds in response categories ‘rarely’ and ‘sometimes’, it needs to be adapted to 0-1-1-2-3 instead of the original scoring structure 0-1-2-3-4. To calculate QOLHEQ scores, SPSS and SAS-syntax can be downloaded at www.qolheq.dermis.net.

The method for handling missing items depends on the analysis performed. Missing items are not a problem in IRT analysis. On the basis of the trait score of a respondent on other items, IRT can make an estimation of the missing item. For structural equation modeling, missing items are more of an issue. For analysis of the QOLHEQ, it is recommended to treat them as ‘never’ (code 0) if there is only one missing value in the subscale. The cases that show more than one missing value per subscale need to be excluded from analysis. For the overall score, a maximum of three missing values is allowed. If there are more than three missing values, the case needs to be excluded from analyses. The handling of missing items should always be included in the validation report, and specified for each individual measurement property that is assessed.

ASSESSMENT OF MEASUREMENT PROPERTIES

Table 2 gives an overview of the variables that need to be assessed to adequately report on the measurement properties of the translated QOLHEQ in the new population. In the next section we will discuss these in more detail.

(10)

8

What to do? What to report?

Scale structure

Internal consistency - Calculate Cronbach’s α (CTT)

- Calculate Person separation index (IRT)

- Cronbach’s α for each subscale - PSI for each subscale Structural validity - Item response theory (IRT): subscale

analysis, Partial Credit Model fit

- Confirmatory Factor Analysis (CFA) using unweighted least squares method of estimation

- χ2_{-statistic over the item-trait interaction}

for each item and subscale (IRT) - Path diagram with correlations (CFA) - Goodness of fit indices (CFA)

Validity – Hypothesis testing

Hypothesis testing Confirm the a priori hypotheses regarding

correlations of single scores on reference instruments in minimally three areas: skin specific HRQOL, generic HRQOL and severity of hand eczema

- A priori stated hypotheses with direction and magnitude of hypothesized correlations

- Results

- Percentage of confirmed/rejected hypotheses

Reproducibility and Responsiveness

Measurement error QOLHEQ needs to be completed at baseline

(T₀) and shortly thereafter (T₁) in patients whose situation did not change

SEM_agreement

Reliability (test-retest) Same as for measurement error ICCagreement

Hypothesis testing QOLHEQ needs to be completed at baseline

(T0) and after 4-6 weeks (T2) in patients

whose situation changed.

Confirm the a priori hypotheses regarding correlations of change scores on reference instruments in minimally three areas: skin specific HRQOL, generic HRQOL and severity of hand eczema.

- A priori stated hypotheses with direction and magnitude of hypothesized correlations

- Results

- Percentage of confirmed/rejected hypotheses

Interpretability (not a measurement property)

Single scores Assess agreement between anchor questions

and the QOLHEQ scores (subscale scores and total score)

Cutoff values for bands indicating how hand eczema affects HRQOL, using weighted kappa

Change scores Calculate SDC from SEM_agreement.Assess

importance of change as perceived by the patient with anchor questions. Estimate MIC using the visual anchor-based minimal important change distribution method.

SDC and MIC of improvement and deterioration

Floor and ceiling effects

Check amount of patients achieving highest and lowest possible scores

Percentage of patients achieving the highest and lowest possible score CFA, confirmatory factor analysis; CTT: classical test theory; ICC: intra-class correlation coefficient; IRT, item response theory; MIC: minimal important change; PSI: person separation index; HRQOL: health-related quality of life; SEM: standard error of measurement; SDC: smallest detectable change.

validity – facevalidity (andcontentvalidity)

If the QOLHEQ is to be used in a new population, it is important to critically look at the questionnaire before testing, to assess whether it is an adequate reflection of HRQOL for the target population (content validity). This is already done in the fifth stage of the translation procedure during the cognitive interview pilot study by patients. As the QOLHEQ was developed by international experts and patients, the face validity is already at a high level. If the translation and cultural adaption of the items was performed in a proper manner, it can be assumed that face validity and content validity are adequately assessed.

(11)

scalestructure Structural validity

First, to determine structural validity in the context of a structural equation model, a confirmatory factor analysis needs to be carried out. This tests whether the gathered data fit the predefined model structure of the original QOLHEQ: a multi-domain model with four unidimensional subscales (factors) assessing the higher-order construct of HRQOL (Figure 2). In order to perform a more detailed inspection of the psychometric properties of the QOLHEQ at an item level, we further advise the performance of an analysis according to IRT for each subscale. Both approaches are described in more detail in Supplement 2.

Internal consistency

Cronbach’s alpha should be reported for each subscale to assess its internal consistency. Optimally, values should be between 0.70 and 0.95.1_{For the IRT model, a}

Person-Separation-Index (PSI) needs to be calculated for each subscale. These values can be interpreted like those for Cronbach’s alpha. RUMM2030 provides the PSI.22

validity – hypothesistesting

Hypotheses about the correlation between the QOLHEQ and related instruments need to be tested to confirm that the QOLHEQ specifically measures the construct of HRQOL in hand eczema, as opposed to related and more general instruments. This component of the validation concerns the validity of single scores, as contrasted with the validity of multiple measurements, which is covered in the paragraph on reproducibility and responsiveness. The correlation tests should be done by the use of Pearson’s correlation coefficient (r). Strong correlation (+++) is defined as r > 0.7; moderate correlation (++) as 0.7 > r > 0.4; and weak correlation (+) as 0.4 > r > 0.2. Validity is considered to be high if <25% of hypotheses are rejected; moderate if 25-50% are rejected and poor if >50% are rejected.1

The following hypotheses should minimally be tested:

1) The QOLHEQ has a strong (+++) correlation with the skin-specific HRQOL instrument(s) (e.g. DLQI and/or Skindex)

2) The QOLHEQ has a moderate (++) correlation with the generic HRQOL instrument(s) (e.g. EQ-5D and/or Short Form)

3) The QOLHEQ has a moderate (++) correlation with the severity-measuring instrument(s) used by the patient (e.g. Photoguide)

4) The QOLHEQ has a weak (+) correlation with the severity-measuring instrument(s) used by the physician (e.g. Photoguide / HECSI).

In addition to the above-mentioned hypotheses, further hypotheses could be added, possibly also based on additional (preferably validated) measurement instruments to acquire more evidence for validity. For instance, a hypothesis could be based on an instrument that specifically tests one of the subscales or that tests discriminative validity, which means whether the instrument is capable of differentiating between patients of known groups (such as patients with different disease severity or with hand eczema of different etiologies). In case of the latter, it is informative to specifically state a priori hypotheses (including magnitude of the expected difference) and to provide box plots, rather than only reporting mean, SD, and statistical significance.

reproducibilityandresponsiveness

(12)

8

which the measurement is free from measurement error) and reliability (the degree to which repeated measurements in stable persons provide similar answers). To test this in the QOLHEQ, it needs to be completed twice. The period in between has to be short because this usually yields the highest percentage of stable patients. Together with the retest, the GRC scale needs to be completed by the patient, to ensure only patients whose situation did not change compared to the first test are taken into account when assessing reproducibility.

Measurement error

This concerns the absolute measurement error. First, the standard error of measurement (SEM_agreement) should be calculated, preferably for the whole scale and for each subscale. For this, measurements of statistical variance (σ2_{) have to be assessed; more specific, the square root}

of the error variance (√ σ2

error) is needed. Included in this σ2error are: the systematic differences

between the time points on which the QOLHEQ was completed (σ2

t) and a residual variance

(random error variance, σ2

residual). These components of error variance are calculated by

performing an analysis of variance (ANOVA) using the restricted maximum likelihood model, in which the total QOLHEQ score is the dependent factor and time point the random factor.23

Reliability (test-retest)

Here, we test how the QOLHEQ performs in distinguishing an individual patient if it is completed twice while nothing in the situation of the patient has changed, despite measurement errors. This is done by calculating the intra-class correlation coefficient (ICC), again for the whole QOLHEQ and for each subscale. It is important to incorporate systematic differences between the measurements at the two time points in the measurement error. Because of this, ICC_agreement should be reported. For this we need the same error variances as for calculation of the SEM_agreement, but added with the variance due to systematic differences between the ‘true’ score of patients (σ2

p), which can be calculated with the same ANOVA as described above.23 An ICC

value > 0.70 is considered acceptable.1

Hypothesis testing

We want to know if the QOLHEQ is able to detect changes over time in patients that actually experienced a change in HRQOL (also known as validity of change scores or responsiveness). This is measured between the first (T₀) and third (T₂) time point, which should be about 4-6 weeks later to ensure a high percentage of changed patients. Again, the GRC scale needs to be scored by the patient; this time to ensure that only patients with actual change in their situation are included in this analysis. The assessment of responsiveness relies on hypothesis testing of change scores, similar to the hypothesis testing in single scores for validity. For this, we recommend to use correlations among the reference instruments, combined with at least the following a priori defined hypotheses:

1) The change score of the QOLHEQ (QOLHEQ T₀– QOLHEQ T₂) correlates better with the GRC than the change score of the skin specific instrument correlates with the GRC. 2) The change score of the QOLHEQ (QOLHEQ T₀– QOLHEQ T₂) correlates better with the

GRC than the change score of the generic instrument correlates with the GRC. 3) The change score of the QOLHEQ (QOLHEQ T₀– QOLHEQ T₂) correlates better with the

change score of HE severity (by physician) than the change score of the skin specific instrument correlates with the change score of HE severity (by physician).

4) The change score of the QOLHEQ (QOLHEQ T₀– QOLHEQ T₂) correlates better with the change score of HE severity (by physician) than the change score of the generic

(13)

instrument correlates with the change score of HE severity (by physician).

For each added instrument, the same cross-comparing hypotheses should be defined

a priori. Responsiveness is considered high if 25% of hypotheses are rejected; moderate if

25-50% are rejected and poor if more than 25-50% are rejected.

interpretability

Finally, we want to get an insight into what single scores or change scores on the QOLHEQ actually mean. For this, anchor questions are needed to function as a gold standard or ‘external criterion’.

Single scores

To interpret single scores, a patient needs to answer anchor questions at baseline (T₀), which should be correlated with the baseline QOLHEQ scores. Obviously, the best anchor question would ask the patient directly about their overall quality of life. However, many patients seem to have a hard time understanding this construct. Therefore, we recommend asking about how hand eczema affects patients. The overall anchor question might therefore be:

How did your hand eczema bother you in your overall health state in the past seven days? Not at all, slightly, moderately, strongly, very strongly.

For the subscales, specific examples of anchor questions can be found in an online supplement (Supplement 3).

The agreement between anchor questions and the QOLHEQ scores (overall score and subscale scores) should be assessed using a weighted kappa, yielding an interpretation for single scores, with bands indicating how hand eczema affects HRQOL.24

Change scores

To interpret change scores, two values are important: the smallest detectable change (SDC) and minimal important change (MIC). The SDC is a change beyond measurement error. To calculate this, the measurement error that was calculated for reproducibility in patients that did not change (SEM_agreement) needs to be used. The formula is: SDC = 1.96 * √2 * SEM_agreement.23

The MIC is the smallest change in QOLHEQ score which patients perceive as important. To assess this, we recommend using the visual anchor-based MIC distribution method, developed by De Vet et al, which is based on the Receiver Operating Characteristic (ROC) method.25_{For this, anchor questions should be asked at T}

2 regarding the perceived change

compared to baseline. For the overall QOLHEQ, this question is similar to the GRC scale. For the subscales, specific examples of anchor questions can be found in an online supplement (Supplement 3). If patients indicate change, a final question should be asked to ascertain whether the indicated change was actually important for the patient. This makes it easier to choose how the response categories will be clustered for the calculation of the MIC (e.g. importantly improved, not importantly changed, importantly deteriorated). The change scores can be plotted, and from this the MICs for improvement and deterioration (which might differ) can be obtained. The SDC should be smaller than the MIC to distinguish important changes from measurement error in individual patients.

Floor and ceiling effects

The amount of patients achieving the lowest or highest possible scores should be reported for the total score and for each subscale. If this is lower than 15%, a floor or ceiling effect is not expected.26

(14)

8

DISCUSSION

We present a guideline for translation and national validation of the QOLHEQ. If researchers follow this guideline to report on various measurement properties, this will likely result in homogenous publications that are readily interpretable. However, several matters should be taken into account.

Often, researchers are tempted to run validation studies alongside a running clinical trial from a time-saving point perspective. This should only be done (in accordance to these guideline), using a GRC scale and multiple (already validated) instruments as comparison for hypothesis testing (e.g. Skindex/DLQI). In studies in which the QOLHEQ by itself (without using any of the aforementioned instruments) is used as an outcome measure, as well as being validated at the same time, the effect of the intervention will intervene with the conclusions about the responsiveness of the QOLHEQ. It will be hard to disentangle whether the outcome tells something about the effect of the intervention or the quality of the QOLHEQ. Also, one should ideally only use already validated measurement instruments in clinical trials.

Once the QOLHEQ is nationally validated, studies in this population can be compared. However, comparison to other countries cannot be made right away. For comparable patients from both populations, responses to an item should be the same (also called measurement invariance). It is advised that after running a national validation study, an international cross-cultural validation is carried out, in order to proof that QOLHEQ scores are comparable also across countries. If differences per item are found (DIF), these should be dealt with, preferably in consultation with the original developers of the QOLHEQ. 27

Calculating minimally important change can be done in several ways. In this guideline, we recommend the visual anchor-based MIC distribution method. To increase interpretability of a measurement instrument, other methods like calculating mean change score or Minimal

Detectable Change (MDC) can be used to obtain a range of values for the MIC.28_However,

because this requires a large amount of calculations and associated values to report, we recommend reporting this in a separate publication.

This guideline is developed to serve as a kind of reference manual when performing a national translation and validation study of the QOLHEQ (and possibly also for other measurement instruments, mainly with a quality of life focus). Using this standardized framework contributes to a higher quality of validation and will generate the opportunity for a meaningful comparison between studies.

(15)

REFERENCES

1. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42.

2. Ofenloch RF, Weisshaar E, Dumke AK, Molin S, Diepgen TL, Apfelbacher C. The Quality of Life in Hand Eczema Questionnaire (QOLHEQ): validation of the German version of a new disease-specific measure of quality of life for patients with hand eczema. Br J Dermatol. 2014;171(2):304-312.

3. Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. 5th ed. New York: Oxford University Press; 2015.

4. de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine. 1st ed. New York: Cambridge University Press; 2011.

5. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life

Res. 2012;21(4):651-657.

6. Bullinger M, Alonso J, Apolone G, et al. Translating health status questionnaires and evaluating their quality: the IQOLA Project approach. International Quality of Life Assessment. J Clin Epidemiol. 1998;51(11):913-923. 7. Wild D, Grove A, Martin M, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104.

8. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186-3191.

9. Carbone ET, Campbell MK, Honess-Morreale L. Use of cognitive interview techniques in the development of nutrition surveys and interactive nutrition messages for low-income populations. J Am Diet Assoc. 2002;102(5):690-696.

10. Anthoine E, Moret L, Regnault A, Sebille V, Hardouin JB. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014;12:172-176.

11. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175-191.

12. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407-415.

13. Finlay AY, Khan GK. Dermatology Life Quality Index (DLQI)--a simple practical measure for routine clinical use.

Clin Exp Dermatol. 1994;19(3):210-216.

14. Nijsten TE, Sampogna F, Chren MM, Abeni DD. Testing and reducing skindex-29 using Rasch analysis: Skindex-17. J Invest Dermatol. 2006;126(6):1244-1250.

15. Chren MM, Lasek RJ, Flocke SA, Zyzanski SJ. Improved discriminative and evaluative capability of a refined version of Skindex, a quality-of-life instrument for patients with skin diseases. Arch Dermatol. 1997;133(11):1433-1440.

16. Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727-1736.

17. Hays RD, Sherbourne CD, Mazel RM. The RAND 36-Item Health Survey 1.0. Health Econ. 1993;2(3):217-227. 18. Ware Jr J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233.

19. Coenraads PJ, Van Der Walle H, Thestrup-Pedersen K, et al. Construction and validation of a photographic guide for assessing severity of chronic hand dermatitis. Br J Dermatol. 2005;152(2):296-301.

20. Held E, Skoet R, Johansen JD, Agner T. The hand eczema severity index (HECSI): a scoring system for clinical assessment of hand eczema. A study of inter- and intraobserver reliability. Br J Dermatol. 2005;152(2):302-307. 21. Weistenhofer W, Baumeister T, Drexler H, Kutting B. An overview of skin scores used for quantifying hand

(16)

8

eczema: a critical update according to the criteria of evidence-based medicine. Br J Dermatol. 2010;162(2):239-250.

22. Anonymous. Cronbach’s α and the Person Separation Index (PSI). Available at: http://www.rummlab.com.au/ rmrelidx2030.pdf.

23. de Vet HC, Terwee CB, Knol DL, Bouter LM. When to use agreement versus reliability measures. J Clin Epidemiol. 2006;59(10):1033-1039.

24. Charman CR, Venn AJ, Ravenscroft JC, Williams HC. Translating Patient-Oriented Eczema Measure (POEM) scores into clinical practice by suggesting severity strata derived using anchor-based methods. Br J Dermatol. 2013;169(6):1326-1332.

25. de Vet HC, Ostelo RW, Terwee CB, et al. Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach. Qual Life Res. 2007;16(1):131-142.

26. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293-307.

27. Ofenloch RF, Oosterhaven JAF, Susitaival P, et al. Cross-Cultural Validation of the Quality of Life in Hand Eczema Questionnaire (QOLHEQ). J Invest Dermatol. 2017;137(7):1454-1460.

28. van der Roer N, Ostelo RW, Bekkering GE, van Tulder MW, de Vet HC. Minimal clinically important change for pain intensity, functional status, and general health status in patients with nonspecific low back pain. Spine (Phila

(17)

SUPPLEMENT S1 – TRANSLATION REPORTING TEMPLATE FOR THE QUALITY OF LIFE IN HAND ECZEMA QUESTIONNAIRE (QOLHEQ)

QOLHEQ header page 1

English Please indicate how often you were bothered by the following situations

during the last seven days. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

Review by expert committee Pre final version

Cognitive interview pilot Post pilot final

QOLHEQ header page 2/3

English Please refer to the last 7 days and to the skin of your hands only!

T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item categories

English Never, rarely, sometimes, often, all the time

(18)

8

QOLHEQ repeating sentence

English I have been bothered by the skin condition of my hands…

QOLHEQ item 1

English (I have been bothered by the skin condition of my hands)

… being painful. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 2

… restricting/impairing me in my job. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(19)

QOLHEQ item 3

… restricting/impairing me in doing everyday home duties. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 4

… because I have to wear gloves. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 5

… making me feel frustrated. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(20)

8

QOLHEQ item 6

… itching. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 7

… because treatment is time consuming. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 8

… making me feel annoyed. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(21)

QOLHEQ item 9

… causing loss of sleep. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 10

… making me feel anxious about the future. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 11

… fissuring. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(22)

8

QOLHEQ item 12

… restricting/imparing me in my leisure time activities (e.g. sports, hobbies). T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 13

… because I have to use creams. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 14

… causing problems washing myself. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(23)

QOLHEQ item 15

… causing problems dressing myself. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 16

… making me feel I have to hide my hands. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 17

… because it leads to me avoiding contact with other people. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(24)

8

QOLHEQ item 18

… because I have to visit a physician. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 19

… making me feel sad / depressed. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 20

… because of redness. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(25)

QOLHEQ item 21

… making me feel irritated. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 22

… because I have to avoid contact with certain things. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 23

… bleeding. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(26)

8

QOLHEQ item 24

… because of worrying about side effects of treatment. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 25

… affecting my family life and friendships. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 26

… because of the treatment costs I have to cover myself. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(27)

QOLHEQ item 27

… making me feel embarrassed. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 28

… because of dryness. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

QOLHEQ item 29

… when touching my family or partner. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(28)

8

QOLHEQ item 30

… making me feel nervous. T1 T2 T-12 Optional: BT1 items Optional: BT2 items Comments on BT

(29)

SUPPLEMENT S2 – STATISTICAL ASSESSMENT OF SCALE STRUCTURE OF THE QUALITY OF LIFE IN HAND ECZEMA QUESTIONNAIRE (QOLHEQ)

STRUCTURAL VALIDITY (STRUCTURAL EQUATION MODEL)

To perform a confirmatory factor analysis (CFA), a parameter estimation method is needed for which the model meets the assumptions. QOLHEQ scores are not expected to be normally distributed, therefore it is recommended to use the unweighted least squares method, since this method is robust against violations of the assumption of normality. The correlations (factor weights or loadings) between items and factors, along with the error variance of the items, should be reported in a path diagram of the QOLHEQ model, similar to the one that can be found as online supplement to the protocol (Supplement 4). We recommend reporting several goodness-of-fit indices, combined with their respective thresholds of fit (good/acceptable/ non fitting) according to rules of thumb published by Shermelleh-Engel et al1_.

STRUCTURAL VALIDITY (ITEM RESPONSE THEORY MODEL)

To apply an IRT model to the QOLHEQ, the four unidimensional subscales (which should be confirmed in the CFA) need to be assessed individually, because IRT assumes unidimensionality. The IRT model-fit analysis should be done using a 2-parameter logistic model for polytomous response categories: the partial credit model. For this, we use the software RUMM2030 (RummLab Pty Ltd, Duncraig, WA, Australia). Before checking the IRT model-fit of the scales, the data needs to be assessed on a person level, to identify extreme outliers over the whole (sub)scale (compare a person’s observed and expected item score across test items: person-fit

residual). If the researcher assumes that outlying data of a respondent has probably occurred

by something other than chance, that respondent should be excluded for the IRT model-fit analysis.

Fit of the items and overall subscales to the IRT model need to be reported using the χ2_{-statistic over the item-trait interaction. A P-value < 0.01 for the overall subscales and a}

P-value < 0.007 or < 0.006 for the individual items indicate a poor fit to the model (Bonferroni corrected, dependent on number of items in the subscale)2_{. Should this be encountered, it}

is advised to investigate the data on item/response category level, to check if the misfit has occurred due to a misunderstanding of respondents or wrongly responding to certain items (item-fit residual). In parallel, the possibility of disordered thresholds (two overlapping response categories within an item) should be considered, checking item threshold order and/or item characteristic curves. If this is seen, these response categories should be given the same score and model-fit needs to be checked again.

Finally, it should be checked if respondents with equal HRQOL impairment, who differ in sex or age answer the same on every item. When this is not the case, the item shows Differential Item Functioning (DIF). If there is no plausible explanation for this, it is possible to adapt the model and calculate adjusted scores3_.

REFERENCES

1. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of psychological research online 2003;8(2):23-74.

2. Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995 Jan 21;310(6973):170. 3. Embretson SE, Reise SP. Item Response Theory for Phychologists. London: Lawrence Erlbaum; 2000.

(30)

8

SUPPLEMENT S3 – ASSESSMENT OF THE INTERPRETABILITY OF THE QUALITY OF LIFE IN HAND ECZEMA QUESTIONNAIRE (QOLHEQ) – SUGGESTED ANCHOR QUESTIONS

BASELINE (T₀) – INTERPRETABILITY OF SINGLE SCORES

generalanchorquestion:

How did your hand eczema bother you in your overall health state in the past seven days? • Not at all

• Slightly • Moderately • Strongly • Very strongly

symptomssubscaleanchorquestion:

How did the symptoms of your hand eczema (like pain, itch, fissuring, redness) bother you in the past seven days?

• Not at all • Slightly • Moderately • Strongly • Very strongly

emotionssubscaleanchorquestion:

How strong did your hand eczema affect your emotional well-being (e.g. making you angry, frustrated, or anxious about the future) in the past seven days?

treatmentsubscaleanchorquestion:

How did treatment and prevention of your hand eczema bother you in the past seven days? • Not at all

• Slightly • Moderately • Strongly • Very strongly

functioningsubscaleanchorquestion:

How strong did your hand eczema affect your functioning (e.g. performing your (home)work or doing hobbies) in the past seven days?

(31)

FOLLOW UP (T_{1 OR 2}) – INTERPRETABILITY OF CHANGE SCORES

generalanchorquestion (sameastheglobalratingofchangescale):

Overall, has there been any change in how your hand eczema bothers you since the last time you completed the QOLHEQ?

• Much improvement • Moderate improvement • Minor improvement

• No change  no more questions needed

• Minor deterioration • Moderate deterioration • Much deterioration

If you indicated a change (improvement or deterioration), was this change important for you? • No

• Yes

symptomssubscaleanchorquestion:

Has there been any change in how the symptoms of your hand eczema (like pain, itch, fissuring, redness) bother you since the last time you completed the QOLHEQ?

• Much improvement • Moderate improvement • Minor improvement • No change • Minor deterioration • Moderate deterioration • Much deterioration

• Yes

emotionssubscaleanchorquestion:

Has there been any change in how strongly your hand eczema affects your emotional well-being (e.g. making you angry, frustrated or anxious about the future) since the last time you completed the QOLHEQ?

(32)

8

treatmentsubscaleanchorquestion:

Has there been any change in how the treatment and prevention of your hand eczema bother you since the last time you completed the QOLHEQ?

• Yes

functioningsubscaleanchorquestion:

Has there been any change in how strong your hand eczema affects your functioning (e.g. performing your (home)work or doing hobbies) since the last time you completed the QOLHEQ? • Much improvement • Moderate improvement • Minor improvement • No change • Minor deterioration • Moderate deterioration • Much deterioration

(33)

SUPPLEMENT S4 – QUALITY OF LIFE IN HAND ECZEMA QUESTIONNAIRE (QOLHEQ) PATH DIAGRAM TEMPLATE FOR STRUCTURAL EQUATION MODELING

(34)

(35)