Cover Page The handle

(1)

Cover Page

The handle

http://hdl.handle.net/1887/74052

holds various files of this Leiden University

dissertation.

Author: Winder, J.Y.

(2)

The UHDRS and UHDRS-FAP assessments in

Huntington’s disease

(3)

The printing of this thesis was financially supported by Topaz, Vereniging van Huntington and Stichting Alkemade-Keuls

Design cover: Bregje Jaspers – www.proefschriftontwerp.nl Printed by: Gildeprint – www.gildeprint.nl

ISBN: 978-94-6323-594-5 © Jessica Winder

(4)

The printing of this thesis was financially supported by Topaz, Vereniging van Huntington and Stichting Alkemade-Keuls

Design cover: Bregje Jaspers – www.proefschriftontwerp.nl Printed by: Gildeprint – www.gildeprint.nl

ISBN: 978-94-6323-594-5 © Jessica Winder

The UHDRS and UHDRS-FAP assessments in

Huntington’s disease

Proefschrift

ter verkrijging van

de graad van Doctor aan de Universiteit Leiden, op gezag van Rector Magnificus prof. mr. C.J.J.M. Stolker,

volgens besluit van het College voor Promoties te verdedigen op donderdag 20 juni 2019

klokke 16:15 uur

door

Jessica Yndra Winder

geboren te Alkmaar

(5)

Promotoren: Prof. dr. R.A.C. Roos Prof. dr. W.P. Achterberg

Leden promotiecommissie: Prof. dr. T.P.M. Vliet Vlieland Prof. dr. A. Tibben

(6)

Promotoren: Prof. dr. R.A.C. Roos Prof. dr. W.P. Achterberg

Leden promotiecommissie: Prof. dr. T.P.M. Vliet Vlieland Prof. dr. A. Tibben

Prof. dr. J. Schols, Maastricht Universitair Medisch Centrum, Maastricht Dr. R. Reilmann, George-Huntington-Institute, Münster, Duitsland

CHAPTER 1

(9)

| Chapter 1 8

Huntington’s disease

Huntington’s disease (HD) is an autosomal dominant, progressive neurodegenerative disorder characterized by motor disturbances, cognitive decline, psychiatric symptoms, and functional disability. It is caused by a cytosine-adenine-guanine (CAG) trinucleotide

repeat expansion in the Huntingtin gene on chromosome 4.1_{A CAG expansion of 36}

repeats or more is associated with HD, although only a CAG repeat length of 40 or more is

considered fully penetrant and will cause HD inevitably.2_{The mean age of disease onset is}

30-50 years, with a mean disease duration of 17-20 years.3_{Age of onset is inversely}

correlated to CAG repeat length.4,5 Brain atrophy is most pronounced in the caudate

nucleus and putamen of the striatum.6 Atrophy is already detectable before disease onset

and progresses throughout the course of the disease.7

HD is a rare disorder with a prevalence of 5-10 per 100,000 in the Caucasian population.3

In the Netherlands, the total number of HD patients is approximately 1,700. About 6,000 to 9,000 people are at risk for developing HD. Genetic testing is available for individuals at risk. This test can identify premanifest gene carriers, who do not show symptoms or signs of HD yet, but will develop HD in the future. After disease onset, patients are referred to

as manifest. Severity of HD progresses from early stage HD to late stage HD over time.8

Clinical features

HD is clinically characterized by a triad of motor, cognitive, and behavioral symptoms. The most distinctive motor symptom of HD is chorea; involuntary, irregular movements of the face, trunk, and limbs. Other motor symptoms include bradykinesia, dystonia, impairment of oculomotor function, and gait/balance problems. As the disease progresses, chorea tend to decline, while dystonia, bradykinesia, dysarthria and dysphagia become more

prominent.3,9 Cognitive decline is another sign of HD. Typically, impairment in executive

functioning, memory, and psychomotor speed arise in HD.10_{The third main clinical feature}

of HD is behavioral change. Frequently reported psychiatric symptoms are depression,

anxiety, irritability, and apathy.11_{Depression and anxiety are common in the}

mild-to-moderate disease severity stages, while apathy is especially prevalent in more advanced

severity stages.12-15

(10)

9 Introduction |

Huntington’s disease

Huntington’s disease (HD) is an autosomal dominant, progressive neurodegenerative disorder characterized by motor disturbances, cognitive decline, psychiatric symptoms, and functional disability. It is caused by a cytosine-adenine-guanine (CAG) trinucleotide

repeat expansion in the Huntingtin gene on chromosome 4.1_{A CAG expansion of 36}

repeats or more is associated with HD, although only a CAG repeat length of 40 or more is

considered fully penetrant and will cause HD inevitably.2_{The mean age of disease onset is}

30-50 years, with a mean disease duration of 17-20 years.3_{Age of onset is inversely}

correlated to CAG repeat length.4,5 Brain atrophy is most pronounced in the caudate

nucleus and putamen of the striatum.6 Atrophy is already detectable before disease onset

and progresses throughout the course of the disease.7

HD is a rare disorder with a prevalence of 5-10 per 100,000 in the Caucasian population.3

In the Netherlands, the total number of HD patients is approximately 1,700. About 6,000 to 9,000 people are at risk for developing HD. Genetic testing is available for individuals at risk. This test can identify premanifest gene carriers, who do not show symptoms or signs of HD yet, but will develop HD in the future. After disease onset, patients are referred to

as manifest. Severity of HD progresses from early stage HD to late stage HD over time.8

Clinical features

HD is clinically characterized by a triad of motor, cognitive, and behavioral symptoms. The most distinctive motor symptom of HD is chorea; involuntary, irregular movements of the face, trunk, and limbs. Other motor symptoms include bradykinesia, dystonia, impairment of oculomotor function, and gait/balance problems. As the disease progresses, chorea tend to decline, while dystonia, bradykinesia, dysarthria and dysphagia become more

prominent.3,9 Cognitive decline is another sign of HD. Typically, impairment in executive

functioning, memory, and psychomotor speed arise in HD.10_{The third main clinical feature}

of HD is behavioral change. Frequently reported psychiatric symptoms are depression,

anxiety, irritability, and apathy.11_{Depression and anxiety are common in the}

mild-to-moderate disease severity stages, while apathy is especially prevalent in more advanced

severity stages.12-15

HD is presently incurable and it is not possible to delay either onset or progression of the disease. Due to the progressive nature of the disease, the clinical features ultimately lead to functional decline and loss of independency. As a result, it becomes more difficult for care to be provided at home, which may lead to nursing home admission.

Assessment scales

Multiple rating scales and instruments to detect and monitor the severity and progression of clinical features in HD have been developed over the years. Some assessment scales, such as the Unified Huntington’s Disease Rating Scale (UHDRS) and the Unified Huntington’s Disease Rating Scale-For Advanced Patients (UHDRS-FAP), have a categorical

and semi-quantitative design.16,17_{Other measurement instruments, such as eye-tracking}

equipment, tongue force analysis, and quantitative-motor (Q-Motor) assessments, have a

continuous, quantitative, and more objective design.18-20

The clinical assessment of symptoms and signs in HD is usually performed with the UHDRS,

developed by the Huntington Study Group (HSG).16_{The UHDRS is divided into four}

domains: motor performance, cognitive function, behavioral abnormalities, and functional abilities. The motor section assesses chorea, dystonia, eye movements, bradykinesia/rigidity, and gait/balance. Cognitive function is measured by the Verbal Fluency test, the Symbol Digit Modalities test, and the Stroop test (color naming, word

reading, and interference).21-23_{The behavioral domain comprises depression, anxiety,}

irritability/aggression, obsessive-compulsive behaviors, psychosis, and apathy. Functional ability is assessed by the Total Functional Capacity (TFC), the Functional Assessment Scale (FAS), and the Independence Scale (IS). The UHDRS has been developed to follow individual patients systematically over time, and monitor disease progression for both research purposes and for use in clinical practice. The motor, cognitive, and functional domains of the UHDRS have demonstrated to be sensitive to detect longitudinal changes

in manifest HD patients.16,24-28

In late stage HD, ceiling and floor effects of the UHDRS hamper the detection of changes and, therefore, disease progression is difficult to measure in patients with advanced HD.28,29_{For this reason, the UHDRS-FAP has been developed.}17_{This scale consists of four}

sections, which are a motor, cognitive, somatic, and behavioral section. The items of the domains are adjusted for more severely affected patients.

This thesis

(11)

| Chapter 1 10

symptoms over time. A teaching video has been developed by the European Huntington’s Disease Network (EHDN), in collaboration with the HSG, and an annual online certification

has been implemented for this purpose in 2009.30_In_{chapter 2, we aimed to investigate}

the interrater reliability of the UHDRS-TMS and of its subitems, using the ratings of the online certification. We also examined the impact of the annual certification on rater performance. In chapter 3, we focus on the oculomotor items of the UHDRS-TMS. The aim was to find out which of these items were affected in premanifest gene carriers compared to healthy controls and might be useful for detecting early clinical signs of HD.

In advanced stages of HD, knowledge is limited about the course of the clinical manifestations. Hardly any information on sensitive disease outcome measures to track disease progression is available and guidelines for management and care in long-term care

facilities are limited.31_{Therefore, we aimed to explore the properties of the UHDRS-FAP}

(12)

11 Introduction |

symptoms over time. A teaching video has been developed by the European Huntington’s Disease Network (EHDN), in collaboration with the HSG, and an annual online certification

has been implemented for this purpose in 2009.30_{In chapter 2, we aimed to investigate}

the interrater reliability of the UHDRS-TMS and of its subitems, using the ratings of the online certification. We also examined the impact of the annual certification on rater performance. In chapter 3, we focus on the oculomotor items of the UHDRS-TMS. The aim was to find out which of these items were affected in premanifest gene carriers compared to healthy controls and might be useful for detecting early clinical signs of HD.

In advanced stages of HD, knowledge is limited about the course of the clinical manifestations. Hardly any information on sensitive disease outcome measures to track disease progression is available and guidelines for management and care in long-term care

facilities are limited.31_{Therefore, we aimed to explore the properties of the UHDRS-FAP}

and UHDRS in patients with advanced HD residing in a nursing home or receiving day-care. We investigated the capacity of the scales to differentiate between patients in the later stages of the disease using our cross-sectional data (chapter 4). Internal consistency and interrater reliability of both scales are also reported. The rating scales were administered again after six months. With our longitudinal data, we examined if the UHDRS-FAP and UHDRS could detect disease progression in patients with late stage HD (chapter 5). In chapter 6, we aimed to identify predictors for institutionalization by examining differences between nursing home residents and day-care patients with HD. Identification of predictors may lead to interventions and treatment strategies that can postpone the need for nursing home admission. In the final chapter (chapter 7), we discuss our conclusions and make recommendations for future research.

References

1. Huntington's Disease Collaborative Research Group. A novel gene containing a trinucleotide that is expanded and unstable on Huntington’s disease chromosomes. Cell 1993; 72: 971–983.

2. Rubinsztein DC, Leggo J, Coles R, Almqvist E, Biancalana V, Cassiman J-J, et al. Phenotypic characterization of individuals with 30-40 CAG repeats in the Huntington disease (HD) gene reveals HD cases with 36 repeats and apparently normal elderly individuals with 36-39 repeats. Am J Hum Genet 1996; 59: 16–22.

3. Roos RAC. Huntington’s disease: a clinical review. Orphanet J Rare Dis 2010; 5: 40.

4. Duyao M, Ambrose C, Myers R, Novelletto A, Persichetti F, Frontali M, et al. Trinucleotide repeat length instability and age of onset in Huntington’s disease. Nat Genet 1993; 4: 387–392.

5. Stine OC, Pleasant N, Franz ML, Abbott MH, Folsteln SE, Ross CA. Correlation between the onset age of Huntington’s disease and length of the trinucleotide repeat in IT-15. Hum Mol Genet 1993; 2: 1547– 1549.

6. Vonsattel JP, Myers RH, Stevens TJ, Ferrante RJ, Bird ED, Richardson EP. Neuropathological classification of Huntington’s disease. J Neuropathol Exp Neurol 1985; 44:559–577.

7. Aylward EH, Sparks BF, Field KM, Yallapragada V, Shpritz BD, Rosenblatt A, et al. Onset and rate of striatal atrophy in preclinical Huntington disease. Neurology 2004; 63: 66–72.

8. Shoulson I, Fahn S. Huntington disease: clinical care and evaluation. Neurology 1979; 29: 1–3. 9. Novak MJU, Tabrizi SJ. Huntington’s disease. BMJ 2010; 340: c3109.

10. Dumas EM, Van den Bogaard SJA, Middelkoop HAM, Roos RAC. A review of cognition in Huntington’s disease. Front Biosci 2013; 5: 1–18.

11. Van Duijn E, Kingma EM, Van der Mast RC. Psychopathology in verified Huntington’s disease gene carriers. J Neuropsychiatry Clin Neurosci 2007; 19: 441–448.

12. Thompson JC, Harris J, Sollom AC, Stopford CL, Howard E, Snowden JS, et al. Longitudinal evaluation of neuropsychiatric symptoms in Huntington’s disease. J Neuropsychiatry Clin Neurosci 2012; 24: 53–60. 13. Paulsen JS, Nehl C, Ferneyhough Hoth K, Kanz JE, Benjamin M, Conybeare R, et al. Depression and

stages of Huntington’s disease. J Neuropsychiatry Clin Neurosci 2005; 17: 496–502.

14. Craufurd D, Thompson JC, Snowden JS. Behavioral changes in Huntington disease. Neuropsychiatry Neuropsychol Behav Neurol 2001; 14: 219–226.

15. Kingma EM, Van Duijn E, Timman R, Van der Mast RC, Roos RAC. Behavioural problems in Huntington’s disease using the Problem Behaviours Assessment. Gen Hosp Psychiatry 2008; 30: 155–161.

(13)

| Chapter 1 12

17. Youssov K, Dolbeau G, Maison P, Boissé M-F, Cleret de Langavant L, Roos RAC, et al. The Unified Huntington’s Disease Rating Scale For Advanced Patients: validation and follow-up study. Mov Disord 2013; 28: 1995–2001.

18. Blekher TM, Yee RD, Kirkwood SC, Hake AM, Stout JC, Weaver MR, et al. Oculomotor control in asymptomatic and recently diagnosed individuals with the genetic marker for Huntington’s disease. Vision Res 2004; 44: 2729–2736.

19. Reilmann R, Bohlen S, Klopstock T, Bender A, Weindl A, Saemann P, et al. Tongue force analysis assesses motor phenotype in premanifest and symptomatic Huntington’s disease. Mov Disord 2010; 25: 2195–2202.

20. Reilmann R, Bohlen S, Kirsten F, Ringelstein EB, Lange HW. Assessment of involuntary choreatic movements in Huntington’s disease – Toward objective and quantitative measures. Mov Disord 2011; 26: 2267–2273.

21. Benton AL, Hamsher K DeS. Multilingual aphasia examination manual. Iowa City: Univ Iowa, 1978. 22. Smith A. Symbol digit modalities test manual. Los Angeles: West Psychol Serv, 1973.

23. Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol 1935; 18: 643–662.

24. Siesling S, Van Vugt JPP, Zwinderman KAH, Kieburtz K, Roos RAC. Unified Huntington’s Disease Rating Scale: a follow up. Mov Disord 1998; 13: 915–919.

25. Meyer C, Landwehrmeyer B, Schwenke C, Doble A, Orth M, Ludolph AC. Rate of change in early Huntington’s disease: a clinicometric analysis. Mov Disord 2012; 27: 118–124.

26. Toh EA, MacAskill MR, Dalrymple-Alford JC, Myall DJ, Livingston L, Macleod SAD, et al. Comparison of cognitive and UHDRS measures in monitoring disease progression in Huntington’s disease: a 12-month longitudinal study. Transl Neurodegener 2014; 3: 15.

27. Feigin A, Kieburtz K, Bordwell K, Como P, Steinberg K, Sotack J, et al. Functional decline in Huntington’s disease. Mov Disord 1995; 10: 211–214.

28. Marder K, Zhao H, Myers RH, Cudkowicz M, Kayson E, Kieburtz K, et al. Rate of functional decline in Huntington’s disease. Neurology 2000; 54: 452–458.

29. Baake V, Reijntjes RHAM, Dumas EM, Thompson JC, Roos RAC. Cognitive decline in Huntington’s disease expansion gene carriers. Cortex 2017; 95: 51–62.

30. Reilmann R, Roos R, Rosser A, Grimbergen Y, Kraus P, Craufurd D, et al. A teaching film, video library and online certification for the Unified Huntington’s Disease Rating Scale Total Motor Score. Akt Neurol 2009; 36: 474.

(14)

13 Introduction |

18. Blekher TM, Yee RD, Kirkwood SC, Hake AM, Stout JC, Weaver MR, et al. Oculomotor control in asymptomatic and recently diagnosed individuals with the genetic marker for Huntington’s disease. Vision Res 2004; 44: 2729–2736.

19. Reilmann R, Bohlen S, Klopstock T, Bender A, Weindl A, Saemann P, et al. Tongue force analysis assesses motor phenotype in premanifest and symptomatic Huntington’s disease. Mov Disord 2010; 25: 2195–2202.

20. Reilmann R, Bohlen S, Kirsten F, Ringelstein EB, Lange HW. Assessment of involuntary choreatic movements in Huntington’s disease – Toward objective and quantitative measures. Mov Disord 2011; 26: 2267–2273.

21. Benton AL, Hamsher K DeS. Multilingual aphasia examination manual. Iowa City: Univ Iowa, 1978. 22. Smith A. Symbol digit modalities test manual. Los Angeles: West Psychol Serv, 1973.

23. Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol 1935; 18: 643–662.

29. Baake V, Reijntjes RHAM, Dumas EM, Thompson JC, Roos RAC. Cognitive decline in Huntington’s disease expansion gene carriers. Cortex 2017; 95: 51–62.

(15)

Jessica Y. Winder1_{, Raymund A.C. Roos}1_{, Jean-Marc Burgunder}2,3_{, Johan Marinus}1_, Ralf Reilmann4,5,6

1_{Department of Neurology, Leiden University Medical Center, Leiden, The Netherlands} 2_{Department of Neurology, University of Bern, Bern, Switzerland}

3_{Swiss Huntington Center, Gümligen, Switzerland} 4_{George Huntington Institute, Muenster, Germany}

5_{Department of Radiology, University of Muenster, Muenster, Germany}

6 _{Department of Neurodegenerative Diseases and Hertie-Institute for Clinical Brain Research, University of Tuebingen,}

Tuebingen, Germany

(16)

CHAPTER 2

Interrater reliability of the

(17)

| Chapter 2 16

Abstract

Background: The clinical assessment of motor symptoms in Huntington’s disease is usually performed with the Unified Huntington’s Disease Rating Scale-Total Motor Score (UHDRS-TMS). A high interrater reliability is desirable to monitor symptom progression. Therefore, a teaching video and a system for annual online certification has been developed and implemented.

Objectives: The aim of this study is to investigate the interrater reliability of the UHDRS-TMS and of its subitems, and to examine the performance of raters in consecutive years. Methods: Data from the online UHDRS-TMS certification were used. The interrater reliability was assessed for all first-time participants (n = 944) between 2009 and 2016. Intraclass correlation coefficients (ICC) were calculated for each year separately and the mean was taken as the total ICC.

Results: The UHDRS-TMS (ICC = 0.847), tandem walking (0.824), pronate/supinate hands left (0.713), and retropulsion pull test (0.706) showed good interrater reliability. Poor interrater reliability was found for maximal dystonia of the left and right upper extremity (0.187 and 0.322, respectively), maximal dystonia of the left and right lower extremity (0.200 and 0.256, respectively), and maximal dystonia of the trunk (0.389), tongue protrusion (0.266), and rigidity arms left (0.390). Raters performed significantly worse on follow-up certification compared to their first certification.

(18)

17 Interrater reliability of the UHDRS-TMS |

Abstract

Background: The clinical assessment of motor symptoms in Huntington’s disease is usually performed with the Unified Huntington’s Disease Rating Scale-Total Motor Score (UHDRS-TMS). A high interrater reliability is desirable to monitor symptom progression. Therefore, a teaching video and a system for annual online certification has been developed and implemented.

Objectives: The aim of this study is to investigate the interrater reliability of the UHDRS-TMS and of its subitems, and to examine the performance of raters in consecutive years. Methods: Data from the online UHDRS-TMS certification were used. The interrater reliability was assessed for all first-time participants (n = 944) between 2009 and 2016. Intraclass correlation coefficients (ICC) were calculated for each year separately and the mean was taken as the total ICC.

Results: The UHDRS-TMS (ICC = 0.847), tandem walking (0.824), pronate/supinate hands left (0.713), and retropulsion pull test (0.706) showed good interrater reliability. Poor interrater reliability was found for maximal dystonia of the left and right upper extremity (0.187 and 0.322, respectively), maximal dystonia of the left and right lower extremity (0.200 and 0.256, respectively), and maximal dystonia of the trunk (0.389), tongue protrusion (0.266), and rigidity arms left (0.390). Raters performed significantly worse on follow-up certification compared to their first certification.

Conclusions: Our results suggest that the rating of dystonia (absent, slight, mild, moderate, or marked) is subjective and difficult to interpret, especially on video. Therefore, changing the dystonia items of the UHDRS-TMS should be explored. We also recommend that raters should watch the UHDRS-TMS teaching video before each certification.

Introduction

Huntington’s disease (HD) is an autosomal dominant, progressive neurodegenerative disorder characterized by motor disturbances, cognitive decline, psychiatric symptoms

and functional disability.1_{It is caused by a cytosine-adenine-guanine (CAG) trinucleotide}

repeat expansion on chromosome 4 in the Huntingtin gene.2_{The mean age at disease}

onset is between 30 and 50 years.3

The clinical assessment of symptoms and signs in HD is usually performed with the Unified Huntington’s Disease Rating Scale (UHDRS), developed by the Huntington Study Group

(HSG).4_{The UHDRS assesses four domains: motor function, cognitive function, behavioral}

abnormalities, and functional capacity. It was developed to follow individual patients systematically over time and monitor disease progression for both research purposes and for use in clinical practice. The sensitivity of the scale to detect longitudinal changes has been demonstrated by observational studies in manifest HD patients. In particular, the Total Motor Score (TMS) and the Total Functional Capacity (TFC) have been shown to

deteriorate significantly during one-year follow-up.4-9

The UHDRS is widely used in therapeutic clinical trials in HD and the UHDRS-TMS often serves as primary endpoint to assess efficacy of interventions. Therefore, a high interrater reliability is desirable to determine the course of the motor symptoms over time. Available data suggest that the interrater reliability of the TMS is high, with an intraclass correlation

coefficient of 0.94.4

In order to establish a high interrater reliability of the UHDRS-TMS, the European Huntington’s Disease Network (EHDN), in collaboration with the HSG, developed a teaching video for the TMS and established an annual online system of

UHDRS-TMS certification.10,11_{Despite this effort, we hypothesize that several items of the TMS are}

highly subjective and difficult to score. We therefore aim to investigate the interrater reliability of the UHDRS-TMS and of its subitems in the large database of ratings performed for annual online certification. We also aim to examine the impact of annual certification on rater performance to assess if certification increases accuracy over time.

Methods

(19)

| Chapter 2 18

perform/severely affected; see Supplement 1 for all response options per item). Hence, the total range of the UHDRS-TMS is 0 to 124. Higher scores generally indicate more severe motor impairment. Data from the annual UHDRS-TMS online certification were used for this study. The procedure consists of assessing three videos of patients with HD and providing the UHDRS-TMS ratings. In order to pass certification, it is necessary to rate at least two of the three videos correctly, which means 24 out of the 31 items must be rated correctly, the range of acceptable answers being defined by a panel of EHDN and HSG experts. If a rater fails, he is given the chance to rate a new set of three videos, but if he fails again, he will not be certified to perform the UHDRS-TMS and will be referred for further expert training.

UHDRS-TMS certification started in 2009.10 The present study includes the first attempt of

each rater in each year between 2009 and 2016. In 2012, biannual certification was proposed and implemented, but was soon reversed, with annual certification resuming in 2014. Therefore, the raters who participated in 2012 did not have to perform the certification in 2013, and the raters who participated in 2013 scored the same three videos as the raters who participated in 2012.

Statistical analysis was performed using IBM Statistical Package for the Social Sciences (SPSS) version 23. The interrater reliability of the UHDRS-TMS and its subitems was assessed for all first attempts with the intraclass correlation coefficient (ICC). We used a two-way random model with absolute agreement. ICC values were calculated for each year separately and the mean of this ICC was taken as the total ICC. An ICC higher than 0.7

was defined as good; a value lower than 0.4 was considered poor.12,13_{The percentage of}

(20)

perform/severely affected; see Supplement 1 for all response options per item). Hence, the total range of the UHDRS-TMS is 0 to 124. Higher scores generally indicate more severe motor impairment. Data from the annual UHDRS-TMS online certification were used for this study. The procedure consists of assessing three videos of patients with HD and providing the UHDRS-TMS ratings. In order to pass certification, it is necessary to rate at least two of the three videos correctly, which means 24 out of the 31 items must be rated correctly, the range of acceptable answers being defined by a panel of EHDN and HSG experts. If a rater fails, he is given the chance to rate a new set of three videos, but if he fails again, he will not be certified to perform the UHDRS-TMS and will be referred for further expert training.

UHDRS-TMS certification started in 2009.10 The present study includes the first attempt of

each rater in each year between 2009 and 2016. In 2012, biannual certification was proposed and implemented, but was soon reversed, with annual certification resuming in 2014. Therefore, the raters who participated in 2012 did not have to perform the certification in 2013, and the raters who participated in 2013 scored the same three videos as the raters who participated in 2012.

Statistical analysis was performed using IBM Statistical Package for the Social Sciences (SPSS) version 23. The interrater reliability of the UHDRS-TMS and its subitems was assessed for all first attempts with the intraclass correlation coefficient (ICC). We used a two-way random model with absolute agreement. ICC values were calculated for each year separately and the mean of this ICC was taken as the total ICC. An ICC higher than 0.7

was defined as good; a value lower than 0.4 was considered poor.12,13_{The percentage of}

raters who scored the separate motor items of the UHDRS correctly was determined using all first efforts. This was compared to the range of acceptable answers defined by the panel of HD experts. Follow-up performance was based on the results of raters who had carried out at least one rating, one year after their first participation. Because videos in consecutive years might differ in difficulty, we also assessed the follow-up of raters who participated for the first time in 2010 and again in 2012, years in which the same three videos were used. The raters were not informed about the use of the same videos. The percentage of UHDRS motor items rated correctly was calculated per certification, and compared using a paired sample t-test. A p-value of <0.05 was considered statistically significant.

Results

The total number of examinations performed between 2009 and 2016 was 1,982 and the number of first participations was 944, therefore many raters participated multiple times. The majority of raters in this study were physicians with different levels of expertise; however, nurses and other health care professionals also took part. The interrater reliability of the UHDRS-TMS, the separate subitems, and the total dystonia and chorea scores is shown in Table 1. The mean UHDRS-TMS of all videos was 31.9 (range 0 to 82). ICC values were high for the TMS (0.847), tandem walking (0.824), pronate/supinate hands left (0.713), and retropulsion pull test (0.706). Low ICC values were found for maximal dystonia of the left and right upper extremity (LUE and RUE; 0.187 and 0.322, respectively), maximal dystonia of the left and right lower extremity (LLE and RLE; 0.200 and 0.256, respectively), maximal dystonia of the trunk (0.389), tongue protrusion (0.266), and rigidity arms left (0.390).

For every subitem of the UHDRS-TMS, we calculated the percentage of raters who rated the item within the accepted range at their first participation (Table 1). Often, the expert panel defined more than one answer as correct, and therefore the mean number of options defined as correct is also reported. Gait (95.8%), retropulsion pull test (95.2%), dysarthria (94.1%), finger taps left (93.0%), and maximal chorea of the trunk (90.5%) were the items that were most often rated correctly. Saccade velocity vertical (76.5%), saccade velocity horizontal (76.8%), maximal chorea LUE (76.8%), luria (77.4%), and maximal chorea RUE (78.5%) were more difficult to score.

Table 1. Interrater reliability for all UHDRS-TMS items at first participation (n = 944)

ICC with 95% CI Percentage

of raters who scored the item correctly Mean number of options defined as correct

1. Ocular pursuit horizontal 0.623 (0.509-0.737) 83.9% 2.0

2. Ocular pursuit vertical 0.662 (0.591-0.732) 84.7% 2.1

3. Saccade initiation horizontal 0.592 (0.462-0.721) 83.7% 2.1

4. Saccade initiation vertical 0.610 (0.413-0.807) 84.5% 2.0

5. Saccade velocity horizontal 0.591 (0.431-0.751) 76.8% 2.0

6. Saccade velocity vertical 0.564 (0.438-0.689) 76.5% 2.3

(21)

| Chapter 2 20

8. Tongue protrusion 0.266 (-0.031-0.562) 83.5% 1.2

9. Finger taps right 0.413 (0.114-0.711) 84.3% 1.7

10. Finger taps left 0.569 (0.310-0.828) 93.0% 1.9

11. Pronate/supinate hands right 0.598 (0.375-0.821) 88.8% 1.5

12. Pronate/supinate hands left 0.713 (0.547-0.879) 88.9% 1.7

13. Luria 0.478 (0.188-0.768) 77.4% 1.3

14. Rigidity arms right 0.598 (0.518-0.678) 89.4% 1.7

15. Rigidity arms left 0.390 (0.174-0.607) 89.2% 1.5

16. Bradykinesia body 0.588 (0.454-0.723) 87.4% 2.1

17. Maximal dystonia trunk 0.389 (0.277-0.500) 87.7% 1.9

18. Maximal dystonia RUE 0.322 (0.095-0.549) 86.7% 1.7

19. Maximal dystonia LUE 0.187 (0.058-0.316) 83.8% 1.7

20. Maximal dystonia RLE 0.256 (0.134-0.377) 83.8% 1.7

21. Maximal dystonia LLE 0.200 (0.061-0.338) 83.3% 1.7

22. Maximal chorea face 0.597 (0.516-0.677) 88.7% 2.1

23. Maximal chorea BOL 0.551 (0.415-0.687) 86.5% 2.2

24. Maximal chorea trunk 0.685 (0.541-0.829) 90.5% 2.1

25. Maximal chorea RUE 0.508 (0.359-0.657) 78.5% 1.9

26. Maximal chorea LUE 0.446 (0.213-0.679) 76.8% 1.9

27. Maximal chorea RLE 0.572 (0.447-0.696) 85.1% 2.1

28. Maximal chorea LLE 0.615 (0.499-0.732) 89.6% 2.1

29. Gait 0.642 (0.421-0.864) 95.8% 1.6

30. Tandem walking 0.824 (0.693-0.955) 88.1% 1.4

31. Retropulsion pull test 0.706 (0.524-0.888) 95.2% 1.4

UHDRS-TMS 0.847 (0.766-0.928) NA NA

Total dystonia score 0.369 (0.182-0.555) NA NA

Total chorea score 0.670 (0.516-0.825) NA NA

ICC values were calculated using a two-way random model with absolute agreement. ICC values were calculated for each year separately and the mean of this ICC was taken as the total ICC (with 95% CI).

(22)

8. Tongue protrusion 0.266 (-0.031-0.562) 83.5% 1.2

9. Finger taps right 0.413 (0.114-0.711) 84.3% 1.7

10. Finger taps left 0.569 (0.310-0.828) 93.0% 1.9

11. Pronate/supinate hands right 0.598 (0.375-0.821) 88.8% 1.5

12. Pronate/supinate hands left 0.713 (0.547-0.879) 88.9% 1.7

13. Luria 0.478 (0.188-0.768) 77.4% 1.3

14. Rigidity arms right 0.598 (0.518-0.678) 89.4% 1.7

15. Rigidity arms left 0.390 (0.174-0.607) 89.2% 1.5

16. Bradykinesia body 0.588 (0.454-0.723) 87.4% 2.1

17. Maximal dystonia trunk 0.389 (0.277-0.500) 87.7% 1.9

18. Maximal dystonia RUE 0.322 (0.095-0.549) 86.7% 1.7

19. Maximal dystonia LUE 0.187 (0.058-0.316) 83.8% 1.7

20. Maximal dystonia RLE 0.256 (0.134-0.377) 83.8% 1.7

21. Maximal dystonia LLE 0.200 (0.061-0.338) 83.3% 1.7

22. Maximal chorea face 0.597 (0.516-0.677) 88.7% 2.1

23. Maximal chorea BOL 0.551 (0.415-0.687) 86.5% 2.2

24. Maximal chorea trunk 0.685 (0.541-0.829) 90.5% 2.1

25. Maximal chorea RUE 0.508 (0.359-0.657) 78.5% 1.9

26. Maximal chorea LUE 0.446 (0.213-0.679) 76.8% 1.9

27. Maximal chorea RLE 0.572 (0.447-0.696) 85.1% 2.1

28. Maximal chorea LLE 0.615 (0.499-0.732) 89.6% 2.1

29. Gait 0.642 (0.421-0.864) 95.8% 1.6

30. Tandem walking 0.824 (0.693-0.955) 88.1% 1.4

31. Retropulsion pull test 0.706 (0.524-0.888) 95.2% 1.4

UHDRS-TMS 0.847 (0.766-0.928) NA NA

Total dystonia score 0.369 (0.182-0.555) NA NA

Total chorea score 0.670 (0.516-0.825) NA NA

ICC values were calculated using a two-way random model with absolute agreement. ICC values were calculated for each year separately and the mean of this ICC was taken as the total ICC (with 95% CI).

Abbreviations: BOL, buccal-oral-lingual; CI, confidence interval; ICC, intraclass correlation coefficient; LLE, left lower extremity; LUE, left upper extremity; NA, not applicable; RLE, right lower extremity; RUE, right upper extremity; UHDRS-TMS, Unified Huntington's Disease Rating Scale-Total Motor Score.

Table 2. Performance of raters on the UHDRS-TMS certification over time Percentage of UHDRS-TMS items scored correctly

p-value

First certification (n = 226) 86.6% (±6.3) 0.004

Second certification one year later (n = 226) 85.1% (±6.7)

First certification in 2010 (n = 81) 87.8% (±5.0) 0.045

Same certification in 2012 (n = 81) 86.4% (±6.3)

Data are mean (with standard deviation). p-values were calculated using paired sample t-tests. Abbreviations: UHDRS-TMS, Unified Huntington's Disease Rating Scale-Total Motor Score.

The performance of raters on the UHDRS-TMS certification over time is presented in Table 2. The number with one or more follow-up participations one year after their first rating was 226. At their first examination, the raters scored 86.6% of the motor subitems correctly. One year later, at their second certification, the score dropped significantly to 85.1% (p = 0.004). Eighty-one raters participated for the first time in 2010 and saw the same three videos in 2012. The percentage of items scored correctly dropped significantly from 87.8% in 2010 to 86.4% in 2012 (p = 0.045).

Discussion

This study in a large cohort of raters showed a good interrater reliability of the UHDRS-TMS. We found an intraclass correlation coefficient of 0.847, which is lower than the ICC

of 0.94, previously reported by the HSG.4_{An even higher ICC of 0.97 was found in patients}

with advanced HD.14 An important difference between the previously mentioned studies

(23)

| Chapter 2 22

When we investigated the subitems of the UHDRS-TMS, we found a good interrater reliability for tandem walking, pronate/supinate hands left, and retropulsion pull test. To our knowledge no other study investigated the interrater reliability of the subitems of the UHDRS-TMS. However, the HSG did provide the ICC for the total chorea score (i.e., 0.82),

and for the total dystonia score (i.e., 0.62).4_{These ICC values are also higher than the ICC}

values for the total chorea score and total dystonia score in our study (0.670 and 0.369, respectively). Especially, the subitems maximal dystonia LUE, maximal dystonia LLE, and maximal dystonia RLE showed poor interrater reliability in our analysis. We also found low interrater reliability for tongue protrusion. This is an unexpected result, because this item is characterized by clear cutoffs for each category, partly based on time, which is an objective measure. Perhaps the poor interrater reliability is caused by the difficulty of determining if the tongue is fully protruded or not.

Our study showed that all dystonia items of the UHDRS-TMS exhibit poor interrater reliability, suggesting that the rating of dystonia (absent, slight, mild, moderate, or marked) is rather subjective and very difficult to interpret, especially on video. Removing or changing (part of) the dystonia items or providing clearer response options should therefore be explored, especially because poor interrater reliability hampers monitoring HD progression in both individual patients and clinical trials. Siesling et al. performed a

factor analysis on the UHDRS-TMS.15 They found a Cronbach’s alpha of 0.966 for the

UHDRS-TMS. When they omitted all dystonia items, the value rose slightly, suggesting that the internal consistency of the UHDRS-TMS is still high without dystonia. However, the motor phenotype in HD is not homogenous and dystonia-predominant HD (in contrast to

chorea-predominant HD) is present in some patients.16_{Accordingly, the dystonia items}

cannot be removed from the UHDRS-TMS entirely, since this would hamper the content validity of the scale. It is worth pointing out that the chorea items have the same scoring (absent, slight, mild, moderate, or marked) as the dystonia items. Nonetheless, the interrater reliability of the chorea items was better, possibly due to the fact that chorea is more common than dystonia in HD, particularly in patients in the earlier stages of HD seen regularly in out-patient clinics. Raters may therefore have more experience with different levels of severity of chorea. The ICC value of 0.62 for the total dystonia score found by the HSG demonstrated moderate interrater reliability and also suggested that dystonia is

more difficult to score than chorea, as the ICC for the total chorea score was 0.82.4

(24)