Acoustic measures of prosody and their relation with lateralization of language areas in schizophrenia.

(1)

January, 2021

Faculty of Science

Acoustic measures of prosody and their

relation with lateralization of language

areas in schizophrenia

Jana Bersee

Supervisors: Alban Voppel, msc & Dr. Bob Oranje

(2)

Abstract January, 2021

Abstract

Introduction: Alterations in brain areas and speech production of patients with schizophrenia have both been observed as differing from nonpsychiatric controls. However, their relation has not been investigated. The relation between language and the brain provides fundamental knowledge that can be useful for the improvement of diagnosing and treating schizophrenia. Here, we focus on aspects of language that are common in negative symptoms.

Method: Pitch variability was extracted from a semi-structured recorded interview. The volumetric lateralization of language areas in the brain was calculated using a structural MRI scan.

Results: We found a significant correlation between pitch variability and laterality index of the an-gular gyrus in the patient group. However, this correlation was not present in the control group and laterality index and pitch variability were not significantly different between groups.

Discussion: It is possible that limitations such as low symptom severity and small sample size have influenced the. Future research should use a larger sample size and take into account the symptom severity or a more specific diagnosis.

Keywords: schizophrenia spectrum disorder, pitch variability, lateralization, language areas, angular gyrus, structural MRI

(3)

January, 2021

1 Introduction 2 2 Method 4 2.1 Participants . . . 4 2.2 Speech acquisition . . . 4 2.3 Brain imaging . . . 4 2.4 Lateralization . . . 5 2.5 Data analysis . . . 5 3 Results 7 3.1 Demographic characteristics . . . 7 3.2 Laterality index . . . 7 3.3 Pitch variability . . . 8 3.4 Correlational analysis . . . 9

3.5 Total intercranial brain volume . . . 10

4 Discussion 12 5 REFERENCES 15 6 Appendix 18 6.1 Appendix A Interview . . . 19

(4)

Introduction January, 2021

Chapter 1

Introduction

Schizophrenia is a serious mental disorder characterized by the development of both positive and negative symptoms. Disorganised speech is, like delusions, hallucinations, and catatonic behaviour, one of the positive symptoms of schizophrenia (American Psychiatric Association, 2013). However, other parts of speech are part of the negative symptoms, i.e., longer pauses, reduced complexity of sentences, impaired semantics, difficulties with lexical access, impaired pragmatics and differences in prosody (Covington et al., 2005). A meta-analysis of Parola et al. (2020) reported that pitch variabil-ity, proportion of spoken time, speech rate and especially pause duration are most often significantly different compared with nonpsychiatric controls. This makes speech production an interesting aspect of schizophrenia. Among other scientists, De Boer et al. (2020) reported that certain aspects of speech production can be a useful tool for diagnosing schizophrenia. Interestingly, there are differences in the grey matter volume of the brains of schizophrenic people that also include the language areas (Torres et al., 2016) and in the lateralization of the language areas (Van Veelen et al., 2011). Although, findings of language alterations and brain abnormalities were often replicated separately, it would be useful as well to look at their relation. This could provide fundamental knowledge that can be used for better diagnostic methods and the development of better treatments.

Improving the treatment for people with schizophrenia is important because the disorder has a large impact on the quality of life. Schizophrenia is one of the disorders in which the most loss of health appears when compared to other disorders (Salomon et al., 2012). One of the reasons why the disorder is so impactful, is the reduction of life expectancy, since this is about 20 years shorter than average (Laursen et al., 2014). Even though the lifetime prevalence of schizophrenia is 4 per 1000 (Saha et al., 2005), which is low in comparison with other (mental) disorders like depression, the impact of schizophrenia on the quality of life and thus on society is large. Better diagnosis is especially impor-tant, because the outcomes for the patient are better when the disorder is treated earlier (Covington et al., 2005).

The current treatments of schizophrenia often consist of antipsychotics and reduce the positive symp-toms, thereby increasing the quality of life (Gr¨under et al., 2016). Patients that get antipsychotic drugs also relapse less often (Leucht et al., 2012). Even though the medication reduces the symptoms and improves the state of the patients both subjectively and objectively, this improvement is not enough to bring the quality of life back to normal (Patel et al., 2014). Moreover, the current treatments have a reduced impact on negative symptoms while these symptoms, i.e. flat affect, reduced self-care, reduced social interaction and alterations in language production (Azorin et al., 2014) have a large impact on the quality of life (Tahir et al., 2019). Therefore, it is relevant to have more fundamental knowledge about the negative symptoms.

(5)

Introduction January, 2021

Some research has been done in order to find the biological substrate of these language anomalies. Studies already reported that there are alterations in the language areas in schizophrenics, for example, there are reductions of grey matter volume in the areas involved in auditory information processing (Karlsgodt et al., 2010), and alterations in the medial temporal region which is involved in analysing and producing language (Pantelis, 2003). Besides region-specific alterations, the lateralization of the language areas is altered, since language areas are reduced left lateralized in patients with schizophre-nia compared with healthy controls (Sheng et al. 2013). This is related to the higher prevalence of left-handedness in the population of schizophrenics, which also influences the lateralization of brain areas (LeMay, 1976). However, when left-handed participants were excluded, reduced left lateraliza-tion of the language areas was still present (Van Veelen et al., 2011). Reduced left lateralizalateraliza-tion is already present in the first psychotic episode when medication could not have influenced the brain yet (Sheng et al., 2013; Van Veelen et al., 2011). This may indicate that the altered lateralization is one of the biological underpinnings of schizophrenia.

Following this line of research, the processing of prosody in schizophrenics in combination with the lateralization of language areas was studied (Bach et al., 2009). Prosody is an aspect of language which consists of the rhythm, stress, and intonation (Covington et al., 2005). Bach et al. (2009) found a reduced left-lateralization of the language areas in patients with schizophrenia. However, this reduction did not predict the processing of emotional prosody. It is important to stress the differ-ence between processing and producing prosody, since processing happens when someone else speaks. Moreover, according to a recent meta-analysis, the production of speech is most consistently abnormal (Cohen et al., 2014). Therefore, we expect a correlation between the production of prosody instead of the processing of prosody and the lateralization of language areas.

Intonation is an aspect of prosody that is easily measurable. Intonation itself is dependent on the individual and on gender (Pernet & Belin, 2012), therefore, it is more useful to use the standard deviation of intonation, also known as pitch variability. The closest variable for this measure is the standard deviation of the intonation. Previous research found more differences in language produc-tion, between patients and nonpsychiatric controls, in free speech than in a monologue (Parola et al., 2020). Therefore, a semi-structured interview is used to measure the intonation in free speech. Therefore, this study will examine the lateralization of various brain volumes related to language in combination with the production of prosody. First of all, we expect that schizophrenics have significantly less pitch variability during the interview compared to nonpsychiatric controls. Secondly, that the language areas in the brain are reduced left lateralized than in healthy controls. Finally, we expect reduced left lateralization to correlate with reduced variation in pitch. To assess this, pitch variability is measured in recorded interviews and MRI scans are made of patients with schizophrenia and nonpsychiatric controls. This will help to assess the relation between language areas in the brain and speech disturbances. This fundamental knowledge can help improve diagnosis and treatment of schizophrenia.

(6)

Method January, 2021

Chapter 2

Method

2.1 Participants

A total of 57 subjects from a study of Begemann et al. (2015) about using simvastatin as supplementary medication participated in this study. 26 participants with a psychotic disorder and 31 healthy controls were interviewed (table 2.1). The participants were matched for age, handedness, gender, and the years of education (YoE) of their parents. Symptom severity was measured using the positive and negative syndrome scale (PANSS) for schizophrenia (Kay et al., 1987). The inclusion criteria for the simvastatin study were; an onset of the psychosis less than three years ago and either a diagnosis of schizophrenia, schizoaffective disorder, schizophreniform disorder, or psychotic disorder not otherwise specified for the patients. The other in- and exclusion criteria of the simvastatin study were of importance for MRI scanning; for extensive information see the study protocol (Begemann et al., 2015). For the recorded interviews, participants also had to be native Dutch speakers, be older than eightteen and possess no uncorrected hearing or speech disorders. Before participation, all participants gave informed consent.

2.2 Speech acquisition

An interview was used to measure intonation in free speech. The interview (appendix A) consisted of questions unrelated to the disorder that gave insights in the daily lives of people. The interview took about 15 minutes per participant and was recorded with a digital TASCAM DR-40 recorder using separate microphones at about 3cm distance from the mouth for both the participant and the interviewer. The interviewer asked initial open-ended questions and interrupted the participants as little as possible. The participants were allowed to skip a question if they did not feel comfortable in answering. After extracting speech by the participants the interview was analysed using PRAAT (Boersma & Weenink, 2013), the Praat Script of Quen´e, Persoon and de Jong (2011) and openSMILE (Eyben et al., 2010) using the eGemaps package (Eyben et al., 2016). This software calculates the fundamental frequency (F0) which is the same as the intonation. For this study, the standard deviation of the F0 was used as a quantitative measurement of the pitch variability.

2.3 Brain imaging

Structural brain images were made using magnetic resonance imaging (MRI). The volume and the lateralization of language areas were calculated using these images. T1-weighted anatomical scans were obtained with the following parameters: 200 slices, slice thickness: 0.8 mm, no gap, repetition time (TR): 30ms, echo time (TE): 4.6ms, field of view: 240x240x160 mm3, voxel size: 0.79x0.79x0.8 mm3. For analysing the regions of interest, the a2009s map (Destrieux et al., 2010) was used to

(7)

Table 2.1: Demographic features of the participants (SD = standard deviation) a The dose of the 21 patients’ antipsychotics was calculated to the equivalent of chlorpromazine. The medication of the patients consisted of aripiprazole, clozapine, haloperidol, olanzapine , paliperidone, quetiapine. There was one patient medicated with amisulpride which cannot be calculated to the equivalent of chlorpromazine.

locate the regions in the structural MRI scans. According to previous research in language, several temporal and parietal regions are brain regions related to language. The regions of interest (ROI) for this study are the inferior frontal gyrus (Broca’s area), superior temporal gyrus (Wernicke’s area), middle temporal gyrus, supramarginal gyrus and angular gyrus (Bach et al., 2009; Rutten et al., 2002; Van Veelen et al., 2011). The total intercranial brain volume was calculated to use as a covariate in statistical analysis.

2.4 Lateralization

The Laterality Index (LI) is calculated using the following formula:

LI = (ROIlef t− ROIright)/(ROIlef t+ ROIright)

LI = (ROIleft – ROIright/ (ROIleft + ROIright) Whereas ROIleft is the volume of a certain brain area in the left hemisphere, ROIright is the volume of the corresponding area in the right hemisphere. This formula will provide a LI between -1 and 1, where -1 means the brain volume is more right lateralized and 1 means a more left lateralized brain area. An outcome of zero would mean the brain area has as much volume in the left hemisphere as in the right.

2.5 Data analysis

Data analyses were performed in RStudio (Rstudio Team, 2015) using the ‘car’ package (Fox & Weisberg, 2011). An analysis of covariance (ANCOVA) was used to analyse the lateralization data,

(8)

with total estimated brain volume as a covariate. Both the intonation and the lateralization are continuous dependent variables. In both the analyses the group, with or without psychotic disorder, was the categorical predictor. Firstly, the difference in lateralization between people with and without a psychotic disorder was analysed. Next, group differences in the standard deviation of the intonation were analysed using a T-test. When the assumptions of the T-test were violated a Mann-Whitney test was used instead. Finally, the relation between volumetric lateralization of the language areas and pitch variability, and the relation of the LI and the pitch variability with the PANSS score was analysed. A p-value below 0.05 was used as significance threshold in all analyses.

(9)

Results January, 2021

Chapter 3

Results

3.1 Demographic characteristics

In total 57 subjects participated in the study. There were no significant differences regarding age, gender, handedness or YoE of the parents between the patient group (n = 26) and the control group (n = 31). The total PANSS score of the patients was 52.3 ± 11.5, the positive PANSS score was 10.9 ± 3.9, the negative PANSS score 14.2 ± 5.0 and the general PANSS score 27.2 ± 6.3 (table 2.1).

3.2 Laterality index

Most of the regions of interest were left lateralized in both groups, except for the angular gyrus and the middle temporal gyrus, which were right lateralized in both groups (figure 3.1). The mean lateralization of the patient group (mean = 0.027; SD = 0.0298) and the control group (mean = 0.0219; SD = 0.0328) were not significantly different (F(1,53) = 0.193, p = 0.662) when tested with the ANCOVA and total intercranial volume as a covariate. In addition, no significant differences were found between groups when the ROI were analysed individually using the ANCOVA (table 3.1). Furthermore, the correlation between the LI and the PANSS scores was analysed. This led to a significant correlation between the PANSS general and the LI of the inferior frontal cortex (r = 0.43, p = 0.028) (table 3.2). All results of the correlational analysis between the PANSS and the LI can be seen in Appendix B. When left-handed persons were excluded from the analyses, because of the difference in the lateralization between right handed and left handed persons, an ANCOVA did not lead to significant differences between the patients and controls. Therefore, both right- and left-handed persons were included in all subsequent analyses.

(10)

Figure 3.1: Laterality index of ROI for patients and controls with the mean LI per ROI and group. A number closer to 1 means more left lateralized and a number closer to -1 more right lateralized.

Table 3.1: ANCOVA results of lateralization per brain area with total intercranial brain volume as covariate, LI = laterality index, SD = standard deviation

3.3 Pitch variability

A Wilcoxon signed-rank test was used to compare the standard deviation of the intonation between the patients and controls, because of violation of the assumption of normality in the control group.

(11)

Table 3.2: Results Pearson’s correlation analysis between inferior frontal gyrus LI and PANSS scores, Df = degrees of freedom, r = Pearson’s correlation coefficient, * means the result is significant.

The test showed there was no significant difference between the pitch variability of the control group (mean = 0.193, SD = 0.039) and of the patient group (mean = 0.198, SD = 0.042; W = 376, p = 0.674) (figure 3.2). Besides that, there were no significant correlations found between the pitch variability and the PANSS scores (Appendix B).

Figure 3.2: Pitch variability of the patient and control group between which there was no significant difference. SD of F0 semitone is the same as pitch variability.

3.4 Correlational analysis

In the correlational analysis between lateralization and pitch variability, only the LI of the angular gyrus showed a significant negative correlation with pitch variability in the patient group (r = -0.456, p = 0.0192), but not in the control group (r = -0.215, p = 0.2446) (figure 3.3 and 3.4). For the other ROI’s there was no significant correlation between the LI and pitch variability in both groups. For full results see appendix B.

(12)

Figure 3.3: Relation between the laterality index of the angular gyrus and the standard deviation of the intonation in the patient group (a) and the control group (b) with Pearson’s correlation coefficient.

3.5 Total intercranial brain volume

The total brain volume (in dm3) between the control group (mean = 1.59, SD = 1.74) and the patient group (mean = 1.53, SD = 1.60) was not significantly different (t(54.517) = -1.455, p = 0.1513). However, a significant negative correlation for the control group (r = -0.412, p ¡ 0.05) was found, when the total brain volume was tested for a correlation with the intonation. The negative correlation for the patient group was not significant (r = -0.0559, p = 0.786). In the control group, a few outliers were present, these were not removed because of the small sample size.

(13)

Figure 3.4: Relation between pitch variability and the total intercranial brain volume for the control and the patient group.

(14)

Discussion January, 2021

Chapter 4

Discussion

In this study, we measured the volumetric lateralization of language areas using structural MRI and the pitch variability using a semi-structured interview. The lateralization and the pitch variability were compared between patients with schizophrenia and nonpsychiatric controls and the relation be-tween lateralization and pitch variability was studied within groups. No significant differences were found in the lateralization of the language areas or in the pitch variability when comparing groups. Nonetheless, there was a significant correlation between the pitch variability and the lateralization of the angular gyrus in the patient group. Evidence is provided for a relation between the lateralization of the angular gyrus and intonation in patients, but it cannot be concluded that there is a significant difference in lateralization of language areas or pitch variability between schizophrenics and healthy controls.

Contrary to our hypothesis, we did not find significant differences in pitch variability between patients with schizophrenia and healthy controls. We analysed free speech, which should lead to more differ-ences in speech according to Parole et al. (2020). Therefore, it was surprising to not find a significant difference in pitch variability. The same is true for the lateralization of language areas. In previous research it was found that language areas are reduced left lateralized in patients compared to controls (Van Veelen et al., 2011), but this finding was not replicated in this study. We did find a significant relation between lateralization and pitch variability, but only in one brain area, namely the angular gyrus, and only in the patient group. The angular gyrus was most likely to be related with pitch variability, since prosody is one of the language aspects that is more right lateralized (Lindell, 2006; Mitchell & Crow, 2005) and the angular gyrus was almost significantly more right lateralized. This has also been reported in previous research (Niznikiewicz et al., 2000). However, we did not expect to find a correlation after the absence of significant findings in pitch variability and language lateraliza-tion and are unable to explain why the correlalateraliza-tion is only present in the patient group. Possibly, this could be caused by confounding factors, like antipsychotics or symptom severity that were not correct for. Further research is needed, for example using functional MRI and Diffusion Tensor Imaging, so functional lateralization and the white matter connectivity can also be studied. Finally, we found a significant correlation with total intercranial brain volume and pitch variability in the control group and not in the patient group. However, this could be due to outliers in the control group.

It is important to note that this study has some limitations. First of all, the symptom severity of our participants is low compared with other studies. For example the average PANSS total of this study was 52 while in other studies the average was 71 (Van Veelen et al., 2011) and 93 (Sheng et al., 2013). The absence of a difference in the lateralization of language areas could be caused by these population differences in the patient sample. Sheng et al. (2013) additionally found a correlation with the symptom severity and the symmetry quotient. However, van Veelen et al. (2011) did not find

(15)

any significant correlations between the symptom severity and brain areas. In this study, a significant positive correlation was found between the PANSS general and LI of the inferior frontal gyrus (table 3.2). Future research should investigate the relation between PANSS scores and LI to get clearer insights between the relation of symptom severity and brain areas.

The relatively low symptomatology in our subjects can also be the reason why there was no significant difference in pitch variability between patients and controls in our sample. However, in a meta-analysis of Parola et al. (2020) there were no significant correlations found between pitch variability and PANSS. This lack of correlation gives reason to question the previous interpretation. On the other hand, studies using the Scale for the Assessment of Negative Symptoms (SANS) as a measure for symptom severity did find a correlation. When we studied the relation between pitch variability and symptom severity measured with the PANSS, this did not lead to significant results (Appendix B). Future studies should use the SANS when looking at the relation between negative symptoms and language production.

Secondly, schizophrenics are more often left-handed, and handedness and lateralization are related (LeMay, 1976), so handedness may have influenced the lateralization results. However, few left-handed persons participated and our results were not significantly influenced when left handed persons were excluded. Additionally, in previous research the language areas of schizophrenics were also a more right lateralized when left-handed persons were excluded, so it is not likely that the handedness has caused the absence of reduced left lateralized language areas in our patient group. Some of the ROI in both groups were almost significantly more right lateralized, while language areas are thought to be more left lateralized. This is also found in previous studies who concluded that language areas are reduced left lateralized than is often thought (Lindell, 2006; Scheppele et al., 2019). Our findings support that interpretation.

Furthermore, it is possible that the use of medication, and illness duration influence pitch variability and brain volume as measured with structural MRI. So, it is interesting to see what the differences between patients and control groups are before treatment and when the illness has just started. There-fore, this participant group, with an onset of psychosis of less than three years ago, is especially useful, since this is a shorter illness duration than most other studies. However, most of our participants were still using antipsychotics during the study and short-term effects of the medication could still have influenced the results. A study with a mean illness duration of around 4 and 5 years found significant influences of antipsychotics on language production (de Boer et al., 2020), suggesting it is likely that the medication had some effect on the pitch variability. Other studies found that antipsychotics reduce brain volume (Ho et al., 2011). However this does not mean that the lateralization is also effected. Consequently, future research should use a medication na¨ıve participant group.

Another limitation of this study is the sample size . Although, we do not have a small sample size of participants, a bigger sample size would have improved certain aspects of the analysis. Some results that were almost significant, i.e., the reduced left lateralization of the angular gyrus, could have been significant with more participants. Besides that, the outliers in the control group would have had less effect on the results or could have been removed if more subjects participated. There were also not enough participants to analyse males and females separately. This would be interesting due to previous research which has reported differences between males and females in, symptom severity, brain morphology and lateralization, although these findings are inconsistent (Abel et al., 2010; Kim et al., 2017; Shaywitz et al., 1995). Future research should try to recruit more participants so the previous mentioned limitations could be prevented.

(16)

Moreover, this study focusses on the schizophrenia spectrum disorder and in the spectrum there is a heterogeneity in the symptoms and outcome of the participants (Voineskos et al., 2020). This makes it more difficult to find the biological substrate and could have led to the absence of significant results in the lateralization and intonation. Future research should make subgroups based on symptoms or subgroups in schizophrenia. Also a subgroup with and without flat affect would be useful to include in future research, since it is reported that especially in schizophrenic people with flat affect the pitch variability is altered (Alpert et al., 2000).

It is known that emotional state influences the intonation (Grandjean et al., 2006; Sobin & Alpert, 1999), so it could be useful if future research investigates the emotional state as confounding factor. Depending on the results, it might be necessary to control for emotional state or use different parts of produced language that are less dependent on emotions. On the other hand, the influence of emotions on language is also part of what we want to measure, since there is a difference in long term emo-tions between patients with schizophrenia and nonpsychiatric controls, including depressive symptoms. In conclusion, this study was unable to replicate the reduced left lateralization and reduced pitch variability in schizophrenics compared with nonpsychiatric controls as was reported in previous re-search. We did find a relation between the lateralization of the angular gyrus and the pitch variability in the patient group. Yet, it is unclear why this relation was not present in the control group and why it was present in the patient group. It is still likely that lateralization and pitch variability are altered in schizophrenia and might be related to symptom severity. These findings might be helpful in understanding how language areas and speech disturbances are related in schizophrenia.

(17)

REFERENCES January, 2021

Chapter 5

REFERENCES

Abel, K. M., Drake, R., & Goldstein, J. M. (2010). Sex differences in schizophrenia. In International Review of Psychiatry. https://doi.org/10.3109/09540261.2010.515205

Alpert, M., Rosenberg, S. D., Pouget, E. R., & Shaw, R. J. (2000). Prosody and lexical accuracy in flat affect schizophrenia. Psychiatry Research. https://doi.org/10.1016/S0165-1781(00)00231-6 American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author.

Azorin, J. M., Belzeaux, R., & Adida, M. (2014). Negative symptoms in schizophrenia: Where we have been and where we are heading. CNS Neuroscience and Therapeutics. https://doi.org/10.1111/cns.12292 Bach, D. R., Herdener, M., Grandjean, D., Sander, D., Seifritz, E., & Strik, W. K. (2009). Altered lateralisation of emotional prosody processing in schizophrenia. Schizophrenia Research, 110(1–3), 180–187. https://doi.org/10.1016/j.schres.2009.02.011

Begemann, M. J. H., Schutte, M. J. L., Slot, M. I. E., Doorduin, J., Bakker, P. R., van Haren, N. E. M., & Sommer, I. E. C. (2015). Simvastatin augmentation for recent-onset psychotic disorder: A study protocol. BBA Clinical. https://doi.org/10.1016/j.bbacli.2015.06.007

Boersma, P., & Weenink, D. (2013). Praat: doing phonetics by computer [Computer program]. In Glot International.

Cohen, A. S., Mitchell, K. R., & Elvev˚ag, B. (2014). What do we really know about blunted vocal affect and alogia? A meta-analysis of objective assessments. In Schizophrenia Research. https://doi.org/10.1016/j.schres.2014.09.013

Covington, M. A., He, C., Brown, C., Na¸ci, L., McClain, J. T., Fjordbak, B. S., Semple, J., & Brown, J. (2005). Schizophrenia and the structure of language: The linguist’s view. Schizophrenia Research, 77(1), 85–98. https://doi.org/10.1016/j.schres.2005.01.016

de Boer, J. N., Voppel, A. E., Brederoo, S. G., Wijnen, F. N. K., & Sommer, I. E. C. (2020). Lan-guage disturbances in schizophrenia: the relation with antipsychotic medication. Npj Schizophrenia. https://doi.org/10.1038/s41537-020-00114-3

Destrieux, C., Fischl, B., Dale, A., & Halgren, E. (2010). Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. NeuroImage, 53(1), 1–15.

https://doi.org/10.1016/j.neuroimage.2010.06.010

Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., & Truong, K. P. (2016). The Geneva Minimalistic Acoustic Param-eter Set (GeMAPS) for Voice Research and Affective Computing. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2015.2457417

Eyben, F., W¨ollmer, M., & Schuller, B. (2010). OpenSMILE - The Munich versatile and fast open-source audio feature extractor. MM’10 - Proceedings of the ACM Multimedia 2010 International Conference. https://doi.org/10.1145/1873951.1874246

(18)

language and affect. In Progress in Brain Research. https://doi.org/10.1016/S0079-6123(06)56012-1 Gründer, G., Heinze, M., Cordes, J., Mühlbauer, B., Juckel, G., Schulz, C., Rüther, E., & Timm, J. (2016). Effects of first-generation antipsychotics versus second-generation antipsychotics on quality of life in schizophrenia: a double-blind, randomised study. The Lancet Psychiatry.

https://doi.org/10.1016/S2215-0366(16)00085-7

Ho, B.-C., Andreasen, N. C., Ziebell, S., Pierson, R., & Magnotta, V. (2011). Long-term Antipsy-chotic Treatment and Brain Volumes. Archives of General Psychiatry.

https://doi.org/10.1001/archgenpsychiatry.2010.199

John Fox and Sanford Weisberg (2011). An R Companion to Applied Regression, Second Edition. Thousand Oaks CA: Sage. http://socserv.socsci.mcmaster.ca/jfox/Books/Companion

Karlsgodt, K. H., Sun, D., & Cannon, T. D. (2010). Structural and functional brain abnormalities in schizophrenia. Current Directions in Psychological Science. https://doi.org/10.1177/963721410377601 Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin. https://doi.org/10.1093/schbul/13.2.261

Kim, G. W., Kim, Y. H., & Jeong, G. W. (2017). Whole brain volume changes and its correlation with clinical symptom severity in patients with schizophrenia: A DARTEL-based VBM study. PLoS ONE. https://doi.org/10.1371/journal.pone.0177251

Laursen, T. M., Nordentoft, M., & Mortensen, P. B. (2014). Excess early mortality in schizophrenia. In Annual Review of Clinical Psychology. https://doi.org/10.1146/annurev-clinpsy-032813-153657 LeMay, M. (1976). Morphological cerebral asymmetries of modern man, fossil man, and nonhuman primate. Annals of the New York Academy of Sciences.

https://doi.org/10.1111/j.1749-6632.1976.tb25499.x

Leucht, S., Tardy, M., Komossa, K., Heres, S., Kissling, W., Salanti, G., & Davis, J. M. (2012). Antipsychotic drugs versus placebo for relapse prevention in schizophrenia: A systematic review and meta-analysis. In The Lancet. https://doi.org/10.1016/S0140-6736(12)60239-6

Lindell, A. K. (2006). In your right mind: Right hemisphere contributions to language processing and production. Neuropsychology Review. https://doi.org/10.1007/s11065-006-9011-9

Mitchell, R. L. C., & Crow, T. J. (2005). Right hemisphere language functions and schizophrenia: The forgotten hemisphere? In Brain. https://doi.org/10.1093/brain/awh466

Niznikiewicz, M., Donnino, R., McCarley, R. W., Nestor, P. G., Iosifescu, D. V., O’Donnell, B., Levitt, J., & Shenton, M. E. (2000). Abnormal angular gyrus asymmetry in schizophrenia. American Journal of Psychiatry. https://doi.org/10.1176/appi.ajp.157.3.428

Pantelis, C. (2003). Neuroanatomical abnormalities before and after onset of psychosis. LANCET -LONDON-, 1(9354), 281–288.

Parola, A., Simonsen, A., Bliksted, V., & Fusaroli, R. (2020). Voice patterns in schizophrenia: A systematic review and Bayesian meta-analysis. Schizophrenia Research, 216, 24–40.

https://doi.org/10.1016/j.schres.2019.11.031

Patel, K. R., Cherian, J., Gohil, K., & Atkinson, D. (2014). Schizophrenia: Overview and treatment options. In P and T.

Pernet, C. R., & Belin, P. (2012). The role of pitch and timbre in voice gender categorization. Fron-tiers in Psychology. https://doi.org/10.3389/fpsyg.2012.00023

Rutten, G. J. M., Ramsey, N. F., Van Rijen, P. C., & Van Veelen, C. W. M. (2002). Reproducibility of fMRI-determined language lateralization in individual subjects. Brain and Language, 80(3), 421–437. https://doi.org/10.1006/brln.2001.2600

Saha, S., Chant, D., Welham, J., & McGrath, J. (2005). A systematic review of the prevalence of schizophrenia. In PLoS Medicine. https://doi.org/10.1371/journal.pmed.0020141

Salomon, J. A., Vos, T., Hogan, D. R., Gagnon, M., Naghavi, M., Mokdad, A., Begum, N., Shah, R., Karyana, M., Kosen, S., Farje, M. R., Moncada, G., Dutta, A., Sazawal, S., Dyer, A., Seiler, J., Aboyans, V., Baker, L., Baxter, A., . . . Murray, C. J. L. (2012). Common values in assessing health

(19)

outcomes from disease and injury: Disability weights measurement study for the Global Burden of Disease Study 2010. The Lancet. https://doi.org/10.1016/S0140-6736(12)61680-8 // Scheppele, M., Evans, J. L., & Brown, T. T. (2019). Patterns of structural lateralization in cortical language areas of older adolescents. Laterality. https://doi.org/10.1080/1357650X.2018.1543312 // Shaywitz, B. A., Shaywltz, S. E., Pugh, K. R., Constable, R. T., Skudlarski, P., Fulbright, R. K., Bronen, R. A., Gore, J. C., Shankweiler, D. P., Katz, L., & Gore, J. C. (1995). Sex differences in the functional organization of the brain for language. Nature. https://doi.org/10.1038/373607a0

Sheng, J., Zhu, Y., Lu, Z., Liu, N., Huang, N., Zhang, Z., Tan, L., Li, C., & Yu, X. (2013). Altered volume and lateralization of language-related regions in first-episode schizophrenia. Schizophrenia Research. https://doi.org/10.1016/j.schres.2013.05.021

Sobin, C., & Alpert, M. (1999). Emotion in speech: The acoustic attributes of fear, anger, sadness, and joy. Journal of Psycholinguistic Research. https://doi.org/10.1023/A:1023237014909

Tahir, Y., Yang, Z., Chakraborty, D., Thalmann, N., Thalmann, D., Maniam, Y., Rashid, N. A. binte A., Tan, B. L., Keong, J. L. C., & Dauwels, J. (2019). Non-verbal speech cues as objective measures for

negative symptoms in patients with schizophrenia. PLoS ONE. https://doi.org/10.1371/journal.pone.0214314 Team, Rs. (2015). RStudio: Integrated development for R. RStudio. RStudio, Inc., Boston, MA URL

Http://Www.Rstudio.Com.

Torres, U. S., Duran, F. L. S., Schaufelberger, M. S., Crippa, J. A. S., Louz˜a, M. R., Sallet, P. C., Kanegusuku, C. Y. O., Elkis, H., Gattaz, W. F., Bassitt, D. P., Zuardi, A. W., Hallak, J. E. C., Leite, C. C., Castro, C. C., Santos, A. C., Murray, R. M., & Busatto, G. F. (2016). Patterns of regional gray matter loss at different stages of schizophrenia: A multisite, cross-sectional VBM study in first-episode and chronic illness. NeuroImage: Clinical. https://doi.org/10.1016/j.nicl.2016.06.002

Van Veelen, N. M. J., Vink, M., Ramsey, N. F., Sommer, I. E. C., van Buuren, M., Hoogendam, J. M., & Kahn, R. S. (2011). Reduced language lateralization in first-episode medication-naive schizophrenia. In Schizophrenia Research (Vol. 127, Issues 1–3, pp. 195–201). https://doi.org/10.1016/j.schres.2010.12.013 Voineskos, A. N., Jacobs, G. R., & Ameis, S. H. (2020). Neuroimaging Heterogeneity in Psychosis: Neurobiological Underpinnings and Opportunities for Prognostic and Therapeutic Innovation. In Bi-ological Psychiatry. https://doi.org/10.1016/j.biopsych.2019.09.004

Quen´e, H., Persoon, I. & de Jong, N. (2011). Praat Script Syllable Nuclei v2 https://sites.google.com/site/speechrate/Home/praat-script-syllable-nuclei-v2.

(20)

Appendix January, 2021

Chapter 6

(21)

6.1 Appendix A Interview

1. Kun je vertellen over je zwemles van vroeger? Hoe vond je dat? Wat vond je het moeilijkst? En het leukst?

Can you tell about your swimming lesson from when you were young? How did you like it? What did you find the most difficult? And what did you like the best?

2. Kun je vertellen over je tandartservaringen (slechte en goede ervaringen)? Kun je bijvoorbeeld vertellen over de laatste keer dat je bent geweest? En hoe heb je dat als kind ervaren?

Can you tell about your dentist experiences (bad and good experiences)? For example, can you tell me about the last time you’ve been? And how did you experience going as a child?

3. Heb je een rijbewijs? Zo ja, kun je wat vertellen over hoe de lessen gingen en hoe het examen ging? Do you have a driving license? If so, can you tell me how the lessons went and how the exam went? 4. Ben je wel eens in een pretpark geweest? Kun je daar wat over vertellen? Heb je een favoriet pretpark, vertel daar eens over? Waarom is het je favoriete park?

Have you ever been to an amusement park? Can you tell me about that? Do you have a favorite amusement park, tell me about it? Why is it your favorite park?

5. Naar welke Nederlandstalige Tv-programma’s kijk je vaak? En aan welke heb je een hekel? Waarom?

Which Dutch TV shows do you often watch? And which do you hate? Why?

6. Kijk je wel eens naar sport, zoals voetbalwedstrijden? Zo ja, naar welke wedstrijden en wat vind je daarvan? En ben je een fan van een bepaalde club of sporter?

Do you ever watch sports, like football matches? If so, what matches and what do you think about it? And are you a fan of a particular club or athlete?

7. Kun je je laatste droom beschrijven? Can you describe your last dream?

8. Hoe was je laatste verjaardag? Hoe vier je normaal je verjaardag? How was your last birthday? How do you usually celebrate your birthday? 9. Wat zou je doen als je een miljoen zou winnen?

What would you do if you were to win a million?

10. Als je voor altijd een bepaalde leeftijd kon hebben, welke leeftijd zou dat dan zijn? Waarom? If you could have a certain age forever, what age would it be? Why?

11. Als je overal ter wereld heen mocht, waar zou je heengaan? If you could go anywhere in the world, where would you go?

12. Welk klusje in huis vind je echt vreselijk om te doen? Waarom? Doe je het dan? What kind of household chore do you really hate? Why? Do you do it then?

13. Ga je wel eens op vakantie? Wat is je favoriete vakantiebestemming? Waarom? Do you go on vacation? What is your favorite vacation destination? Why?

14. Welke oorlog in de geschiedenis heeft indruk op je gemaakt? Waarom? What war in history has impressed you? Why?

15. Wat voor werk doen/deden je ouders?

What kind of work do your parents / did your parents do?

16. Wat voor werk doe je/heb je recent gedaan? Wat sprak je daarin aan? What kind of work do you / have you done recently? What did you talk about?

17. Kun je wat vertellen over je laatste sollicitatiegesprek? Was je zenuwachtig? Ben je aangenomen? Can you tell me about your last job interview? Were you nervous? Were you hired?

18. Wat is je favoriete jeugdherinnering? Wie waren erbij? Waarom vind je dat zo’n fijne herinnering? What is your favorite childhood memory? Who were there? Why do you think that’s a nice memory? 19. Hoe werden bij jou thuis de feestdagen gevierd?

(22)

20. Waar ben je geboren? Welke stad, ziekenhuis of thuis? Wie waren daarbij? Where were you born? Which city, in hospital or at home? Who were there? 21. Wat voor soort kind was je toen je klein was?

What kind of child were you when you were small?

22. Had je vroeger een lievelingsknuffel? Of iets anders dat je altijd bij je wilde hebben?

Did you have a favorite toy when you were young? Or something else that you always wanted to have with you?

23. Wat was je favoriete boek toen je klein was? En waarom was je daar zo dol op? What was your favorite book when you were small? And why were you so fond of it? 24. Lijk je op je ouders? Op wie lijk je het meest, waarom?

Do you look like your parents? Who do you resemble the most, why?

25. Heb je broers of zussen? Op wie lijk je het meest, waarom? Met wie had je het meest ruzie? Do you have any brothers or sisters? Who do you resemble the most, why? Who did you argue with most?

26. Wat wilde je vroeger worden (wat voor baan)? Heb je dat lang gedacht? Zou je dat nu nog willen? What did you want to become when you were young (what kind of job)? Did you want that for a long time? Would you still like that?

27. Heb je wel eens een huisdier gehad? Zou je een huisdier willen? Wat voor een? Hoe was je band daarmee?

Have you ever had a pet? Would you like a pet? What kind? How was your relationship with it? 28. Welke mensen waren belangrijk voor je in je jeugd? Waarom?

What people were important to you in your youth? Why?

29. Was je weleens ziek als kind? Heb je weleens in het ziekenhuis gelegen? Were you sometimes sick as a child? Have you ever been to the hospital? 30. Wat was je favoriete eten toen je klein was?

What was your favorite food when you were small?

31. Wat deed je vroeger waardoor je in de problemen kwam? Werden je ouders dan boos? What did you do in the past that caused you to get into trouble? Did your parents get angry? 32. Wat voor spellen speelde je vroeger? Waar speelde je meestal? Buiten/binnen? Met wie?

What games did you play when you were young? What did you usually play? Outside inside? With whom?

33. Kun je je de laatste keer herinneren dat je goed nieuws kreeg? Wat voor nieuws was dat? Wat voor effect had dat op je?

Can you remember the last time that you got good news? What kind of news was that? What effect did that have on you?

34. Hoe heb je je beste vriend/in ontmoet? How did you meet your best friend?

35. Ben je wel eens verliefd geweest? Weet je nog hoe jullie in gesprek kwamen? Was je verlegen? Weet je nog hoe jullie eerste afspraakje was? Was je zenuwachtig?

Have you ever fallen in love? Do you still know how you got talking? Were you shy? Do you remember how your first date was? Were you nervous?

36. Wat zijn belangrijke momenten in je leven geweest? Looking back, what were important moments in your life?

37. Welke mensen spelen nu een belangrijke rol in je leven? Waarom? Which people now play an important role in your life? Why?

38. Wat was een van de beste feestjes waar je ooit bent geweest? What was one of the best parties you’ve ever been to?

39. Als je een tijdmachine had, naar welke tijd in de toekomst/verleden zou je gaan? Waarom? If you had a time machine, what time in the future / past would you go? Why?

(23)

40. Als je met ´e´en persoon op aarde nu mocht spreken, met wie zou dat zijn? If you were to speak to one person on earth, who would that be?

41. Als je een dier was, welk dier zou je dan zijn en waarom? If you were an animal, what animal would you be and why? 42. Wat is je favoriete bezigheid in de zomer? Waarom? What is your favorite activity in the summer? Why?

43. Stel je wordt gedropt op een onbewoond eiland en moet daar een jaar blijven, welke 3 dingen zou je dan meenemen?

Imagine being dropped on an uninhabited island and staying there for a year, what 3 things would you bring?

44. Als je ´e´en van je zintuigen op moest geven welke zou het dan zijn? (horen, zien, voelen, ruiken of proeven) Waarom?

If you had to give up one of your senses, what would it be? (hear, see, feel, smell or taste) Why? 45. Wat is het leukste cadeau dat je ooit hebt gekregen? Van wie kreeg je het?

What is the best gift you ever received? Who did you get from? 46. Wat is je favoriete stripfiguur? Waarom die?

What is your favorite cartoon character? Why that one? 47. Wat is het beste/leukste dat je deze week is overkomen? What is the best / fun thing you have come across this week?

48. Wat is het vreemdste dat je ooit hebt gegeten? (bv. slak, oester) What is the strangest thing you have ever eaten? (e.g. snail, oyster)

49. Als je vandaag ´e´en wereldprobleem mocht oplossen, welk probleem zou dat zijn? Waarom? If you could solve one world problem today, what problem would that be? Why?

50. Als je nu een auto mocht kopen, wat voor auto zou dat dan zijn? If you were to buy a car, what kind of car would it be?

51. Wat voor soort huis is je droomhuis? Waar zou dit huis staan? Met wie zou je er willen wonen? What kind of house is your dream home? Where would this house be? Who would you like to live with?

52. Welke taal zou je nog willen leren spreken? Waarom deze taal? What language would you still like to learn? Why this language? 53. Als je voor ´e´en dag God zou zijn, wat zou je dan doen? If you could be God for one day, what would you do?

54. Hoe ziet jouw perfecte pizza eruit? Wat zit er allemaal op? What does your perfect pizza look like? What’s on it?

55. Wat is het raarste kledingstuk dat je ooit hebt gedragen? What’s the strangest piece of clothing you’ve ever worn?

56. Wat zou je doen als je een dag onzichtbaar zou kunnen zijn? What would you do if you could be invisible one day?

57. Van welk beroep droomde je als kind? Wat trok je hier toen aan aan?

What profession did you dream of becoming as a child? What did you like about it?

58. Als je getuige zou kunnen zijn van elke gebeurtenis in het verleden, heden of toekomst, welke zou het dan zijn?

If you could witness any event in the past, present or future, what would it be? 59. Als je elke willekeurige fictieve persoon zou kunnen zijn, wie zou je dan kiezen? If you could be any fictional person, who would you choose?

60. Als je uit iedereen in de wereld kon kiezen, met wie zou je dan uit eten willen? If you could choose from anyone in the world, who would you like to have dinner with? 61. Heb je liever een priv´evliegtuig of een priv´e-eiland? En waarom?

(24)

6.2 Appendix B Supplementary results

Table 6.1: Pearson’s correlation analysis results for LI and pitch variability, Df = degrees of freedom, r = Pearson’s correlation coefficient, * means the result is significant.

Table 6.2: Results Pearson’s correlation analysis between LI of brain areas and PANSS scores, Df = degrees of freedom, r = Pearson’s correlation coefficient.

(25)

Table 6.3: Results correlational analysis between PANSS and pitch variability, Df = degrees of freedom, r = Pearson’s correlation coefficient.

Table 6.4: Results correlational analysis between PANSS and total intercranial brain volume, Df = degrees of freedom, r = Pearson’s correlation coefficient.

Acoustic measures of prosody and their relation with lateralization of language areas in schizophrenia.

Faculty of Science