Characterising the Speech of People with Mental Illness

This year, my contribution to Psychonomics 2017 is in collaboration with Kristin Nicodemus of the University of Edinburgh and Alex Cohen of Louisiana State University.

My main research interest is to support people with chronic conditions. I also have a longstanding interest in the complex information that people convey in their speech and language – both intentionally, a signals to others, and unintentionally, as an expression of their socialisation, their anatomy, their physiology, and their current health.

This piece of work brings both together. Alex Cohen has a large collection of speech samples (17K+) from people with varying mental health conditions, which were analysed using a standard set of 88 features that have been used to describe aspects of speech and voice that are relevant to expressing emotion, the Geneva Minimalistic Acoustic Parameter Set (GeMAPS). GeMAPS is attractive, because it represents a consensus from the top researchers and practitioners in the field, and it comes with open source software for extracting those parameters from the speech signal.

GeMAPS contains 88 features and has been used mainly for classification of large data sets, but for smaller studies, it can be tricky to manage. Using principal component analysis in R, we reduced GeMAPS to a smaller set  of  features that are relatively easy to interpret from a phonetic point of view.

Using this reduced feature set, we’ve been able to identify distinct acoustic traces in the speech of people who have a history of depression and the speech of people who have a history of psychosis. These traces on their own are not enough to spot or diagnose mental illness or a history thereof, because they can be caused by many different factors. Instead, they reflect small, subtle changes, one of many traces that a person’s mental health leaves in their behaviour.

PDF of the poster:

Psychonomics 2017 PDF


Cohen A, Elvevåg B. 2014. Automated computerized analysis of speech disturbances in psychiatric disorders. Curr Opin Psychiatry 27:203–209

Cummins N, Scherer S, Krajewski J, Schnieder S, Epps J, Quatieri TF. 2015. A review of depression and suicide risk assessment using speech analysis. Speech Commun 71:10–49

Elvevåg B, Cohen AS, Wolters MK, Whalley HC, Gountouna V-E, Kuznetsova KA, Watson AR, Nicodemus KK. 2016. An Examination of the Language Construct in NIMH’s Research Domain Criteria: Time for Reconceptualization!. Am J Med Genet Part B 171B:904–919

Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, Sanislow C, Wang P. 2010. Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. Am J Psychiatry 167:748–751.

RDoC web page: Cognitive Systems / Language

The data used in this study come from these studies:

Cohen, A. S., Dinzeo, T. J., Donovan, N. J., Brown, C. E., & Morrison, S. C. (2015). Vocal acoustic analysis as a biometric indicator of information processing: Implications for neurological and psychiatric disorders. Psychiatry Research, 226(1), 235–241.

Cohen, A. S., Mitchell, K. R., Docherty, N. M., & Horan, W. P. (2016). Vocal Expression in Schizophrenia: Less Than Meets the Ear. Journal of Abnormal Psychology, 125(2), 299–309.

Cohen, A. S., Renshaw, T. L., Mitchell, K. R., & Kim, Y. (2016). A psychometric investigation of “macroscopic” speech measures for clinical and psychological science. Behavior Research Methods, 48(2), 475–486.



Leave a Reply

Your email address will not be published. Required fields are marked *