Development and Implementation of a Perception Toolkit to Evaluate the Impact of Synthetic Speech on the Hearing Impaired

Abstract

In our ageing society, healthcare is in demand. This gives rise to the healthcare robot solution to carry out simple tasks such as medication reminders. The robot communicates with its patients through its text-to-speech system, and therefore makes it imperative for the synthetic voice to be intelligible, especially for the hearing impaired. In this thesis, we have created 4 sets of new New Zealand English voices using statistical synthesis to be used on the healthcare robot, adding to the existing diphone voice. To make these voices more intelligible for the hearing impaired, we need to understand the sounds they have trouble with, and what sounds they hear instead. Three intelligibility tests in varying complexity of phonetic intelligibility have been designed to tease out how well the hearing impaired can identify frequent consonants in English as well as one quality test to see how they feel about the voices. The tests were carried out in the form of a web-based survey using the diphone voice, one of the statistical voices created using the HMM method and one or two natural voice depending on the test. We were able to gather 160 complete responses, more than half of which experience hearing loss. We found that while participants preferred the natural voice over the synthetic voices, intelligibility wise it depends on the complexity of the phonetic environment. The diphone synthetic voice seems to fare the best when the words are unfamiliar and complex, such as medication names, whereas the natural voice performs the worst in complex environment, but more easily recognisable in familiar context. For more in-depth analysis on the phonemic level, new visualisation tools were developed to evaluate the sounds and confusions made by the hearing impaired participants in comparison with their normal hearing counterparts. These tools and the tests designed to evaluate synthetic voices can give us insight to what can be done to enhance these voices and make them more intelligible.