GPU-accelerated Personalisation of Automatic Speech Recognition for Electrolarynx Speakers
Project Overview
An electrolarynx(EL) is a device used to restore speech in people who have undergone a total laryngectomy. Automatic Speech Recognition (ASR) is computer understanding of speech. It is useful as a communication aid, for example subtitling on video conferencing.
In this project, we experimented with using synthetic data as an intermediate tuning step for existing ASR systems to improve their understanding of EL speech. We explore different methods of speech synthesis, and methods of personalising both the intermediate set and final tuning to a given speaker.
The project involved writing a tuning system which can be configured for these experiments, giving full reproducibility. The code also allows the experiments to be distributed across many GPUs for added efficiency.
What were the key results of your research project?
The results show promise with two-step tuning as a method of improving ASR for EL voices. Using a large synthetic dataset mitigates some of the problems caused when adapting ASR systems with very limited data. Careful synthesis setups and additional personalisation can help improve this further.
In addition, the results show that modern synthesis methods, like the neural audio codec based model we used, can capture speaker-specific characteristics that improve ASR adaptation for the speaker.
GitHub Repository: https://github.com/echogarratt/whisper-twostage-tune
A presentation of this research will be shared here and on our YouTube site when available.
How do you feel you have benefitted from completing this internship and has it made you consider future career paths?
Writing clean research code is a skill that I did not have coming into this internship. I have enjoyed learning about writing clean and efficient code, and ensuring it is easily configurable and experiments are easily documented for reproduction. Working with and learning about HPC code and job management has also been very interesting. I am keen to explore this more, and in my next academic steps, look for PhDs which will allow me to keep building these skills.
Download presentation slides
Internships 2026 - Echo Garratt