In the Language Bank: Tamás Grósz

Who are you?

My name is Tamás Grósz and I am a Research Fellow in the  at the .

What is your research topic?

During my PhD years, my research was focused on Speech Technology, specifically on developing new deep-learning-based solutions for Automatic Speech Recognition (ASR). Although my main interest was acoustic modelling, I was also active in other areas. Paralinguistics, in particular, piqued my interest, and I worked on a wide variety of tasks. I regularly participated in the  and won several times over the years. Perhaps the most notable of our systems is the one that automatically assesses the condition of patients suffering from Parkinson’s disease. Besides the competitions, I was also part of a project that concentrated on developing a speech-based solution for early detection of mild cognitive impairment. In the last years of my studies, my focus shifted towards silent speech interfaces. I had the pleasure of working with state-of-the-art prototypes and developing new systems that could generate speech from ultrasound tongue movement videos.

After graduation, I joined Mikko Kurimo’s lab as a postdoc, where I had an opportunity to work on other topics, including language modelling and AI explainability. Initially, I worked on subword-based language models for agglutinative languages like Hungarian and Finnish. While working with various models, I noticed the importance of curriculum learning. As a spin-off project, I have started investigating different ways of estimating the difficulties of training samples and constructing new curriculums for AI models.

Simultaneously, working on projects like ,  and  enabled me to learn more about children’s ASR, speech assessment and tools that can aid language learners. Our best models have been successfully integrated into a  that can aid immigrants in learning the Finnish language.

In 2022, we developed a system that can recognize different kinds of stuttering (e.g. word/phrase repetition, prolongation, sound repetition and others) and won the INTERSPEECH 2022 Stefan Steidl Computational Paralinguistics Award. Later, we investigated how the emotional state of speakers can be recognized from non-verbal vocal expressions (such as laughter, cries, moans, and screams). Our system achieved first place for both tasks in the ACMMM CompParE competition. Since then, I have also worked on multimodal solutions for Emotion and Humor detection.

My current work mainly focuses on training and understanding Self-Supervised Foundation models as part of our  and the . Explainable AI and model interpretation has been a long-term interest of mine, and with these new models and computational resources, I had the opportunity to explore new techniques. Recently, I have developed ways to find the relevant subspaces inside large foundation models and explore the concepts discovered by the models during pre-training, as well as understand the changes caused by the fine-tuning process. These techniques enabled us to better understand our models and guided us in designing new, better training algorithms.

How is your research related to Kielipankki?

As modern speech recognizers require a considerable amount of data, it became a priority to collect and annotate suitable corpora. In 2020, I joined the team creating the . This corpus, with its approximately 3200 hours of donated speech, enabled various other projects, including our FinW2V2 project at LUMI. Using this dataset and , we have developed numerous Finnish ASR systems over the years.

Currently, I am also involved in the , building large speech foundation models and making them available for Industrial partners.

Recent publications

Getman, Y., Grósz, T., Hiovain-Asikainen, K. & Kurimo, M. (2024), , in Proc. of Interspeech.

Karakasidis, G., Kurimo, M., Bell, P. & Grósz, T. (2024), , Speech Communication p. 103113.

Moisio, A., Porjazovski, D., Rouhe, A., Getman, Y., Virkkunen, A., AlGhezi, R., Lennes, M., Grósz, T., Linden, K. & Kurimo, M. (2023), , Language Resources and Evaluation 57(3), 1295–1327.

Phan, N., von Zansen, A., Kautonen, M., Grósz, T. & Kurimo, M. (2024), , in Proc. of Interspeech.

Virkkunen, A., Sarvas, M., Huang, G., Grósz, T. & Kurimo, M. (2024), , in Proc. of IEEE ICASSP 2024, pp. 11196–11200.

Getman, Y., Phan, N., Al-Ghezi, R., Voskoboinik, E., Singh, M., Grósz, T., Kurimo, M., Salvi, G., Svendsen, T., Strömbergsson, S. et al. (2023), , in IEEE Access.

Grósz, T., Getman, Y., Al-Ghezi, R., Rouhe, A. & Kurimo, M. (2023), , in Proc. of Interspeech.

Grósz, T., Virkkunen, A., Porjazovski, D. & Kurimo, M. (2023), , in Proc. of the 4th Multimodal Sentiment Analysis Challenge and Workshop, pp. 27–34.

Porjazovski, D., Getman, Y., Grósz, T. & Kurimo, M. (2023), , in Proc. of the 31st ACM International Conference on Multimedia, pp. 9477–9481.

Corpora

The  consortium consists of a group of Finnish universities along with CSC – IT Center for Science and the Institute for the Languages of Finland (Kotus). FIN-CLARIN helps the researchers of Social Sciences and Humanities to use, refine, preserve and share their language resources.  is the collection of services that provides the language materials and tools for the research community.