App to help immigrants learn the language

Expertise in psychology, language pedagogy and speech technology can generate solutions which may have extensive financial economic impact as immigration increases.

Organising language education is expensive, but could speech technology support immigrants in learning the local language? PhD Sari Ylinen heads a project which seeks to answer this question. In June, the project got into the second round in the Tekes Challenge Finland competition. A total of six University of Helsinki projects received the initial funding.

 “Our application helps a beginner language learner accrue a more extensive vocabulary and can evaluate pronunciation. This way, the learner will focus on producing understandable speech from the very beginning. Most applications which take advantage of automatic speech recognition can parse the speech of native speakers, but problems arise when the speaker is a child or an immigrant," Ylinen explains.

Previously, she has worked on the Academy of Finland funded game Say it again, kid! which is intended to help Finns learn English. The game will be piloted in schools this autumn.

Aiming for a functional, motivating learning tool

The speech interface, available on mobile devices, is suitable for a range of age groups and even caters to the illiterate, or those whose native language uses an alphabet which is radically different from ours. The idea is to rapidly introduce learners to the basic vocabulary which can help them cope with everyday situations and to focus on spoken language.

 “During the autumn, we will be testing the Say it again, kid! game and how automatic speech recognition compares to the evaluation of a human listener. We are interested in how gamification and the feedback provided by the app impact learning. The efficiency of the app can be measured, for example, by taking an EEG both before and after playing. Learning will be reflected in the brain responses,” Sari Ylinen explains.

 “It’s wonderful that this topic has been recognised in the Tekes competition. I consider the language education of immigrants an important social issue, as language proficiency is the key to the work force as well as social circles and it helps with integration.”

Challenge Finland has a name that is reminiscent of the Helsinki Challenge competition, organised last year by the University of Helsinki. The competition seeks commercially viable solutions to significant problems.

The full name of Ylinen’s project is Technology-enhanced language learning with mobile devices (TELL me). She has partnered with Professor Mikko Kurimo’s research group from the Aalto University’s Department of Signal Processing and Acoustics, and negotiations for cooperation are underway with several language and education technology companies. The current stage of Challenge Finland will run until mid-November.

Projects from all campuses

The goal of the Challenge Finland competition is to combine Finnish top research and the research and development activities of companies as well as to boost new Finnish exports and services to the market.

The medical projects from the University of Helsinki which have been accepted into the second round deal with micropump technology based on magnetic memory metals, and seek to speed up the clinical testing stage of drug discovery. Read more on the website of the Faculty of Medicine (in Finnish).

Tekes has allocated €30 million in key project funding to support the Challenge Finland projects. The idea projects in the first stage received €5.4 million of this sum.

University of Helsinki projects which proceeded to the second round of Challenge Finland: