In spring 2022, the DigiTala project had just developed a Moodle plugin called DigiTala and investigated how teachers feel about it. The research project also wanted to know teachers’ opinions about the automatic assessment of oral language skills the plugin produces. Four teachers participated in the study. The teachers worked as Finnish or Swedish teachers in a Finnish high school or university. The interviews were carried out via Zoom.
The research revealed that teachers see many opportunities and benefits in the automatic evaluation of speech. Automatic assessment enables the student to practice independently of time and space and still receive feedback on their spoken language skills. Teachers found that the plugin could be used for homework assignments, preparing for oral exams, and as part of the high school language portfolio.
The interviewees emphasized the teacher's role as a complement to the automated assessment. The teachers felt it was important that the plugin gave the teacher the opportunity to correct or comment on the feedback given by the machine. The teacher can soften and explain the the machine-driven feedback in cases where it might seem unjustified for the student. Based on their experience the students also prefer more detailed feedback on their performance.
Technology can bring its own difficulties for teachers: Transferring audio files of speech performances from Moodle to other platforms can be complicated. The teachers also made aware that some of the students are anti-IT. The teachers are cautious about the fairness of the automatic assessment as well as automatic tests' coverage of language skills.
The assessment criteria of the DigiTala plugin - fluency, pronunciation, task completion and range - are similar to what the teachers would use in their own work. On the other hand, the interviews revealed that teachers may understand these criteria in slightly different ways.
The teachers drew attention to the fact that different language skills are emphasized in different ways between students of varying proficiency levels. For example, it may be relevant to assess vocabulary range of only advanced language learners. The DigiTala plugin does not relate assessments to overall proficiency level.
In the DigiTala plugin, the teachers missed the ability to modify the analytic rating scales according to the user's wishes. The holistic proficiency assessment (A1-C2) will soon be applied to spoken language tests in Finnish high schools. Therefore the holistic assessment was considered useful in teaching the students the connection between language competence and proficiency levels.
The interviews showed that an automatic assessment tool like DigiTala would have several application possibilities in the school world. You can read more about the research in the AFinLA’s online article.