Recommendations are increasingly integrated into various services, such as streaming services and e-commerce. AI-based recommender systems can influence what books we read, the movies we watch, and the products we buy. But little is known about how users feel about such systems.
Researchers at the University of Helsinki have recently conducted a study to explore how users perceive cross-domain recommendations. Cross-domain recommendations are where suggestions from one product category are based on users’ preferences in another, such as recommending books to users based on their movie tastes.
Users less trusting of cross-domain recommendations
In a study with 237 participants, the researchers asked half of the participants to list their favourite movies and the other half to list their favourite books. The research team randomly generated book recommendations but told participants that the recommendations were based on the list of movies or books they provided at the start.
The participants that listed their favourite movies then provided feedback on what they believed were cross-domain recommendations (book recommendations based on movie preferences). The rest believed they were given single-domain recommendations (book recommendations based on book preferences).
“Randomly generating recommendations was important because we were not interested in users’ opinions about the quality of recommendations, but their pre-conceptions of the cross-domain setting itself”, explains Alan Medlar, University Researcher in computer science at the University of Helsinki.
The findings suggest that users might be less trusting of cross-domain recommendations compared to traditional single-domain recommendations. The researchers found that just telling users that recommendations were cross-domain significantly reduced trust and lowered their interest in consuming the recommended items.
Recommendation explanations offer a partial solution
The team also studied recommendation explanations, where a system explains why something was recommended to a user.
“For example, an e-commerce website could say something like ‘this book has similar themes to your recent purchases, such as...’ and then goes on to list the themes”, explains Medlar.
When explanations were provided – clarifying why items were recommended – users' understanding and interest in cross-domain recommendations improved. Despite this improvement, however, trust did not reach the same level as single-domain recommendations.
From a practical standpoint, the results also underline the importance of user interface design in the deployment of AI-driven systems.
“From the perspective of industry, our work suggests that the provenance of recommendations should probably be hidden from users to avoid any pre-conceived ideas about whether recommendations are reasonable or not”, says Medlar.
Using algorithmic systems puts trust in the centre
According to the researchers, this is the first user study on cross-domain recommendations ever published.
“To the best of our knowledge, no one has ever attempted to investigate whether and how cross-domain recommendations impact user perceptions or behavioural intentions. Understanding user perceptions of these systems is crucial as it can impact not only user satisfaction but also broader trust in AI-based systems”, says Medlar.
“Recommender systems are usually viewed from an instrumental, behaviourist perspective, where their inclusion is to increase some KPI, such as revenue, user engagement or customer retention. I don't think anyone thinks about how the use of algorithmic systems in certain contexts can also decrease trust for the end-user.”
Publication:
Kotkov, D., Medlar, A., Liu, Y., & Glowacka, D. (2024). On the Negative Perception of Cross-domain Recommendations and Explanations. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2102–2113. New York, NY, USA: Association for Computing Machinery (SIGIR ’24). doi:10.1145/3626772.3657735