In our digitalising world, algorithmic systems are widely used in public governance. Examples range from parking enforcement and loan applications to social benefits allocation and criminal sentencing decisions. Scholars are largely concerned about opaqueness, privacy risks, biases and lacking accountability of algorithmic systems that may undermine the legitimacy of algorithmic governance. Studies show that citizens are also concerned with potential deficiencies of algorithmic systems and often consider algorithmic governance less legitimate compared to governance that involves humans.
While existing studies show what people perceive as (il)legitimate, they fail to explain why we observe a preference towards human-in-the-loop systems, whether this finding holds universally across contexts or what can cause a change in public attitudes. In AGAPP project we argue that in order to explain why we prefer humans-in-the-loop, we need an explicit account of how legitimacy judgments are created and changed in the minds of individuals when they encounter algorithmic systems that govern their lives.
Today, most studies follow a rationalist tradition and imply that legitimacy judgments are based on an assessment of how well a given system of power matches a latent set of collective norms and values. In contrast, cognitive psychologists demonstrate that people tend to use heuristics – colloquially ‘rules of thumb’ - to judge in various social situations. Heuristic judgments rely on structures of the environment rather than abstract norms, reducing complex problem-solving to simple context-bound judgments. AGAPP develops a novel Legitimation Heuristic Model (LHM) to explore how legitimacy judgments are created and changed in the minds of individuals when they encounter algorithmic systems of rule.
In LHM, legitimacy judgments are not produced by assessing a list of objective criteria, but rather by assessing cues present in the evaluation environment. In this way, LHM contrasts with existing explanations of legitimacy that build on goal-attainment, moral values, or both, and offers a distinct lens on the questions of when, why, and how citizens accord legitimacy to algorithmic systems of governance.
LHM gives rise to three interrelated research objectives:
Using rigorous qualitative and experimental methods, AGAPP will address these objectives to provide a new theoretical understanding of perceived legitimacy of algorithmic governance and generate knowledge on mechanisms and dynamics of legitimacy evaluations more generally.
The empirical case that AGAPP investigates is a salient example of algorithmic governance at the EU level: the new European Travel Information and Authorisation System (ETIAS). ETIAS targets citizens of 63 countries who can enter the EU zone visa-free but will have to apply for a mandatory authorisation through ETIAS starting 2025. The core of ETIAS is a risk profiling algorithm that both checks applicants’ personal data against existing databases, and estimates risks of potential future behaviours induced from probabilistic analysis of population statistics and personal data. In a post-pandemic conflict-ridden Europe, ETIAS is controversial: it holds a promise to improve European security, while raising surveillance levels to unprecedented levels, collecting and processing previously unavailable personal data at scale.
The project is organised through five Work Packages (WPs) that jointly contribute to the development of legitimation heuristic model through a combination of conceptual and empirical work (Figure 2). WP 1 addresses Objectives 1-3 conceptually by refining the LHM. WPs 2-4 elaborate the LHM through empirical study of ETIAS, using heuristics (Objective 1), legitimacy cues (Objective 2), and evaluation context dynamics (Objective 3) as points-of-entry. Together they aim at providing a robust empirical illustration to the central claim of this project – that legitimacy judgments tend to rely on heuristics. WP 5 provides an empirical and theoretical synthesis.