One-sentence Summary

The authors advocate computational argumentation as a formal, computable foundation for Evaluative AI, enabling explainable and contestable multi-agent systems where diverse evaluative models from different sources and reasoning styles interact and jointly deliberate on competing hypotheses and evidence for human-centred decision-making.

Key Contributions

• Weighted Quantitative Bipolar Argumentation Frameworks (wQBAFs) are proposed as a formal, ranking-based foundation for Evaluative AI, enabling intrinsic explainability and contestability.
• The argumentative structure yields faithful explanations by making the evaluation process explicit and auditable, and contestability is supported through counterfactual explanations that allow users to modify argument weights to change hypothesis strengths.
• The paper advances a multi-agent vision where diverse evaluative models, derived from different sources and reasoning styles, can interact and deliberate on hypotheses and evidence, supporting collaborative human-in-the-loop decision-making.

Introduction

Explainable AI (XAI) often follows a recommend-and-explain paradigm, where a model first outputs a decision and then that decision is justified. This approach can cause cognitive fixation, leading users to over-rely on or dismiss AI suggestions without sufficient deliberation, which is especially problematic in high-stakes fields like healthcare, law, and finance. Evaluative AI (EAI) has recently been proposed as an alternative that presents multiple plausible hypotheses along with structured evidence for and against each, preserving human agency. However, existing EAI implementations (such as those based on Weight of Evidence) offer limited dependency structures among hypotheses and evidence, and lack a principled framework for explanation and contestability. The authors address these gaps by formalizing EAI as a ranking-based problem and introducing weighted Quantitative Bipolar Argumentation Frameworks (wQBAFs) as a formal foundation. This argumentative approach supports richer relations between hypotheses and evidence, enables faithful explanations of rankings, and allows users to contest any component, thus advancing human-centred, contestable, and potentially multi-agent EAI systems.

Method

The authors formalize Evaluative AI as a ranking-based problem and then propose an argumentative solution grounded in weighted Quantitative Bipolar Argumentation Frameworks (wQBAFs). The overall pipeline consists of three stages: formalizing the evaluation scenario, mapping it into a wQBAF, and deriving a ranking over hypotheses.

The framework begins with a set of hypotheses $\mathcal{H}$H and a set of evidence $\mathcal{E}$E. Each element $x \in (\mathcal{E} \cup \mathcal{H})$x∈(E∪H) is assigned an initial weight $\tau(x)$τ(x) that captures its baseline plausibility or severity before considering any interactions. Relations among these elements are modeled through two disjoint binary relations: pro and con. A pro relation $(x,y)$(x,y) indicates that $x$x supports $y$y, while a con relation indicates that $x$x attacks $y$y. These relations are allowed not only from evidence to hypotheses but also among evidence and among hypotheses, enabling multi-step reasoning chains. Each relation is further quantified by a relation weight $w(r) \in [0,1]$w(r)∈[0,1] that represents the strength of the influence.

Given this formalization, the ranking-based EAI problem is defined as determining a total preorder $\succcurlyeq_E$≽E​ over $\mathcal{H}$H, where $h_1 \succcurlyeq_E h_2$h1​≽E​h2​ means that $h_2$h2​ is not more plausible than $h_1$h1​. The authors argue that this ranking formulation naturally accommodates multiple hypotheses, aligns with the core philosophy of EAI, and provides a concrete computational framework for both generating and evaluating rankings.

To solve the problem, the authors map the EAI setting into a wQBAF. The wQBAF is a quintuple $Q = \langle \mathcal{A}, \mathcal{R}^+, \mathcal{R}^-, \tau, w \rangle$Q=⟨A,R+,R−,τ,w⟩, where $\mathcal{A} = \mathcal{E} \cup \mathcal{H}$A=E∪H is the set of arguments, $\mathcal{R}^+$R+ corresponds to the pro relation, $\mathcal{R}^-$R− corresponds to the con relation, $\tau$τ provides base scores, and $w$w provides edge weights. This mapping allows debate to take place over all evidence and hypotheses equally.

The solution proceeds in two steps. First, the system computes the final strength of every argument by applying a quantitative evaluation method that updates the initial weights based on the support and attack relations. Second, a ranking over hypotheses is derived by comparing their computed strengths. The authors emphasize that this wQBAF-based approach is inherently explainable: the graph structure qualitatively reveals the reasoning paths from evidence to hypotheses, while the final strengths can be quantitatively explained through attribution methods or counterfactual explanations. The framework also supports contestability, as users can modify weights or relations to observe how a hypothesis's strength changes.

To guide the selection and evaluation of ranking methods, the authors introduce a set of desirable principles. Point-wise principles include Monotonicity, which ensures that adding pro evidence for a hypothesis never lowers its rank, and Balance, which requires that rankings remain unchanged when pro and con influences cancel each other out. Pair-wise principles include Equivalence, which mandates equal ranking for hypotheses with identical evidence sets, and Dominance, which rewards hypotheses with superior pro evidence. At the global level, the principles of Robustness, Explainability, and Contestability govern the overall behavior of the ranking system, ensuring stability under perturbations, transparency of the ranking logic, and the ability for users to challenge the results.

Finally, the authors extend the framework to a multi-agent setting. They recognize that real-world evaluations integrate heterogeneous information sources, from human experts to domain tools and LLM-generated evidence. In this view, each wQBAF acts as an autonomous argumentative agent. Agents may engage in argumentative communication to surface new evidence or reconcile inconsistencies through principled exchange protocols. Multiple wQBAFs can also be fused into a group-level evaluation via semantic alignment of arguments, clustering of evidence structures, or ensemble-style aggregation over hypothesis rankings. This multi-agent perspective positions argumentation-based EAI as a foundation for distributed, resilient, and genuinely deliberative evaluation systems.