The Trade-Off Between Accuracy and Agreement in AI Models

This section analyzes PEAR’s effectiveness by calculating consensus across six recognized explainer agreement measures, including as pairwise rank agreement, rank correlation, and feature agreement. PEAR training not only increases agreement between the explainers utilized in the loss (Grad and IntGrad), but it also makes significant progress in generalizing to explainers that are not visible, such LIME and SHAP.


This content originally appeared on HackerNoon and was authored by The Tech Reckoning is Upon Us!

Abstract and 1. Introduction

1.1 Post Hoc Explanation

1.2 The Disagreement Problem

1.3 Encouraging Explanation Consensus

  1. Related Work

  2. Pear: Post HOC Explainer Agreement Regularizer

  3. The Efficacy of Consensus Training

    4.1 Agreement Metrics

    4.2 Improving Consensus Metrics

    [4.3 Consistency At What Cost?]()

    4.4 Are the Explanations Still Valuable?

    4.5 Consensus and Linearity

    4.6 Two Loss Terms

  4. Discussion

    5.1 Future Work

    5.2 Conclusion, Acknowledgements, and References

Appendix

4.1 Agreement Metrics

In their work on the disagreement problem, Krishna et al. [15] introduce six metrics to measure the amount of agreement between post hoc feature attributions. Let [𝐸1(π‘₯)]𝑖 , [𝐸2(π‘₯)]𝑖 be the attribution scores from explainers for the 𝑖-th feature of an input π‘₯. A feature’s rank is its index when features are ordered by the absolute value of their attribution scores. A feature is considered in the top-π‘˜ most important features if its rank is in the top-π‘˜. For example, if the importance scores for a point π‘₯ = [π‘₯1, π‘₯2, π‘₯3, π‘₯4], output by one explainer are 𝐸1(π‘₯) = [0.1, βˆ’0.9, 0.3, βˆ’0.2], then the most important feature is π‘₯2 and its rank is 1 (for this explainer).

\ Feature Agreement counts the number of features π‘₯𝑖 such that [𝐸1(π‘₯)]𝑖 and [𝐸2(π‘₯)]𝑖 are both in the top-π‘˜. Rank Agreement counts the number of features in the top-π‘˜ with the same rank in 𝐸1(π‘₯) and 𝐸2(π‘₯). Sign Agreement counts the number of features in the top-π‘˜ such that [𝐸1(π‘₯)]𝑖 and [𝐸2(π‘₯)]𝑖 have the same sign. Signed Rank Agreement counts the number of features in the top-π‘˜ such that [𝐸1(π‘₯)]𝑖 and [𝐸2(π‘₯)]𝑖 agree on both sign and rank. Rank Correlation is the correlation between 𝐸1(π‘₯) and 𝐸2(π‘₯) (on all features, not just in the top-π‘˜), and is often referred to as the Spearman correlation coefficient. Lastly, Pairwise Rank Agreement counts the number of pairs of features (π‘₯𝑖 , π‘₯𝑗) such that 𝐸1 and 𝐸2 agree on whether π‘₯𝑖 or π‘₯𝑗 is more important. All of these metrics are formalized as fractions and thus range from 0 to 1, except Rank Correlation, which is a correlation measurement and ranges from βˆ’1 to +1. Their formal definitions are provided in Appendix A.3.

\ In the results that follow, we use all of the metrics defined above and reference which one is used where appropriate. When we evaluate a metric to measure the agreement between each pair of explainers, we average the metric over the test data to measure agreement. Both agreement and accuracy measurements are averaged over several trials (see Appendices A.6 and A.5 for error bars).

4.2 Improving Consensus Metrics

The intention of our consensus loss term is to improve agreement metrics. While the objective function explicitly includes only two explainers, we show generalization to unseen explainers as well as to the unseen test data. For example, we train for agreement between Grad and IntGrad and observe an increase in consensus between LIME and SHAP.

\ To evaluate the improvement in agreement metrics when using our consensus loss term, we compute explanations from each explainer on models trained naturally and on models trained with our consensus loss parameter using πœ† = 0.5.

\ In Figure 4, using a visualization tool developed by Krishna et al. [15], we show how we evaluate the change in an agreement metric (pairwise rank agreement) between all pairs of explainers on the California Housing data.

\ Hypothesis: We can increase consensus by deliberately training for post hoc explainer agreement.

\ Through our experiments, we observe improved agreement metrics on unseen data and on unseen pairs of explainers. In Figure 4 we show a representative example where Pairwise Rank Agreement between Grad and IntGrad improve from 87% to 96% on unseen data. Moreover, we can look at two other explainers and see that agreement between SmoothGrad and LIME improves from 56% to 79%. This shows both generalization to unseen data and to explainers other than those explicitly used in the loss term. In Appendix A.5, we see more saturated disagreement matrices across all of our datasets and all six agreement metrics.

4.3 Consistency At What Cost?

While training for consensus works to boost agreement, a question remains: How accurate are these models?

\ To address this question, we first point out that there is a tradeoff here, i.e., more consensus comes at the cost of accuracy. With this in mind we posit that there is a Pareto frontier on the accuracy-agreement axes. While we cannot assert that our models are on the Pareto frontier, we plot trade-off curves which represent the trajectory through accuracy-agreement space that is carved out by changing πœ†.

\ Hypothesis: We can increase consensus with an acceptable drop in accuracy

\ While this hypothesis is phrased as a subjective claim, in reality we define acceptable performance as better than a linear model as explained at the beginning of Section 4. We see across all three datasets that increasing the consensus loss weight πœ† leads to higher pairwise rank agreement between LIME and SHAP. Moreover, even with high values of πœ†, the accuracy stays well above linear models indicating that the loss in performance is acceptable. Therefore this experiment supports the hypothesis.

\ The results plotted in Figure 5 demonstrate that a practitioner concerned with agreement can tune πœ† to meet their needs of accuracy and agreement. This figure serves in part to illuminate why our

\ Figure 4: When models are trained naturally, we see disagreement among post hoc explainers (left). However, when trained with our loss function, we see a boost in agreement with only a small cost in accuracy (right). This can be observed visually by the increase in saturation or in more detail by comparing the numbers in corresponding squares.

\ Figure 5: The trade-off curves of consensus and accuracy. Increasing the consensus comes with a drop in accuracy and the trade-off is such that we can achieve more agreement and still outperform linear baselines. Moreover, as we vary the πœ† value, we move along the trade-off curve. In all three plots we measure agreement with the pairwise rank agreement metric and we show that increased consensus comes with a drop in accuracy, but all of our models are still more accurate than the linear baseline, indicated by the vertical dashed line (the shaded region shows Β± one standard error).

\ hyperparameter choice is sensibleβ€”πœ† gives us control to slide along the trade-off curve, making post hoc explanation disagreement more of a controllable model parameter so that practitioners have more flexibility to make context-specific model design decisions.

\

:::info Authors:

(1) Avi Schwarzschild, University of Maryland, College Park, Maryland, USA and Work completed while working at Arthur (avi1umd.edu);

(2) Max Cembalest, Arthur, New York City, New York, USA;

(3) Karthik Rao, Arthur, New York City, New York, USA;

(4) Keegan Hines, Arthur, New York City, New York, USA;

(5) John Dickerson†, Arthur, New York City, New York, USA (john@arthur.ai).

:::

:::info This paper is available on arxiv under CC BY 4.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by The Tech Reckoning is Upon Us!


Print Share Comment Cite Upload Translate Updates
APA

The Tech Reckoning is Upon Us! | Sciencx (2025-09-21T05:47:02+00:00) The Trade-Off Between Accuracy and Agreement in AI Models. Retrieved from https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/

MLA
" » The Trade-Off Between Accuracy and Agreement in AI Models." The Tech Reckoning is Upon Us! | Sciencx - Sunday September 21, 2025, https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/
HARVARD
The Tech Reckoning is Upon Us! | Sciencx Sunday September 21, 2025 » The Trade-Off Between Accuracy and Agreement in AI Models., viewed ,<https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/>
VANCOUVER
The Tech Reckoning is Upon Us! | Sciencx - » The Trade-Off Between Accuracy and Agreement in AI Models. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/
CHICAGO
" » The Trade-Off Between Accuracy and Agreement in AI Models." The Tech Reckoning is Upon Us! | Sciencx - Accessed . https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/
IEEE
" » The Trade-Off Between Accuracy and Agreement in AI Models." The Tech Reckoning is Upon Us! | Sciencx [Online]. Available: https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/. [Accessed: ]
rf:citation
» The Trade-Off Between Accuracy and Agreement in AI Models | The Tech Reckoning is Upon Us! | Sciencx | https://www.scien.cx/2025/09/21/the-trade-off-between-accuracy-and-agreement-in-ai-models-2/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts