When Prediction Fails to Explain: The Non-Identifiability Problem in Computational Models of Perception

In the cognitive sciences, deep neural networks have become something akin to computational phrenology. Researchers train these models to predict human perceptual judgments, then dissect their internal representations to infer the mechanisms driving human behavior. The underlying assumption is elegant: if a model predicts what humans see, it might reveal how humans see. Yet a recent manuscript by Re Depaolini and Hasson, titled Non-identifiability of Explanations from Model Behavior in Deep Networks of Image Authenticity Judgments, exposes a critical flaw in this logic. Their work demonstrates that successful behavioral prediction provides almost no constraint on the explanatory mechanisms we derive from these models, fundamentally challenging how we interpret artificial neural networks as theories of human cognition.

The Experimental Architecture and Its Discontents

To understand the scope of this problem, consider the study's meticulous design. The authors focused on the perception of image authenticity, a domain where artificial intelligence has become sophisticated enough to generate content that deceives human observers. Using datasets such as AIGCIQA2023 and PKU-AIGIQA-4K, which contain thousands of AI-generated images paired with human ratings of perceived authenticity, the team trained lightweight regression heads atop frozen, pretrained vision architectures. This approach isolates the feature extraction capabilities of established backbones, including VGG variants, EfficientNetB3, and self-supervised models like Barlow Twins.

The technical methodology deserves attention. Rather than fine-tuning entire networks, the researchers preserved the pretrained representations and only optimized small linear layers to map features to human authenticity ratings. This design choice ensures that any observed behavior reflects the inherent biases and feature preferences of the architectures themselves, not artifacts of training dynamics. To probe these models, the authors deployed a triad of explainability techniques: Gradient-weighted Class Activation Mapping (Grad-CAM), Local Interpretable Model-agnostic Explanations (LIME), and multiscale pixel masking. Each method offers a distinct perspective on attribution, whether through gradient flow, local perturbations, or brute-force occlusion analysis.

The performance results were impressive by conventional standards. Several architectures achieved correlations with human judgments reaching approximately 80% of the noise ceiling, approaching the theoretical limit imposed by inter-subject variability. From an engineering standpoint, this represents a successful modeling effort. However, the authors immediately complicate this narrative with a crucial finding regarding the VGG family of models. Despite their high predictive accuracy, VGG architectures appeared to solve the authenticity task primarily by tracking image quality, such as sharpness and compression artifacts, rather than authenticity-specific cues. This dissociation between predictive success and mechanistic validity serves as the paper's first warning: behavioral alignment does not guarantee cognitive alignment.

The Crisis of Cross-Architectural Divergence

The central contribution of Re Depaolini and Hasson's work lies in their systematic evaluation of explanation consistency. Within individual architectures, attribution maps proved reasonably stable across random initializations, particularly for EfficientNetB3 and Barlow Twins. Moreover, the study noted an intriguing pattern: consistency was higher for images that humans judged as more authentic, suggesting that unambiguous cases may elicit more convergent feature use across model instances.

However, the critical test involved agreement across different architectures. Here, the results are sobering. Even when models achieved nearly identical behavioral performance, their attribution maps showed weak agreement. Two models might predict that an image receives an authenticity rating of 6.5 out of 10, yet one focuses on facial texture details while the other attends to background lighting consistency. This phenomenon constitutes the non-identifiability of explanations: multiple, mechanistically distinct solutions can produce indistinguishable behavioral outputs. The authors demonstrate that we cannot infer which psychological cues drive human judgment simply by observing which pixels influence our computational models.

This finding extends beyond technical limitations of specific explainability methods. The divergence persisted across Grad-CAM, LIME, and pixel masking, suggesting that the problem is not an artifact of saliency calculation but rather a property of the solution space itself. Deep networks can exploit numerous, orthogonal feature combinations to arrive at the same perceptual prediction. When modeling objective properties like object category, this multiplicity might seem benign. But when modeling subjective phenomena like authenticity perception, where specific cues carry psychological significance, this equifinality renders post-hoc explanations scientifically suspect.

Original Insights: The Underdetermination of Mechanism by Behavior

The implications of this work reach into the philosophy of science and the foundations of computational psychology. The non-identifiability problem described by Re Depaolini and Hasson is essentially an instance of underdetermination: empirical data, in this case behavioral predictions, necessarily underdetermine the theoretical mechanisms that generate them. In cognitive science, we have treated deep networks as if they were virtual subjects, assuming that forcing a model to match human behavior would constrain its internal logic to resemble human processing. This assumption now appears naive.

Consider the contrast with traditional cognitive modeling. When constructing a model of memory or decision making, theorists explicitly encode representational assumptions, then test whether these assumptions generate the observed behavior. The model is falsified if it predicts incorrectly, but it is also interpretable by construction because its mechanisms were specified a priori. Deep learning inverts this logic. We begin with a flexible function approximator, optimize for behavioral fit, and then attempt to reverse-engineer the mechanisms. The current study suggests this reverse-engineering fails because the forward mapping from mechanism to behavior is many-to-one.

Furthermore, the finding that VGG models rely on quality heuristics while other architectures presumably latch onto different regularities raises the specter of species-specific artifacts. Each architecture family represents a distinct evolutionary path through design space, shaped by different inductive biases, training objectives, and architectural constraints. When we compare attribution maps across these families, we are not converging on human-like explanations; we are simply observing the diverse strategies that different computational species employ to solve the same ecological problem. The ensemble methods proposed in the paper, while improving predictive accuracy through model averaging, do not resolve this fundamental diversity. They provide better prediction without providing identifiable explanation.

Conclusion: Toward Mechanistically Constrained Models

The work of Re Depaolini and Hasson should prompt a methodological recalibration in cognitive computational neuroscience. The field must recognize that behavioral prediction is a necessary but insufficient criterion for model validity. Future research should prioritize mechanistic constraints that force models to use human-like cues, perhaps through architectural decisions that privilege specific feature types, or through training regimes that penalize reliance on known shortcuts like image quality metrics.

One promising direction involves causal interventions: systematically perturbing specific image regions and verifying that models and humans show similar patterns of judgment change. Another approach requires multimodal constraints, integrating behavioral prediction with neural data from human observers to ensure that model representations align with biological processing stages. Without such constraints, we risk building ever more accurate predictors that tell us nothing about the mind.

The manuscript ultimately leaves us with a paradox. Deep networks can predict human authenticity judgments with remarkable fidelity, yet they offer no identifiable insight into the cues that drive those judgments. In an era of synthetic media, understanding human perception of authenticity carries obvious importance. But as this study shows, merely predicting that perception is not the same as explaining it. We must remain skeptical of our own creations, recognizing that correlation, even when perfect, masks a multitude of mechanistic sins.