Locating the Convexity Prior: Mechanistic Interpretability Meets Gestalt Psychology in Vision Transformers

Introduction

The internal computations of Vision Transformers remain largely opaque, despite their dominance in computer vision. While we can measure their performance on benchmarks, understanding how these models organize visual information into coherent percepts requires careful mechanistic analysis. A fundamental test case for such interpretability lies in figure-ground organization, the process by which visual systems segregate scenes into foreground objects and background surfaces. In their paper "Locating and Editing Figure-Ground Organization in Vision Transformers," Stefan Arnold and Rene Gröbner provide a rigorous examination of where one specific Gestalt prior, convexity, is physically instantiated within the BEiT architecture. Their work moves beyond behavioral observations to demonstrate that perceptual biases are not distributed statistical artifacts but localized computational operations that can be identified and manipulated.

The Perceptual Conflict Paradigm

Arnold and Gröbner operationalize figure-ground organization through a elegant conflict stimulus based on synthetic "dart" shapes. The experimental design exploits a masked image modeling task, consistent with BEiT's BERT-style pretraining objective. Specifically, they mask the notch region of a concave polygon, creating a perceptual ambiguity where the local geometric evidence favors a concave completion (reconstructing the notch), while the global organizational prior favors a convex completion (closing the shape into a triangle).

This paradigm is methodologically significant because it decouples two competing influences: local edge evidence versus global shape priors. When BEiT reliably chooses convex completion, it demonstrates the same convexity bias observed in human visual psychology, where convex regions are preferentially assigned border ownership and perceived as figures rather than ground. By systematically varying the stimulus and measuring reconstruction probabilities, the authors establish that BEiT does not merely approximate convexity through memorization but actively employs it as an organizational principle during visual reasoning.

The synthetic nature of these stimuli allows precise control over the conflict region, defined as the patch-wise set difference between the concave shape and its convex hull. This geometric precision ensures that any preference observed can be attributed specifically to the convexity prior rather than texture statistics or dataset biases.

Tracing Computation Through the Transformer

The core technical contribution involves mapping this perceptual preference onto specific internal components using logit attribution. By projecting internal activations into BEiT's discrete visual codebook space, the authors trace how the convexity preference propagates through the network's residual streams. Their analysis reveals a characteristic temporal dynamic: figure-ground organization remains ambiguous through early and intermediate layers, with resolution occurring abruptly in deeper layers.

Critically, they identify attention head L0H9, located in the zeroth layer, as an early "seed" that introduces a weak initial bias toward convexity. This finding challenges simplistic views of transformer processing where early layers merely extract low-level features. Instead, L0H9 appears to function as a specialized circuit that seeds organizational hypotheses before they are amplified and resolved by downstream computations.

To establish causality rather than mere correlation, Arnold and Gröbner employ activation scaling, an intervention technique that modulates the functional contribution of specific components. By downscaling the output of L0H9, they demonstrate a reliable shift in the distributional mass of completions across the decision boundary. Reducing the influence of this single head causes the model to abandon its convex preference and instead follow the local concave evidence. This proves that the convexity prior is not an epiphenomenon of distributed representations but an active force generated by identifiable functional units within the attention subspace.

Implications of Modular Perceptual Organization

These findings carry substantial implications for how we conceptualize visual reasoning in artificial networks. The localization of the convexity bias to a specific attention head suggests a modular organization of perceptual priors within Vision Transformers. Rather than emerging from holistic, distributed patterns of activity, fundamental organizational principles like convexity appear to be computed by dedicated subcircuits.

This modularity validates long-standing neuroscientific analogies between artificial and biological vision. Just as the human visual cortex contains domain-specific processing streams and specialized regions for particular visual attributes, transformers may develop functionally distinct pathways for individual Gestalt laws. The identification of L0H9 as a convexity circuit parallels the discovery of face patches in primate temporal cortex or specialized neurons for convexity detection in V2.

If this modularity extends beyond convexity to other organizational principles such as closure, continuity, or symmetry, it would indicate that Vision Transformers develop specialized machinery for individual Gestalt laws rather than generic figure-ground mechanisms. This has practical significance for model editing and alignment. Current intervention methods often risk collateral damage to unrelated capabilities. However, if perceptual priors are indeed modular, we could precisely adjust specific biases, for instance, reducing texture bias while preserving shape sensitivity, or modifying border ownership assignments without disrupting object recognition.

The "seed and amplification" dynamic observed in BEiT, where early weak biases are resolved into definitive organizational decisions in later layers, also suggests an computational architecture reminiscent of predictive processing frameworks in neuroscience. Early layers may generate multiple competing hypotheses about figure-ground assignment, with later layers selecting and refining the most probable interpretation based on global consistency constraints.

Limitations and Forward Questions

Several limitations temper these conclusions. The study focuses exclusively on synthetic dart stimuli, leaving open whether these findings generalize to natural images where convexity operates alongside multiple competing cues. Additionally, the experiments isolate only the convexity principle; whether other Gestalt laws localize to distinct heads or share computational substrates remains unknown.

The specificity to BEiT also requires consideration. BEiT's discrete visual tokens and masked image modeling objective may encourage sharper localization of organizational priors compared to contrastive or autoregressive training paradigms. Whether similar circuits exist in CLIP, DINO, or other Vision Transformer variants demands further investigation.

Conclusion

Arnold and Gröbner's work represents a significant advance in our understanding of how Vision Transformers implement classical perceptual psychology. By locating the convexity prior to specific attention heads and demonstrating causal control through activation scaling, they provide empirical support for the view that these models develop interpretable, modular circuits for organizing visual input. The immediate research trajectory suggests extending this methodology to other Gestalt principles, testing whether closure, continuity, and proximity similarly localize to distinct functional units. If confirmed, this modular organization would not only deepen our neuroscientific understanding of artificial vision but also provide precise intervention targets for improving model robustness and alignment without disrupting the broad visual capabilities that make these systems useful.