The Universal Signature of Memorization: Why Architectural Innovation Cannot Escape Privacy Leaks

Introduction

Membership inference attacks have long occupied an uncomfortable middle ground in machine learning privacy. For language models, we have relied on hand-crafted heuristics; thresholding perplexity, measuring zlib compression ratios, or identifying low-probability token sequences via Min-K%. Each method reflects an intuition about what memorization looks like, but none learn from data. Meanwhile, learned attacks using shadow models remain prohibitive at scale. Training hundreds of large language models to mimic a target’s behavior is computationally expensive and practically infeasible for most auditors.

The paper "Learning the Signature of Memorization in Autoregressive Language Models" by Ilić, Cvejoski, Stanojević, and Grigorenko from JetBrains Research resolves this tension through an elegant observation. Fine-tuning any model on any corpus automatically generates unlimited labeled data; membership is known by construction. This removes the shadow model bottleneck entirely and enables the first transferable learned attack for language models, which the authors term Learned Transfer MIA (LT-MIA). The results challenge fundamental assumptions about architectural boundaries in privacy research.

The Method and the Transfer Paradox

LT-MIA reframes membership inference as sequence classification rather than scalar thresholding. Given a candidate text and black-box access to both a fine-tuned target model and its pre-trained reference, the method extracts per-token distributional statistics capturing how the two models diverge. These features form a sequence fed into a lightweight transformer classifier, which learns to weight positions adaptively. This sequence modeling approach contributes 5.0 AUC percentage points over simple mean pooling, suggesting that the temporal structure of likelihood shifts contains signal invisible to aggregation.

The authors train this classifier exclusively on transformer-based models. When evaluated on held-out transformer combinations, it achieves 0.908 mean AUC. The surprising result comes from zero-shot transfer to architectures sharing no computational mechanisms with transformers. On Mamba-2.8B (selective state-space), RWKV-4-3B (linear attention), and RecurrentGemma-2B (gated recurrence), LT-MIA achieves 0.963, 0.972, and 0.936 AUC respectively. Notably, performance exceeds that on held-out transformers.

These architectures differ fundamentally. Transformers rely on pairwise attention with quadratic complexity. Mamba eliminates attention entirely in favor of input-dependent state transitions. RWKV replaces softmax attention with linear recurrences for constant-memory inference. RecurrentGemma combines gated linear recurrence with local attention windows. Their only commonality is optimization via gradient descent on cross-entropy loss.

The transfer is not method-specific. Even simple loss thresholding achieves 0.867 AUC on Mamba, while reference-based calibration reaches 0.892. This confirms that the signature exists in the likelihood landscape itself, independent of detection methodology. LT-MIA simply captures this signal most effectively, achieving 2.8 times higher true positive rate at 0.1% false positive rate than the strongest heuristic baseline (EZ-MIA) on transformer evaluations.

What the Signature Reveals

The existence of this architecture-agnostic signature implies that memorization manifests primarily in the output distribution rather than internal computation. Cross-entropy training induces a consistent shift; fine-tuned models assign higher likelihood to training samples relative to their pre-trained states, regardless of how they compute those likelihoods. Feature ablation confirms that comparison features, those relating target outputs to reference model outputs, dominate across all four architectural families with identical importance hierarchies.

This finding carries profound implications for privacy research. We have treated privacy leakage as a bug to be patched through architectural innovation. Perhaps attention mechanisms are uniquely prone to memorization, or perhaps recurrence offers natural protection. The evidence suggests otherwise. Leakage is intrinsic to the training objective itself. Any model optimized to minimize cross-entropy on a finite dataset will exhibit this distributional shift, making membership information accessible to any sufficiently sensitive detector.

The training protocol reveals why this universality emerges. The authors train on 30 model-dataset combinations with 600 samples each rather than one combination with 18,000 samples. Despite identical data volume, the diverse training achieves 0.860 AUC on held-out combinations versus 0.796 for the single-source model. This diversity acts as a filter, removing tokenizer-specific artifacts and architectural biases in logit scales while preserving the invariant likelihood shift induced by memorization. By learning to ignore transformer-specific patterns, the classifier discovers the fundamental signal present in any cross-entropy trained model.

Broader Implications and Limitations

These results force a reframing of the privacy problem. If the signature emerges from gradient descent on cross-entropy, then architectural innovation within this paradigm offers no escape. State-space models, linear attention variants, and recurrent architectures all leak information with comparable vulnerability. The privacy community must look beyond model families and toward training objectives.

However, important limitations remain. The paper focuses exclusively on fine-tuning scenarios where reference models are available. Pre-training memorization, arguably the more concerning setting for large-scale privacy violations, remains unaddressed. Additionally, the method requires white-box or API access to both target and reference models, limiting applicability to fully black-box settings where only generated text is available.

The transfer to code (0.865 AUC) despite training exclusively on natural language suggests the signature generalizes across data modalities, but further validation on multimodal or structured data domains would strengthen the claim. The 0.1% false positive rate operating point, while improved over baselines, still presents challenges for high-stakes privacy auditing where false positives carry significant costs.

Conclusion and Open Questions

The work of Ilić et al. establishes that memorization leaves a universal fingerprint in the likelihood landscape of autoregressive models. This signature persists across architectural families that share neither computational primitives nor inductive biases, emerging solely from the shared optimization objective. Until we move beyond gradient descent on cross-entropy, privacy leakage appears to be an unavoidable property of capable sequence models.

Several questions remain open. Can we design training objectives that maintain predictive performance while eliminating this distributional shift? Differential privacy offers theoretical guarantees but practical degradation in model capability. Perhaps alternatives like contrastive objectives or energy-based models might obscure the membership signature without catastrophic forgetting. Additionally, the success of diversity-based training suggests that meta-learning approaches to membership inference could yield even stronger universal detectors.

The field must now confront an uncomfortable reality. Architectural debates about attention versus recurrence, while important for efficiency and capability, do not address the fundamental privacy properties of modern language models. The signature is in the loss function, not the layers.