The Hidden Collapse: When INT4 Quantization Fails After Full Convergence

Post-training quantization has long operated under a deceptively simple premise. If a model converges to a low-loss solution in full precision, surely it occupies a region of weight space that will survive discretization onto a low-bit integer grid. This assumption underpins the entire ecosystem of PTQ methods that compress large language models for deployment without costly retraining. Yet recent work has begun to erode this confidence, suggesting that the final stages of training may actively harm quantization readiness in ways that raw validation loss cannot detect.

In "When Flat Minima Fail: Characterizing INT4 Quantization Collapse After FP32 Convergence," Marcus Armstrong presents a forensic analysis of the Pythia-160m training trajectory that fundamentally challenges our understanding of when quantization fragility emerges. By applying a calibration-free per-group INT4 probe to all 154 publicly available training checkpoints, Armstrong uncovers a structured, three-phase collapse that begins not during learning rate decay, but at the precise moment when FP32 perplexity stops improving. The implications extend beyond a simple cautionary tale; they suggest that our standard training protocols may be systematically pushing models into quantization traps that remain invisible to standard monitoring.

The Architecture of Collapse: Three Phases of Divergence

The paper reveals that INT4 robustness does not degrade gradually or monotonically. Instead, training exhibits a distinct tripartite structure that maps poorly onto traditional loss curves. During the initial rapid-learning phase, both FP32 perplexity and quantization robustness improve in tandem; the model is learning meaningful features that happen to align well with the INT4 grid. This gives way to a meta-stable plateau lasting approximately 70,000 steps, where FP32 perplexity stagnates yet the INT4 gap remains bounded near 10 to 12 percent. This plateau represents a false sense of security; the model appears converged by standard metrics while remaining relatively quantization friendly.

The critical transition occurs at the onset of explosive divergence, where the INT4 gap compounds catastrophically from 11 percent to 517 percent while FP32 perplexity barely moves. This divergence is bit-width specific; INT8 quantization remains below 1 percent gap throughout all 143,000 training steps, constraining the mechanism to the coarseness of the 16-level INT4 grid specifically. The 500-fold difference between INT8 and INT4 sensitivity suggests that the phenomenon is not merely about general weight distribution drift, but about the specific geometry of how weights align with an extremely coarse quantization lattice.

Perhaps most surprisingly, Armstrong demonstrates that this divergence begins not when the learning rate starts decaying, as Catalan-Tatjer et al. previously suggested, but precisely when FP32 perplexity converges. This finer-grained onset predictor implicates post-convergence weight updates, rather than decay magnitude alone, as the proximate cause. The model continues to move in weight space after it stops improving in floating-point loss, and these movements push it into regions where the INT4 grid cannot adequately represent the weight configurations.

Mechanistic Investigations and the Failure of Outlier Accumulation

To understand what drives this collapse, Armstrong investigates a plausible mechanism proposed by concurrent work: outlier accumulation. Park et al. have argued that Adam's diagonal preconditioning causes activation outliers that inflate quantization scale factors, proposing excess kurtosis as a tracking metric. If outlier accumulation were driving the INT4 gap explosion, we would expect weight kurtosis to correlate positively with quantization error during the divergence phase.

Instead, Armstrong finds that weight excess kurtosis peaks at the phase 2 or phase 3 boundary, then declines throughout the explosive divergence phase, exhibiting a negative correlation of r equals negative 0.26 against the INT4 gap. This directly contradicts the outlier-accumulation hypothesis for this specific setting, ruling out a weight-level mechanism that has gained traction in the quantization community. The finding is particularly notable because Armstrong's probe measures weight-only INT4 quantization, distinct from the weight-and-activation W4A4 setting that Park et al. target, suggesting that different quantization regimes may suffer from fundamentally distinct pathologies.

The paper further probes whether oscillatory learning rate schedules can rescue quantization robustness through controlled fork experiments branching from the pre-divergence checkpoint. Comparing cosine continuation, SGDR warm restarts, and a proposed Oscillatory Lock-In method across nine independent runs, the results demonstrate that schedule amplitude, not oscillation alone, determines whether perturbation helps or hurts. SGDR warm restarts at full eta max amplitude uniformly accelerate divergence, achieving zero pairwise wins against cosine across all nine runs. In contrast, OLI's settled cool phases reduce the INT4 gap by 2.2 percentage points on average with high statistical significance. This suggests that calibrated perturbations followed by structured convergence periods can steer weights toward more quantization-friendly basins, while aggressive restarts simply push the model deeper into the collapse zone.

Your Take: The Geometry of Discrete and Continuous Spaces

This work exposes a fundamental tension between continuous optimization and discrete inference that the deep learning community has been slow to acknowledge. The finding that flat minima in the Hessian sense do not imply discrete-space robustness is particularly troubling for methods like SAM or HERO that explicitly pursue geometric flatness. A model can exhibit lambda max approximately zero while simultaneously suffering a catastrophic INT4 gap, confirming that the landscape relevant for 32-bit floating point and 4-bit integer inference are topologically distinct.

The bit-width specificity of the phenomenon suggests that as we push toward extreme compression ratios, we encounter phase transitions in representational capacity that do not scale linearly with precision. The 16-level INT4 grid creates a combinatorial matching problem between weight configurations and representable states that becomes increasingly pathological as training continues past convergence. This aligns with Chen et al.'s finding that quantization error increases with more training tokens, but Armstrong's checkpoint-level analysis reveals that this increase is structured rather than monotonic.

The implications for training practice are immediate. Current early stopping criteria based solely on validation loss or perplexity are insufficient for models intended for INT4 deployment. We need quantization-aware training monitors that track the native INT4 gap using calibration-free probes, treating quantization robustness as a first-class metric rather than a post-hoc deployment concern. The success of Oscillatory Lock-In also suggests that we may need to rethink the final phase of training not as a gentle settling into a minimum, but as an active search process that explicitly accounts for the discrete grid structure of the target deployment environment.

Limitations and Forward Questions

This study is necessarily bounded by its scope. The analysis focuses on Pythia-160m, a relatively small model compared to the 32B parameter systems where quantization challenges become most acute. While the structured nature of the three-phase collapse suggests a fundamental phenomenon, the specific step counts and gap magnitudes may not transfer directly to larger architectures or different optimizers. Additionally, the exclusion of activation quantization means these results speak most directly to weight-only INT4 scenarios rather than the W4A4 settings increasingly common in edge deployment.

Several questions remain open. If outlier accumulation is not the mechanism, what topological property of weight space drives the post-convergence divergence? The negative correlation with kurtosis suggests we need new descriptive statistics for weight distributions that predict discrete grid alignment. Furthermore, can the Oscillatory Lock-In principle be extended to full-scale training, or does the calibration requirement make it prohibitively expensive for billion-parameter models?

As the field moves toward ubiquitous INT4 deployment for large language models, Armstrong's work serves as a necessary correction to the assumption that training convergence implies deployment readiness. The discovery that our models undergo a hidden collapse invisible to standard metrics, occurring precisely when we think they are finished learning, demands that we develop training protocols with quantization awareness built into their fundamental architecture rather than applied as an afterthought. The gap between 32-bit convergence and 4-bit viability is not merely a technical hurdle; it is a structural feature of how we optimize in continuous spaces while deploying in discrete ones.