Optimizer-Model Consistency: Why the Optimizer You Pretrain With Should Follow You Into Finetuning

When we discuss large language model training, we tend to treat the optimizer as a piece of interchangeable infrastructure. You pick AdamW because everyone uses AdamW, you tune a learning rate, and you move on. The architecture, the data, and the objective are where the interesting decisions supposedly live. A recent paper, "Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less" by Yuxing Liu, Jianyu Wang, and Tong Zhang, challenges this assumption in a way I find genuinely worth thinking through. Their claim is simple to state and harder to internalize: the optimizer is not just a procedure for minimizing loss; it is part of the model's prior, and that prior carries forward into how you should finetune.

The Core Observation

The supervised finetuning (SFT) stage sits in an awkward position. You want the model to acquire new skills, math reasoning or code generation for instance, while preserving the broad linguistic competence it picked up during pretraining. The authors call this the learning-forgetting tradeoff, and it is the natural axis along which to compare finetuning strategies.

Their central finding, which they name optimizer-model consistency, is that full finetuning with the same family of optimizer used during pretraining produces the best position on the Pareto frontier of this tradeoff. Concretely, they finetune Llama-2-7B on MetaMathQA for one epoch and plot general capability (averaged accuracy on Hellaswag, ARC-C, and Winogrande) against task capability (GSM8K exact match). Each point on the curve corresponds to a learning rate and training duration choice.

The result is striking on two fronts:

• Full finetuning with AdamW, the same optimizer used to pretrain Llama-2, dominates the frontier. It forgets less while learning the same or more.
• This holds even against LoRA, which is widely assumed to be the gentler, lower-forgetting option precisely because it constrains updates to a low-rank subspace.

The LoRA comparison deserves emphasis. The conventional intuition is that parameter-efficient methods forget less because they touch fewer parameters. The paper argues this intuition is incomplete. Once you actually sweep learning rates properly for full finetuning, the supposedly aggressive method wins on both axes. That should make anyone who has defaulted to LoRA "to avoid catastrophic forgetting" pause and reconsider whether they were comparing methods on a level footing.

Why Optimizers Shape the Landscape

The more interesting part of the paper is the mechanistic explanation. LLM training is nonconvex, and among the enormous set of near-equivalent loss minima, different optimizers systematically prefer solutions with different properties. This is the implicit regularization story, but the authors give it a specific shape.

They lean on the optimizer framework from Pethick et al. (2025), which views optimizers through the lens of matrix induced norms. The argument is that an optimizer applies regularization to the activations in a way tied to the norm on which its update rule is derived. AdamW, with its coordinate-wise adaptive scaling, induces one kind of activation structure; Muon, which orthogonalizes the update via a matrix sign-like operation, induces another.

The consequence is geometric. When you finish pretraining, the loss landscape around the checkpoint is shaped by the optimizer that got you there. The flat and sharp directions, the curvature, the conditioning of the local basin: these are not optimizer-neutral. They reflect the regularization that was being applied throughout training.

From here the logic for SFT follows naturally. If the weights already carry structure imposed by a particular optimizer, then the finetuning update should respect that structure to minimize disturbance to pretrained knowledge. Using the same optimizer in SFT produces updates that follow the same induced-norm geometry, so they move along directions that are "cheap" with respect to forgetting. A mismatched optimizer fights against the structure already baked into the weights, and the cost shows up as forgetting. The authors formalize this under simplified assumptions connecting activation properties to how an SFT update perturbs pretraining knowledge.

I want to flag what I find compelling here. The framing reframes the optimizer as a component of the model's inductive bias rather than a detail of the training procedure. We already accept that architecture is a prior and that data distribution is a prior. The claim that the optimizer co-determines the geometry of the solution, and therefore the right way to perturb it later, is a clean extension of that view.

The Muon Case Study

The paper does not stop at recommending consistency. It uses the consistency principle to argue for a better evaluation protocol for optimizers themselves. If the optimizer is carried through both stages, then an optimizer's quality should be judged on the full pretrain-then-SFT pipeline, not on pretraining loss alone.

Applying this lens to Muon versus AdamW yields a result that complicates the recent enthusiasm for matrix-structured optimizers. Muon often produces a stronger pretrained checkpoint, consistent with what the broader community has reported. But after SFT, its advantage is task-dependent and can vanish on reasoning tasks like math. Through a synthetic language modeling experiment, the authors suggest a mechanism: Muon has a stronger tendency toward rote memorization and may be weaker at acquiring patterns from small datasets. Since SFT operates on relatively little data, this memorization bias hurts exactly where pattern generalization matters most.

This is an important nuance. It means a leaderboard ranking based on pretraining benchmarks can mislead you about downstream utility. An optimizer that memorizes well may bank impressive pretraining numbers and then underperform once the goal shifts to learning transferable reasoning patterns from limited supervision.

My Take

I think the deeper contribution here is conceptual rather than the specific Pareto curves. We have a habit in this field of factorizing the training pipeline into independent decisions: choose architecture, choose data, choose optimizer, choose finetuning method. This paper is evidence that the optimizer and the finetuning method are coupled, and the coupling runs through the geometry of the learned solution.

A few inferences I would draw beyond what the paper states:

• The result likely interacts with learning rate schedules and warmup. If the geometry around the checkpoint is optimizer-specific, then the effective curvature also dictates the safe step size during SFT. Part of why a proper learning rate sweep lets full finetuning beat LoRA may be that the consistent optimizer makes the loss landscape better conditioned for the matching update, widening the range of stable learning rates.
• The LoRA comparison may depend on rank and target modules. The paper tests several ranks (16, 64, 256), but LoRA's forgetting profile is sensitive to where adapters are placed. I would want to see whether very low-rank, attention-only LoRA changes the story before concluding full finetuning is universally Pareto-dominant.
• This has practical consequences for open-weight ecosystems. If you download a checkpoint, you often do not know, or cannot replicate, the exact optimizer state and configuration used in pretraining. Optimizer-model consistency suggests this missing information is more valuable than we assumed. Model cards rarely report optimizer family with the seriousness this implies.

Limitations and Open Questions

The theoretical explanation rests on simplified assumptions modeling the observed activation regularization, so the mechanism should be read as a plausible account rather than a proof for full-scale models. The experiments center on Llama-2-7B and a math finetuning dataset; generalization across model scales, modalities, and finetuning objectives (RLHF, DPO, and other preference methods) remains open. The Muon memorization finding comes from a synthetic setting, which is illuminating but not the same as a controlled study on real reasoning corpora.

The open questions I find most interesting:

• Can we design an SFT update rule that adapts to whatever optimizer produced the checkpoint, recovering consistency without knowing the pretraining recipe?
• If Muon's weakness is memorization bias under low data, can a hybrid scheme, Muon for pretraining and AdamW for SFT, ever beat consistency? The paper suggests not, but the activation-mismatch cost versus the memorization cost is a tradeoff worth quantifying.

The takeaway I will carry forward is the one from the tweet: optimizers are not interchangeable plumbing. They are part of the model's prior, and the field would do well to treat them with the same care we give to data and architecture.