The Thicket Effect: Why Large Language Models Make Post-Training Surprisingly Easy

A fascinating new paper from MIT CSAIL titled "Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights" by Yulu Gan and Phillip Isola challenges a fundamental assumption about how we should adapt large language models to new tasks. The conventional wisdom holds that sophisticated optimization methods like gradient descent are essential for post-training adaptation. This work demonstrates something remarkable: for sufficiently large models, random parameter perturbations followed by simple ensembling can match the performance of established methods like PPO and GRPO.

The Needle vs. Thicket Paradigm

The core insight of this research lies in reconceptualizing what pretraining actually produces. Rather than viewing the pretrained weights as a single point in parameter space that serves as a starting location for further optimization, the authors propose thinking of pretraining as producing a distribution over parameter vectors. Within the support of this distribution, task-specific experts already exist.

The key finding is that this distribution behaves very differently depending on model scale. Small models exist in what the authors call a "needle in a haystack" regime, where good solutions occupy a negligible fraction of the local weight space around the pretrained parameters. In this regime, finding improvements requires structured search methods like gradient descent to navigate the sparse landscape of beneficial modifications.

Large models, however, transition into a "thicket" regime where the neighborhood around pretrained weights becomes dense with task-improving solutions. The authors demonstrate this empirically by sampling random Gaussian perturbations around pretrained weights and measuring the fraction that improve performance on downstream tasks. Their results show this solution density scales dramatically with model size.

Scaling Laws for Solution Density and Diversity

Perhaps the most striking technical contribution is the demonstration that solution density follows a scaling law. The authors systematically study models from 0.5B to 32B parameters across the Qwen2.5 family, measuring how many random perturbations in a Gaussian neighborhood improve task performance. They find that larger, more capable models have exponentially higher densities of beneficial perturbations.

The paper also reveals an important secondary effect: solution diversity. The random perturbations that improve performance are not uniform generalists. Instead, they tend to be specialists where a perturbation that significantly improves performance on one task (say, mathematical reasoning) may actually hurt performance on others (like creative writing). This specialization pattern also scales with model size, with larger models showing greater diversity in their local solution neighborhoods.

This diversity finding has profound implications. It suggests that the pretrained weights don't just sit near many good solutions, but near many different kinds of good solutions. Each random perturbation discovers a different expert hiding in the weight space, specialized for particular capabilities.

RandOpt: When Random Guessing Becomes Viable

Motivated by these observations, the authors develop RandOpt, a remarkably simple post-training method. The algorithm samples N random parameter perturbations, evaluates each on the target task, selects the top K performers, and ensembles their predictions via majority voting. No gradient computation, no iterative updates, just parallel evaluation and selection.

The results are striking. On reasoning tasks like GSM8K, mathematical olympiad problems, and the Countdown task, RandOpt achieves performance competitive with sophisticated baselines including PPO, GRPO, and evolutionary strategies. The method is O(1) in training time since all perturbations can be evaluated in parallel, compared to O(T) for methods requiring T sequential optimization steps.

The inference cost is K times higher due to ensembling, but the authors show this can be mitigated through distillation. More importantly, for some tasks, even K=1 (just selecting the single best random perturbation) achieves reasonable performance.

Implications for Our Understanding of Large Models

These findings force us to reconsider several assumptions about large language models and optimization. First, they suggest that the conventional framing of post-training as "fine-tuning from a good initialization" may be incomplete. Instead, post-training might be better understood as discovering the many task-specific experts that pretraining has already embedded in the local weight space.

Second, the scaling behavior implies that as models grow larger, the optimization problem for post-training becomes progressively easier, not harder. This runs counter to intuitions from smaller-scale machine learning, where larger parameter spaces typically make optimization more challenging. Here, the pretrained representation becomes so rich that beneficial modifications become abundant.

The diversity of solutions also provides a new lens for understanding why ensemble methods work so well with large models. Rather than just reducing variance through averaging, ensembling may be tapping into genuinely different experts that have complementary strengths.

Limitations and Open Questions

While compelling, this work raises several important questions that merit further investigation. The experiments focus primarily on reasoning and mathematical tasks using specific model families. It remains unclear whether the thicket effect generalizes across all task types and model architectures. Tasks requiring substantial new knowledge (rather than reasoning with existing knowledge) might show different density patterns.

The Gaussian perturbation assumption, while convenient for analysis, may not represent the optimal sampling distribution. The authors use a fixed standard deviation for perturbations, but the optimal perturbation magnitude likely varies across layers and parameter types. More sophisticated sampling schemes could potentially improve results.

Additionally, the relationship between solution density and actual task performance deserves deeper analysis. High density of "improving" perturbations doesn't necessarily translate to high-quality solutions. The magnitude of improvements from random perturbations may still lag behind what gradient-based methods can achieve with sufficient computation.

Broader Context and Future Directions

This work connects to several important threads in contemporary AI research. The observation that large models contain many specialized solutions aligns with recent work on mechanistic interpretability suggesting that large models learn diverse, compositional representations. It also relates to findings about mode connectivity in neural networks, where different solutions can be connected by paths of similar performance.

The practical implications extend beyond post-training to areas like model merging, where understanding the density and diversity of solutions could inform better combination strategies. The thicket effect might also explain why techniques like model soups (averaging multiple fine-tuned models) work particularly well for large models.

Looking forward, several research directions emerge naturally. Developing better sampling distributions than simple Gaussian perturbations could improve efficiency. Understanding which types of tasks benefit most from the thicket effect could guide when to apply these methods. Most intriguingly, the scaling laws for solution density might inform decisions about when models are "large enough" that sophisticated optimization becomes unnecessary.

The Neural Thickets paper fundamentally challenges our intuitions about optimization in the era of large language models. It suggests that for sufficiently large models, the hard work of finding good solutions may already be done by pretraining. What remains is not optimization in the traditional sense, but discovery and selection among the many experts already embedded in the weight space. As models continue to scale, this perspective may become increasingly important for efficient and effective post-training adaptation.