Guided Evolution for Neural Architecture Search: Cheap Proxies, Smarter Search

Neural Architecture Search (NAS) promised to automate one of the most labor intensive parts of deep learning: designing the network itself. The early results were genuinely impressive, but they came with a price tag measured in GPU months. A persistent tension has shaped the field ever since. To learn anything reliable about a search space, you have to train candidate architectures, and training is exactly what makes NAS expensive. The paper Guided Evolution for Neural Architecture Search by Lopes, Santos, Degardin, and Alexandre proposes a way to soften this tension. Their method, G-EA, uses a zero-cost proxy to evaluate many architectures per generation while training only the single best one. The result is a search procedure that extracts information from the space continuously without paying the full training cost for each candidate.

In this article I want to unpack how G-EA works, why its design choices matter, and where I think the approach sits in the broader trajectory of efficient NAS.

The core problem: knowledge versus cost

A NAS method has three components: a search space, a search method, and a performance estimation strategy. The search space defines what architectures are possible. The search method navigates that space. The performance estimation strategy tells the search method how good each candidate is.

The estimation strategy is where most of the cost lives. The most trustworthy way to evaluate an architecture is to fully train it and measure validation accuracy. But full training of every candidate is prohibitive, so the literature has produced a long list of shortcuts: low fidelity training, learning curve extrapolation, weight sharing in one-shot supernets, and statistical surrogates. Each shortcut trades reliability for speed.

The paper highlights a specific and well documented failure mode of many prominent NAS methods: they converge too quickly to local minima. As soon as a search method finds architectures that look good, it doubles down on them, and the search collapses around a small region of the space. This is why some celebrated NAS methods fail to generalize to new datasets, a finding emphasized by the NAS-Bench-201 and benchmarking literature the authors cite (Dong and Yang, Yang et al.).

The deeper observation is that the cost problem and the convergence problem are linked. Because evaluation is expensive, you cannot afford to explore widely, so you exploit early, and exploiting early is precisely what drives premature convergence. G-EA attacks both sides at once.

How G-EA works

G-EA combines an evolutionary search strategy with a zero-proxy estimator. Zero-cost proxies, drawn from work such as Mellor et al. and Lopes et al., score an architecture at initialization using statistics computed from a few forward passes, with no training at all. These scores correlate reasonably well with the accuracy the architecture would eventually reach if trained.

The clever structural move is how the proxy is used inside the evolutionary loop:

• In each generation, G-EA generates several candidate architectures.
• All of them are scored with the zero-cost proxy at initialization.
• Only the highest scoring candidate is actually trained and carried into the next generation.

This is the heart of the method, and it is worth dwelling on. Conventional evolutionary NAS would train each member of a population to obtain fitness values. That is what makes exploration expensive. By replacing population-wide training with a population-wide proxy filter, G-EA gets to sample broadly from the space in every generation while only spending real compute on the one architecture most likely to be worth it.

On top of this, G-EA builds in two mechanisms to fight premature convergence:

• Exploitation through descendant generation from the most performant networks, so good structures propagate.
• Exploration through parent mutation and, importantly, an age bias that favors younger architectures over older ones.

The age-based preference is borrowed in spirit from regularized evolution (Real et al.), where aging out old architectures prevents any single high performer from dominating the population indefinitely. By continually retiring older candidates, the search is pushed to keep generating and evaluating new structures rather than settling.

The results in context

G-EA is evaluated on the NAS-Bench-201 search space, which is a standard tabular benchmark with precomputed accuracies for every architecture in the space across three datasets. This is the right place to test the idea because it removes confounds from training tricks; you can compare search methods on equal footing.

The reported mean accuracies are:

• CIFAR-10: 93.98%
• CIFAR-100: 72.12%
• ImageNet16-120: 45.94%

These are state-of-the-art results within the NAS-Bench-201 search space at the time of publication. The numbers are close to the optimal achievable in that space, which matters more than the absolute values. On NAS-Bench-201 the headroom between a good method and the global optimum is small, so improvements at the top end are meaningful signals that the search is finding genuinely strong cells rather than getting lucky.

The practical claim, which the tweet emphasized, is that this performance comes without a large compute budget. Because only one architecture per generation is trained, the cost scales with the number of generations rather than the number of candidates examined.

My take: why the design is more interesting than the numbers

The benchmark numbers are good, but I think the conceptual contribution is more durable than any single accuracy figure.

First, G-EA reframes the proxy not as a replacement for training but as a filter inside the loop. This is a subtle but important distinction. Some zero-cost NAS work tries to rank the entire search space using the proxy alone, which works only as well as the proxy's correlation with true accuracy, and that correlation is imperfect. By keeping a real training signal for one architecture per generation while using the proxy to screen candidates, G-EA hedges against proxy error. The trained architecture provides a reliable anchor; the proxy provides cheap breadth. This is a sensible division of labor, and it makes the method more robust to the known weaknesses of zero-cost metrics.

Second, the combination of proxy screening and age-based exploration addresses the cost and convergence problems jointly rather than separately. Most efficiency work targets only cost. Most diversity work targets only convergence. Coupling them is the right instinct, because in NAS the two failure modes feed each other.

There are limitations worth stating honestly:

• Search space dependence. NAS-Bench-201 is a constrained cell-based space. Results there do not automatically transfer to larger or differently structured spaces, where zero-cost proxies are known to correlate less well.
• Proxy quality is the ceiling. If the zero-cost estimator ranks candidates poorly, the screening step degrades to near-random sampling, and the only safety net is the single trained architecture per generation.
• Generalization of the proxy across datasets. Zero-cost proxies have shown uneven correlation on harder datasets like ImageNet16-120, and the lower absolute accuracy there hints that the screening signal weakens as the problem gets harder.

Conclusion and open questions

Guided Evolution for Neural Architecture Search offers a clean answer to a chronic NAS dilemma: how to learn about a search space without paying to train everything in it. By scoring many candidates per generation with a zero-cost proxy and training only the best, while using mutation and an age bias to keep exploring, G-EA pushes against both the cost wall and the premature convergence trap. Its state-of-the-art results on NAS-Bench-201 across CIFAR-10, CIFAR-100, and ImageNet16-120 validate the design within that benchmark.

The open questions I find most compelling are about scale and proxy reliability. Can a proxy-as-filter strategy hold up in open ended or hardware aware search spaces where the proxy correlation is noisier? Could the method adaptively decide how many candidates to train per generation based on proxy confidence, rather than always training exactly one? And as zero-cost proxies improve, does the trained anchor become unnecessary, or does it remain essential as a guardrail? The most practical implication, for teams without large GPU budgets, is that competitive architecture search no longer requires industrial scale compute, and that alone makes this line of work worth following.