When Driving Models Memorize Instead of Generalize: A Look at Fail2Drive

Introduction

One of the more uncomfortable truths in autonomous driving research is that we rarely know whether a model has learned to drive or merely learned to pass a test. The distinction matters enormously. A system that has internalized robust concepts of free space, obstacles, and right of way will behave sensibly when confronted with a wild animal crossing the road. A system that has memorized the statistical regularities of its training distribution will not, and worse, it will appear competent right up until the moment it fails.

The paper Fail2Drive: Benchmarking Closed-Loop Driving Generalization by Gerstenecker, Geiger, and Renz from the University of Tübingen and the Tübingen AI Center confronts this problem directly. Their central observation is deceptively simple: most CARLA-based benchmarks evaluate models on the same scenarios they were trained on. Under such a protocol, high success rates can reflect memorization rather than generalization. Fail2Drive is designed to separate the two, and the results it produces are not flattering for the current state of the field.

The Paired-Route Design and Why It Matters

The methodological core of Fail2Drive is its paired-route design. The benchmark consists of 200 routes covering 17 new scenario classes, organized into four categories of distribution shift: appearance, layout, behavioral, and robustness. Crucially, each shifted route is matched with an in-distribution counterpart in which the location, road geometry, and surrounding traffic are held fixed. Only the targeted shift varies.

This is the kind of experimental hygiene that is easy to overlook but hard to overstate. When you compare a model's absolute performance on a difficult scenario against its performance on an easy one, you cannot attribute any difference to a single cause. Too many variables move at once. By holding everything constant except the induced shift, Fail2Drive turns a vague qualitative observation ("the model seems worse on rare events") into a clean causal measurement: the generalization gap, defined as the difference in success rate between the in-distribution route and its shifted twin.

This design enables genuinely counterfactual questions. Would a policy still yield if a pedestrian were swapped for an animal, while the road and traffic remained identical? Would it still avoid a road blockage if the obstacle asset were replaced with an unfamiliar one? Because the only thing that changed is the factor of interest, the answer is interpretable. The authors also deliberately derive their scores from planning performance rather than intermediate perception metrics like detection or classification accuracy. This is a meaningful choice. Many robustness studies stop at perception, but a detection error that never propagates into a bad driving decision is arguably irrelevant, while a perfectly perceived obstacle that the planner ignores is exactly the kind of failure that gets people hurt. Fail2Drive measures end-to-end decision degradation, which is closer to what we actually care about.

A further detail worth highlighting is the inclusion of a privileged expert policy to validate solvability. This addresses a subtle pitfall in adversarial benchmarking: it is easy to construct scenarios that are simply unsolvable, in which case low success rates tell you nothing about the model. By confirming that a privileged agent can handle each route, the authors ensure that failures reflect model limitations rather than impossible situations.

What the Failures Reveal

Across seven recent models spanning different paradigms, including end-to-end approaches like TF++ and more recent LLM and reinforcement-learning based systems, the average success rate drops by 22.8 percent under distribution shift. Consistency across paradigms is the part that should give the field pause. If only one architecture struggled, you might blame that architecture. When end-to-end, RL, and language-conditioned models all degrade, the problem is more fundamental.

The qualitative failure modes the authors surface are more telling than the headline number:

• Ignoring objects present in the LiDAR. TF++ was observed disregarding obstacles that were clearly visible in the LiDAR return. This is not a sensing limitation; the information was available and the model chose not to act on it.
• No working concept of free versus occupied space. The authors report that all models fail to learn a generalizable internal representation of obstacles. This is the most damaging finding, because free space estimation is arguably the most basic competency a driving system should possess.
• Missing fallback behavior. When confronted with uncertain situations, such as pedestrians walking in the road, models lacked any conservative default. A human driver who is unsure slows down. These systems did not.

Taken together, these point to a single underlying diagnosis: the models are pattern-matching against training scenarios rather than perceiving and reasoning over their environment. When a scenario falls outside the training manifold, the learned shortcut breaks, and there is no robust fallback concept underneath to catch the failure.

My Take

What strikes me most about Fail2Drive is not that models generalize poorly, which most practitioners suspected, but that the failures are so basic. A 22.8 percent drop could be charitably interpreted as the long tail being hard. But "failing to learn the concept of occupied space" is not a long-tail problem. It is a foundational one. It suggests that the impressive in-distribution numbers we celebrate may rest on a much thinner conceptual foundation than the architecture diagrams imply.

There is a deeper methodological lesson here that extends beyond driving. The entire field of embodied AI has a measurement problem when training and evaluation distributions overlap. We have seen analogous critiques in open-loop planning, where the nuScenes ego trajectory can be predicted from past motion alone without any scene understanding. Fail2Drive is essentially the closed-loop, controlled-shift version of that critique, and the paired design is what gives it teeth. I expect this paired-counterfactual methodology to influence how robustness is reported well beyond CARLA.

I am also intrigued by what the results imply for the current enthusiasm around vision-language-action models. The promise of these systems is that language-scale pretraining endows them with broad world knowledge that should transfer to novel situations. If a VLA model still cannot reason about a wild animal in the road when a pedestrian would have triggered correct behavior, then the transfer we hoped for is not materializing in the way the marketing suggests. Fail2Drive provides a concrete instrument to check whether future VLA claims are real or rhetorical.

Limitations and Open Questions

In fairness, several caveats apply. CARLA remains a simulator, and the visual and behavioral gap between CARLA and the real world means that failures observed here may not map one-to-one onto deployed systems, though if anything I would expect real-world long-tail generalization to be harder, not easier. The 17 scenario classes, while diverse, are still a finite and human-designed set, and a sufficiently determined model could in principle overfit to Fail2Drive itself once it becomes a standard target. The authors clearly anticipated this, which is why the open-source toolbox for generating new scenarios is so important: a benchmark that can be extended cheaply resists saturation.

There are open questions I would like answered. How much of the gap closes with simple data augmentation versus requiring architectural changes that explicitly represent free space? Is there a way to instill fallback behavior, perhaps through uncertainty-aware planning, that recovers performance under shift without retraining on every possible long-tail event? And can the paired-route methodology be ported to real-world datasets through controlled scene editing?

Conclusion

Fail2Drive does something valuable and slightly uncomfortable: it replaces our hope that driving models generalize with a measurement of how poorly they do. The paired-route design, the planning-level scoring, the solvability validation, and the open toolbox together form a reproducible foundation that the field has been missing. The 22.8 percent average drop is a useful number, but the qualitative findings, especially the absence of any robust notion of occupied space, are the ones that should reshape research priorities. If we are serious about closed-loop autonomy, the path forward is not better in-distribution scores. It is building systems that perceive and reason rather than pattern-match, and we now have a benchmark that can tell the difference.