Sampling Without the Network: A Look at Conditional Diffusion Sampling

Introduction

Drawing independent samples from an unnormalized, multimodal probability distribution is one of those problems that looks deceptively simple on paper and turns out to be a persistent bottleneck across machine learning and the physical sciences. Whenever you can write down a density only up to a normalizing constant, and whenever evaluating that density is expensive (think molecular force fields or large Bayesian neural networks), the central question becomes: how do I get high-quality samples with the fewest possible density evaluations?

The paper Conditional Diffusion Sampling (Castro-Macías, Morales-Álvarez, Syed, Hernández-Lobato, Molina, and Hernández-Lobato, ICML 2026) proposes an answer that I find genuinely interesting because of what it removes rather than what it adds. The dominant trend in recent years has been to throw a neural network at the transport problem, learning a drift or a score that bridges a tractable reference to the target. CDS instead derives a transport dynamics that is closed-form and learning-free, then hands the hard part of the problem, global exploration, to a mature MCMC method: Parallel Tempering. The result is a hybrid that sidesteps neural training entirely while retaining the continuous-transport flavor of diffusion samplers.

The Two Worlds Being Bridged

To appreciate the contribution, it helps to recall the two paradigms CDS sits between.

Parallel Tempering (PT) is the gold standard among annealing methods. It defines a sequence of intermediate distributions, typically along a geometric path

$$\pi_\beta(x) \propto \pi_{\text{ref}}(x)^{1-\beta},\pi(x)^{\beta},$$

and runs multiple chains at different temperatures in parallel, swapping states so that information about the target propagates back toward the reference. Modern non-reversible variants (the NRPT of Syed et al., 2022) dominate the older reversible versions. PT is robust at global exploration: its parallel chains are good at discovering separated modes. Its weakness is well documented, namely slow convergence when the reference and target share little overlap, which forces long annealing schedules and many density evaluations.

Diffusion and flow-based samplers take the opposite stance. Following the Stochastic Interpolants framework of Albergo et al. (2025), they transport samples continuously along a probability path governed by an SDE or ODE. Neural Diffusion Samplers extend this to unnormalized targets. The catch, and the authors are honest about it, is cost: these methods generally require either pre-existing data or iterative optimization of a transport map, and both consume target density evaluations. They operate in an amortized regime that is complementary to MCMC rather than a drop-in replacement.

CDS asks whether you can get the continuous local transport of the diffusion view without paying for the neural approximation.

Conditional Interpolants and the Initialization Trick

The technical heart of the paper is the derivation of Conditional Interpolants. The standard stochastic interpolant constructs a marginal probability path between reference and target; its transport SDE has a drift that depends on a marginal score, which is intractable and therefore must be learned. The authors' move is to follow a conditional probability path instead. By conditioning on a reference sample, the transport dynamics admit an exact, closed-form SDE (Eq. 15 in the paper) whose drift depends only on the target score and the interpolant map, both of which are tractable.

There is no free lunch, and the constraint is precise: this exact dynamics cannot be initialized at $t=0$. You must start the integration from samples drawn from the probability path at some $t_0 > 0$, an initialization distribution the authors denote $\pi_{t_0\mid z}$.

The conceptual payoff is in how they turn this constraint into a feature. As $t_0 \to 0$, the conditional probability path concentrates toward a Dirac measure centered on a reference sample. In other words, for short diffusion times the initialization distribution is close to the reference itself, so sampling it is easy. They show both theoretically (vanishing transport cost as diffusion time decreases) and empirically that PT sampling of this near-reference initialization is efficient. The pipeline is then two-stage and clean:

• Stage 1. Use Parallel Tempering to sample the initialization distribution $\pi_{t_0\mid z}$ for near-zero $t_0$. Because the target here is close to the tractable reference, PT does not need to fight the poor-overlap problem that usually slows it down.
• Stage 2. Integrate the closed-form conditional diffusion SDE to transport those samples to the target $\pi$.

This is a sensible division of labor. PT handles global exploration where it is strongest, on a problem deliberately engineered to be easy for it, and the exact SDE handles local transport where deterministic continuous dynamics shine.

Empirical Signal

The evaluation spans 8 target distributions and 4 tasks. The headline result for me is the Alanine Dipeptide (ALDP) experiment in vacuum at $T = 300,$K, shown via Ramachandran histograms. Under a fixed budget of $2 \times 10^5$ density evaluations, only CDS and NRPT recover all modes, while OASMC, DiGS, MALA, and HMC miss modes. That two-method club is telling: NRPT is the strong non-reversible PT baseline, and CDS effectively inherits its mode coverage while adding continuous transport on top. The framing the authors use, a superior trade-off between sample quality and density evaluation cost, is exactly the right axis to measure on for these applications.

My Take

What I find compelling here is the philosophical inversion relative to the neural-sampler literature. The recent reflex has been to learn the hard object (a marginal score or transport map). CDS instead chooses a path along which the hard object becomes analytically available, accepting a non-trivial initialization in exchange. This is a recurring and underappreciated design pattern: reparametrize the problem so the intractable piece moves to a place where you already have a good tool. Here, the intractability is relocated into the initialization distribution, and PT is precisely the tool that handles it well.

A second observation concerns the right metric. As I noted in my earlier summary, the decisive figure of merit for expensive-density targets is effective samples per density call, not wall-clock or sample quality alone. CDS is explicitly optimized for this regime, and the ALDP result under a hard evaluation budget is the kind of comparison that matters. I expect this metric to become the standard reporting axis for samplers aimed at molecular and Bayesian targets.

That said, I want to be honest about limitations and open questions:

• The $t_0$ tradeoff is a knob, not a free parameter. Smaller $t_0$ makes Stage 1 cheap but presumably lengthens or stiffens the Stage 2 SDE integration, since the dynamics must carry samples further. The paper's claim of vanishing initialization cost as $t_0 \to 0$ raises the natural counter-question of how Stage 2 cost behaves in the same limit. The practical sweet spot likely depends on the target geometry.
• Dependence on the target score. The closed-form drift needs the target score $\nabla \log \tilde\pi$. For molecular systems with automatic differentiation through a force field this is available, but the cost of score evaluation relative to density evaluation should be accounted for in any honest budget comparison.
• Inheriting PT's failure modes. If the reference and the near-reference initialization still sit in a badly tempered landscape, Stage 1 could revert to PT's classic slow mixing. The method shifts where PT must work but does not abolish PT's fundamental constraints.
• Scaling to higher dimensions. ALDP is a meaningful but modest-dimensional benchmark. Whether the conditional path stays well behaved for the high-dimensional, rough landscapes of larger biomolecules is the test that will determine real adoption.

Conclusion

Conditional Diffusion Sampling is a clean example of getting more by assuming less. By deriving conditional interpolants with closed-form transport dynamics and pairing them with Parallel Tempering for the initialization, the authors build a sampler that needs no neural training yet captures multimodal structure other gradient-based methods miss at fixed budget.

My forward-looking bet stands: for targets where density evaluations are the binding constraint, training-free hybrids like CDS are well positioned to displace learned diffusion samplers over the next few years, with effective samples per density call as the metric that decides the contest. The open questions, the $t_0$ tradeoff, score-evaluation accounting, and high-dimensional scaling, are exactly the directions I would watch. If those hold up, the lesson generalizes well beyond this one paper: before learning a transport map, ask whether there is a path along which you would not have to.