Forensic Signals in Frozen Representations: Training-Free Detection of Partial Audio Deepfakes

Introduction

The threat landscape of audio manipulation has shifted from crude splicing detectable by spectral artifacts to sophisticated partial deepfakes where synthesized segments are seamlessly embedded within authentic recordings. Unlike fully synthetic utterances, which exhibit global distributional anomalies, partial manipulations preserve the speaker identity, environmental acoustics, and prosodic contours for the majority of the signal. This homogeneity renders conventional detection strategies, which rely on utterance level classification, largely ineffective. The forensic challenge intensifies when considering operational constraints: supervised detection systems require expensive frame level annotations, overfit to specific synthesis pipelines, and demand costly retraining as generative models evolve.

Addressing these limitations, Khan et al. propose TRACE (Training free Representation based Audio Countermeasure via Embedding dynamics), a framework that challenges the assumption that effective deepfake detection requires task specific learning. Their core insight, detailed in "TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models," suggests that self supervised speech foundation models implicitly encode forensic signals within their latent representations. Specifically, genuine speech traces smooth, slowly varying trajectories through embedding space governed by the physical continuity of human articulation, while splice boundaries introduce abrupt disruptions in frame level transition rates. By analyzing these first order dynamics without gradient updates or labeled data, TRACE achieves competitive performance with supervised baselines while offering superior generalization to unseen synthesis methods.

The Geometry of Embedding Trajectories

The theoretical foundation of TRACE rests on the observation that self supervised models such as wav2vec 2.0, HuBERT, and WavLM learn hierarchical representations that capture the temporal continuity inherent to natural speech production. During genuine utterances, vocal tract movements, breathing patterns, and acoustic context change gradually, resulting in embedding vectors that evolve smoothly across consecutive frames. Mathematically, this manifests as low first order dynamics, meaning the Euclidean or cosine distance between embeddings at adjacent time steps remains small and consistent.

Conversely, partial deepfakes splice segments generated by distinct neural text to speech or voice conversion systems into authentic recordings. At splice boundaries, the encoder must suddenly represent acoustic features produced by a different generative process with potentially mismatched prosody, micro dynamics, or spectral characteristics. This discontinuity produces measurable spikes in the embedding transition rate, effectively creating detectable anomalies in the latent trajectory. TRACE exploits this phenomenon by computing frame wise distances between consecutive embeddings and thresholding these values to identify manipulated regions.

This approach requires no architectural modification to the foundation models, which remain frozen throughout the process. The method operates entirely post hoc on extracted representations, making it agnostic to the specific