The Compositional Turn in Video Generation: Motion Control and Identity Preservation in DreamVideo-Omni

Video generation has reached an inflection point where raw fidelity alone is no longer the primary metric of success. As diffusion models have matured, the research community has shifted focus toward controllability; the ability to specify not just what appears in a video, but how it moves, from which angle it is viewed, and how multiple entities interact through time. This transition from monolithic generation to compositional control represents a fundamental reconceptualization of the task. Rather than treating videos as emergent phenomena of text prompts, we are moving toward frameworks where identity, motion, and camera trajectory become independently steerable parameters.

The paper DreamVideo-Omni: Omni Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning by Wei et al. addresses the central technical challenge in this transition: the inherent tension between subject identity preservation and motion control. The authors identify three critical failure modes in existing approaches. First, limited motion granularity restricts users to single control modalities, such as bounding boxes or sparse trajectories, preventing simultaneous control of global placement, local limb dynamics, and camera movement. Second, multi-subject scenarios suffer from control ambiguity, where models cannot determine which motion signal corresponds to which subject. Third, and most subtly, introducing motion control often degrades identity fidelity because identity preservation requires pixel level consistency with static reference images, while motion control necessitates dynamic temporal variation. Standard diffusion training objectives fail to reconcile these competing demands.

Architecting Explicit Control

DreamVideo-Omni's first stage addresses these limitations through a progressive joint training framework that integrates heterogeneous control signals: subject appearances, global motion, local dynamics, and camera movements. The technical innovation here lies in how these signals are encoded and bound to their respective subjects.

To coordinate these diverse inputs, the authors introduce a condition aware 3D rotary positional embedding (3D RoPE). Unlike standard positional encodings that treat all conditioning signals uniformly, this mechanism adaptively represents spatial and temporal relationships across different control modalities. This is crucial because bounding boxes, dense motion flows, and camera parameters occupy fundamentally different representational spaces and temporal resolutions.

More significantly, the architecture solves the binding problem through group embeddings and role embeddings. In multi-subject generation, simply concatenating motion signals leads to ambiguity; the model lacks explicit mechanisms to associate a specific trajectory with a specific identity. By introducing learnable group embeddings that cluster subjects and role embeddings that encode relational positions within scenes, the framework explicitly anchors motion signals to specific identities. This disentangles complex scenes into independently controllable instances, enabling what the authors term harmonious multi-subject customization.

The hierarchical motion injection strategy further refines this control by distinguishing between global object motion and local limb dynamics. Rather than treating motion as a monolithic flow field, the system injects coarse trajectories at deeper network layers while reserving shallow layers for fine grained local movements. This architectural choice mirrors biological motor control systems, where gross motor movements are planned before fine adjustments, and allows the model to maintain coherent global navigation while preserving detailed articulation.

Reinforcement Learning in Latent Space

While Stage 1 establishes controllability, Stage 2 addresses the identity degradation problem through a novel latent identity reward feedback learning paradigm. This represents a significant methodological departure from standard diffusion training.

The core insight is that reconstruction losses, whether pixel level or latent level, are insufficient for preserving fine grained identity under large motion amplitudes. These losses optimize for average case reconstruction, which inevitably sacrifices identity details to accommodate motion variation. Instead, Wei et al. train a latent identity reward model upon a pretrained video diffusion backbone, effectively implementing a form of Reinforcement Learning from Human Feedback (RLHF) adapted for spatiotemporal coherence.

The reward model operates directly in the diffusion latent space, bypassing the computational overhead of decoding frames for