Scale-Aware Conditioning and the Geometry of Control in Portrait Video Generation

The generation of portrait videos under arbitrary camera trajectories represents one of the more subtle challenges in contemporary computer vision. While large video generation models have demonstrated remarkable capabilities in synthesizing realistic motion, the precise control of camera viewpoint in human-centric footage introduces unique geometric constraints. The recent work by Lyu et al., titled "FaceCam: Portrait Video Camera Control via Scale-Aware Conditioning," addresses a fundamental limitation in existing approaches. Specifically, they identify that scene agnostic camera representations, whether encoded as intrinsic and extrinsic parameter matrices or as Plücker ray embeddings, suffer from inherent scale ambiguity. The same camera translation vector produces vastly different visual effects depending on whether the subject is positioned one meter or ten meters from the lens. For human portraits, where millimetric differences in facial geometry carry significant perceptual weight, this ambiguity proves particularly problematic.

From Allocentric Coordinates to Facial Geometry

Traditional camera control methods operate within an allocentric reference frame. They treat the camera as an independent entity moving through absolute space, specifying trajectories via rotation matrices and translation vectors relative to a world coordinate system. This approach mirrors allocentric spatial coding in neuroscience, where locations are represented relative to external environmental cues rather than the observer's body. However, as FaceCam demonstrates, such representations fail to account for the metric properties of the subject itself.

The authors propose a scale-aware conditioning mechanism that fundamentally reorients the control problem. Rather than conditioning the video generation model on abstract camera parameters, FaceCam encodes the relative transformation between source and target poses using rendered facial landmarks. Specifically, the system renders projected facial keypoints under both the source and target camera configurations, creating an image-space representation of pixel correspondences. This representation effectively anchors the camera motion to the subject's intrinsic geometry. A translation of the camera results in a predictable displacement of facial landmarks proportional to the actual facial scale, eliminating the ambiguity that plagues scene agnostic methods.

This conditioning strategy offers several technical advantages. First, it provides deterministic control without requiring explicit 3D reconstruction of the head, thereby avoiding the geometric errors and identity drift common in reconstruction-based approaches. Second, it enables intuitive authoring; users specify camera trajectories by defining how the face should move within the frame, which the system translates into appropriate viewpoint changes. The representation naturally handles the non-rigid dynamics of human faces, accounting for the fact that facial features occupy a specific metric volume rather than existing as dimensionless points in space.

The authors validate this approach on the Ava-256 dataset and diverse in-the-wild videos, demonstrating superior performance in camera control accuracy compared to methods relying on scene agnostic representations or 3D reconstruction pipelines. The metric improvements are particularly evident in the preservation of facial proportions and the elimination of the geometric distortions that typically accompany camera motion in generated portrait videos.

Training Strategies and the Static-to-Dynamic Gap

A significant practical contribution of FaceCam lies in its training methodology. Acquiring paired data of the same dynamic portrait captured simultaneously from multiple moving cameras remains prohibitively expensive and technically challenging. The authors circumvent this limitation through a clever data generation pipeline that enables continuous camera trajectories at inference despite training exclusively on static multi-view captures.

The training regimen utilizes the NeRSemble dataset, a studio captured multi-view human video collection where cameras remain stationary. To simulate camera motion, the authors introduce two complementary strategies: synthetic camera motion generation and multi-shot stitching. Synthetic camera motion involves rendering novel views by interpolating between the static camera positions, creating plausible intermediate viewpoints. Multi-shot stitching, conversely, involves training on sequences where the camera appears to jump discontinuously between different viewpoints of the same temporal moment.

Intriguingly, the authors find that models trained on these discontinuous multi-shot sequences generalize effectively to smooth, continuous camera trajectories during inference. This suggests that the video generation backbone learns robust geometric priors from the static multi-view data, allowing it to interpolate motion between viewpoints without requiring explicit training on physically moving cameras. To prevent overfitting to the controlled lighting conditions of studio captures, the pipeline incorporates in-the-wild videos augmented with synthetic camera motion, ensuring generalization to diverse illumination and background conditions.

This training approach reveals important insights about the inductive biases of large video generation models. The ability to generalize from discontinuous multi-view data to continuous camera paths implies that these models possess strong internal representations of view consistency and temporal coherence. The multi-shot stitching strategy effectively decouples the learning of camera geometry from the learning of temporal dynamics, allowing the model to compose these factors at inference time.

Egocentric Reference Frames and the Future of Conditional Generation

The scale-aware conditioning in FaceCam illustrates a broader principle for controllable generative systems. The shift from allocentric camera parameters to egocentric facial landmarks mirrors fundamental organizational principles in biological visual systems. In neuroscience, egocentric coding represents spatial relationships relative to the observer's body, while allocentric coding uses external reference frames. Proprioception, our sense of body position, functions precisely because it anchors spatial awareness to our intrinsic morphology rather than to the variable dimensions of the surrounding environment.

By conditioning on rendered facial landmarks, FaceCam effectively implements an egocentric reference frame for camera control. The face becomes its own coordinate system, rendering camera movements interpretable regardless of the absolute scene scale. This approach suggests a general design pattern for conditional generation: control signals should anchor to the intrinsic geometry of the subject rather than to external absolute space.

This principle extends beyond portrait video. For any generative domain where subject scale varies, whether in robotic manipulation, architectural visualization, or microscopic imaging, conditioning mechanisms that respect the subject's metric properties will outperform scene agnostic controls. The key insight is that meaningful camera motion is always relative to the content being filmed. A dolly zoom creates dramatic effect precisely because of its relationship to subject scale; representing this motion without reference to the subject's geometry inevitably loses the semantic intent.

However, FaceCam's approach carries inherent limitations. The reliance on facial landmarks restricts the method to scenarios where faces remain visible and detectable. Occlusions, extreme angles, or non-human subjects would break the conditioning mechanism. Additionally, while the multi-shot stitching strategy proves effective, it assumes that the underlying video generation model possesses sufficient capacity to hallucinate physically plausible motion between discontinuous viewpoints. For complex dynamics such as hair simulation or clothing physics, this assumption may introduce temporal inconsistencies. The method also requires accurate facial landmark detection as a preprocessing step, introducing a potential failure mode for extreme head poses or unusual facial geometries.

Conclusion: Toward Subject-Centric Visual Control

FaceCam demonstrates that precise camera control in portrait video generation requires abandoning absolute coordinate systems in favor of subject-centric representations. The scale-aware conditioning strategy resolves the fundamental ambiguity that has limited previous approaches, while the training methodology offers a practical template for leveraging static multi-view data to learn dynamic camera control.

Looking forward, several questions remain open. Can this egocentric conditioning principle extend to full body video, where the larger scale variation and joint articulation introduce additional complexity? How might the approach adapt to scenes with multiple subjects, where the coordinate system becomes ambiguous? Furthermore, as video generation models grow more capable, the integration of explicit geometric priors with learned priors may yield even more robust control mechanisms.

The work represents a significant refinement in how we conceptualize camera control for human-centric content. By recognizing that the camera's relationship to the subject matters more than its position in absolute space, FaceCam points toward a generation paradigm where control signals align with human perceptual intuition. Future systems would benefit from extending this subject-centric philosophy to other modalities of control, such as lighting manipulation or temporal editing, creating generative tools that think in terms of the content rather than the coordinate system.