Fit Aware Virtual Try On: Moving Beyond Appearance Transfer with the FIT Dataset

Virtual try on technology has reached impressive levels of photorealism. Recent diffusion based methods can transfer garment appearance onto person images with remarkable fidelity. Yet as Karras et al. observe in "FIT: A Large Scale Dataset for Fit Aware Virtual Try On", these systems harbor a fundamental limitation. They optimize for visual coherence while ignoring physical fit. An extra large shirt on an extra small person renders as if tailored, erasing the very information shoppers need. This collapse of the fit distribution represents not merely a technical oversight, but a dataset problem. Real world try on datasets scraped from retail catalogs contain almost exclusively well fitted garments, creating a training bias toward aesthetic perfection regardless of size mismatch.

The FIT dataset addresses this gap through 1.13 million triplets of garment images, person images, and try on results, complete with precise body and garment measurements. The collection strategy leverages synthetic data generation to capture "ill fit" scenarios, garments that are significantly too large or too small, with physically accurate draping and deformation.

The Synthetic Pipeline: Physics, Geometry, and Photorealism

The data generation strategy described by Karras et al. operates in three stages, each solving a distinct technical challenge. First, the authors employ GarmentCode to procedurally generate 3D garments with parameterized measurements. Unlike artist created assets, this approach scales to millions of unique items while providing exact ground truth dimensions.

Second, the pipeline simulates garment drape using physics engines. This step proves crucial for fit aware training data. When a garment encounters a body smaller or larger than its target size, the fabric exhibits specific deformation patterns: stretching across the bust, pooling at the waist, or straining at the shoulders. Physics simulation captures these wrinkles and tension maps with accuracy that 2D image manipulation cannot replicate.

The third stage addresses the simulation to real domain gap. Raw synthetic renderings lack the texture complexity of real photographs. The authors fine tune Flux.1.dev, a foundational image generation model, to perform geometry preserving retexturing. The model takes synthetic normal maps and text descriptions as input, outputting photorealistic person images while strictly maintaining the underlying 3D geometry. This retexturing framework represents a significant technical contribution, as it solves the identity preservation problem simultaneously.

Paired Data and the Supervision Problem

Beyond fit representation, the FIT dataset solves a structural limitation in virtual try on research: the absence of paired training data. Existing methods must rely on self supervised reconstruction, where the same image serves as input and target, or on pseudo triplet synthesis which introduces masking artifacts and identity drift.

The synthetic pipeline enables controllable generation of the same parametric body in identical poses wearing different garments. This produces true paired supervision, where the network learns to modify garment geometry while preserving person identity. The distinction matters for practical deployment. Self supervised approaches struggle to maintain consistent facial features and body shape across try ons, while pseudo triplets leak size information from the source garment into the target. FIT provides ground truth pairings that enable direct supervised learning of the mapping from garment plus person to try on result.

Measurements as Structured Conditioners

Each sample in FIT includes precise annotations: bust, waist, hip circumference for bodies