EventHub: Rethinking Event-Based Stereo Through Proxy Data Generation

The asynchronous, high dynamic range nature of event cameras has long positioned them as superior alternatives to frame-based imaging in scenarios involving fast motion or extreme lighting variations. Yet despite their theoretical advantages, event-based stereo matching has remained constrained by a fundamental data bottleneck. Traditional deep stereo architectures require millions of labeled examples with dense, accurate ground truth depth, typically acquired through costly LiDAR rigs or laborious photogrammetric pipelines. This dependency on active sensors has effectively gated entry into event-based vision research to well-funded institutions with specialized hardware.

The paper EventHub: Data Factory for Generalizable Event-Based Stereo Networks without Active Sensors by Bartolomei et al., accepted at CVPR 2026, dismantles this assumption. The authors demonstrate that state-of-the-art event stereo networks can be trained entirely without LiDAR supervision, instead leveraging novel view synthesis from standard RGB images to generate proxy training data. Their resulting framework not only matches LiDAR-supervised baselines in domain but achieves substantially superior generalization, reducing mean absolute error by nearly 50% on out-of-distribution benchmarks.

The Data Factory Architecture

EventHub operates through two complementary data generation pipelines, both designed to eliminate the need for active sensor ground truth. The first pipeline addresses scenarios where only monocular RGB sequences are available. Using NeRFSt, a neural radiance field approach optimized for stereo synthesis, the framework generates novel viewpoint images from a single camera trajectory. From these synthesized views, the system derives proxy depth labels through multi-view geometric consistency. Simultaneously, proxy event streams are synthesized by computing brightness changes between rendered views, effectively simulating the event camera's response to motion and illumination changes without physical event sensors.

The second pipeline handles the increasingly common scenario where paired RGB and event data exist but lack depth annotations. Here, EventHub implements cross-modal knowledge distillation. State-of-the-art RGB stereo foundation models, such as FoundationStereo, process the color image pairs to generate high-quality disparity estimates. These predictions serve as pseudo-labels for training event stereo networks on the temporally synchronized event streams. This distillation mechanism proves particularly powerful because RGB foundation models have been trained on massive diverse datasets, encoding rich priors about scene geometry that transfer robustly to the sparse, asynchronous event domain.

Both pipelines feed into repurposed RGB stereo architectures. The authors adapt FoundationStereo, a transformer-based model originally designed for frame-based inputs, to process event representations by treating accumulated event frames or voxel grids as standard image inputs. This architectural repurposing suggests that the feature extraction and matching mechanisms developed for RGB stereo are largely modality-agnostic, requiring primarily appropriate data curation to unlock event-based performance.

Generalization and the LiDAR Paradox

The experimental results presented in the paper reveal a surprising paradox. On DSEC, the standard in-domain benchmark for event stereo, EventHub-trained E-FoundationStereo achieves a mean absolute error (MAE) of 0.89 pixels, statistically equivalent to EMatch (0.95 pixels), which relies on LiDAR supervision. However, the divergence becomes dramatic when evaluating out-of-domain generalization on M3ED, a dataset with substantially different camera characteristics and scene distributions. EMatch degrades to 4.05 pixels MAE, while E-FoundationStereo maintains robust performance at 2.53 pixels MAE, representing a 37.5% relative improvement.

This performance inversion suggests that LiDAR supervision, while providing precise metric depth, may introduce domain-specific biases that limit model generalization. LiDAR point clouds exhibit specific noise characteristics, occlusion patterns, and sparsity distributions that models may overfit during training. In contrast, proxy depth derived from neural radiance fields or foundation model distillation appears to encourage more geometrically consistent feature learning. The synthesis process inherently smooths over sensor-specific noise while preserving structural edges and textures crucial for stereo matching.

Furthermore, the paper demonstrates bidirectional knowledge transfer. When EventHub-trained event models are used to generate pseudo-labels for challenging nighttime RGB scenes, they improve the performance of RGB foundation models in low-light conditions. This synergy arises because event cameras naturally handle high dynamic range scenarios that saturate traditional frame-based sensors. The event model's robustness to extreme lighting effectively teaches the RGB model to better handle edge cases in its own domain, suggesting a future where multimodal training occurs through iterative cross-pollination rather than isolated unimodal optimization.

Critical Analysis and Limitations

While EventHub represents a significant methodological advance, several limitations warrant consideration. The reliance on novel view synthesis introduces a synthetic-to-real gap that the paper only partially addresses. NeRF-based rendering, while photorealistic, may fail to accurately model certain event camera artifacts such as junction leakage, shot noise, or the precise temporal jitter inherent in biological-inspired silicon retinas. Event cameras respond to logarithmic intensity changes with microsecond precision; approximating this response from frame-based renderings at 30 or 60 Hz necessarily discards high-frequency temporal information that may prove critical for ultra-fast motion scenarios.

Additionally, the distillation pipeline assumes geometric calibration between RGB and event cameras, a non-trivial requirement that may limit applicability to existing datasets where spatial and temporal alignment are imperfect. The scalability of the approach also depends on the computational cost of per-scene NeRF optimization, though the authors frame EventHub as a data factory, suggesting potential for amortized inference across large video collections.

My analysis suggests that EventHub's success fundamentally reframes the event vision problem as a data curation challenge rather than an architectural one. The community has invested significant effort in developing specialized event encoders, spike-based neural networks, and temporal aggregation mechanisms, often assuming that standard computer vision architectures were inadequate for asynchronous data. EventHub demonstrates that with sufficiently diverse and well-generated proxy data, conventional stereo architectures generalize effectively to event inputs. This finding aligns with broader trends in foundation model research, where scale and data diversity often supersede task-specific inductive biases.

Conclusion and Forward Outlook

EventHub opens several promising research trajectories. The framework's success suggests that similar proxy generation techniques could extend to other event-based tasks such as optical flow, visual odometry, or 3D reconstruction, potentially unlocking training at scales previously impossible due to annotation costs. The bidirectional improvement between RGB and event modalities also hints at a future of symbiotic sensor fusion, where each modality compensates for the other's failure modes through iterative distillation rather than simple early or late fusion.

Open questions remain regarding the theoretical limits of proxy supervision. Can synthetic events generated from RGB sequences ever fully capture the temporal fidelity of real event streams, or will there remain a fundamental performance ceiling for proxy-trained models in high-speed applications? Additionally, as event cameras become more prevalent in consumer devices, will the need for proxy generation diminish, or will it remain valuable for bootstrapping new architectures before deployment-scale datasets accumulate?

Ultimately, EventHub democratizes access to high-quality event stereo training, enabling researchers without LiDAR hardware to develop competitive models. By severing the dependency on active sensors, the framework accelerates the maturation of event-based vision from a specialized niche to a broadly accessible technology.