LuMamba: Rethinking EEG Foundation Models Through Geometric Invariance and State-Space Efficiency

The proliferation of foundation models in medical signal processing has encountered a fundamental tension in electroencephalography. On one hand, transformer architectures have demonstrated remarkable representational capacity for neural time series, enabling sophisticated abnormality detection and pathology classification. On the other hand, these models face dual constraints: computational scaling laws that impose quadratic complexity with respect to sequence length, and topological heterogeneity arising from variable electrode configurations across clinical sites. The recent work by Broustail et al., titled "LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling," presents a compelling resolution to this dilemma through an elegant fusion of geometric invariant theory and state-space modeling.

Architectural Synthesis and Computational Efficiency

At the core of LuMamba lies a hybrid architecture that marries two previously distinct innovations. The LUNA framework contributes a learned-query cross-attention mechanism that projects variable electrode layouts into a unified latent space, while FEMBA provides bidirectional Mamba blocks that process temporal sequences with linear computational complexity.

This synthesis addresses a critical pain point in EEG foundation modeling. Prior approaches either sacrificed generalization by training separate models per electrode configuration, or discarded valuable data by retaining only shared channels across datasets. LuMamba's cross-attention mechanism operates as a learnable projection that maps arbitrary montages ranging from 16 to 26 electrodes into a fixed dimensional representation. This enables zero-shot transfer across topological variations without architectural modification.

The temporal backbone employs bidirectional state-space models rather than self-attention. Mamba blocks utilize input-dependent selection mechanisms that maintain content-aware processing while scaling linearly with sequence length. The bidirectional variant processes both forward and backward temporal dependencies, capturing non-stationary neural dynamics characteristic of EEG signals. This architectural choice yields substantial efficiency gains: with merely 4.6 million parameters, LuMamba requires 26 times fewer floating-point operations than LUNA and 377 times fewer than comparable transformer baselines such as LaBraM at equivalent sequence lengths. Critically, this efficiency translates to practical scalability; the model accommodates sequences 12 times longer than transformer alternatives before encountering typical GPU memory constraints.

Self-Supervised Objectives and Representation Geometry

Perhaps the most technically nuanced contribution involves the first systematic adaptation of Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA) to biosignal time series. The authors conduct a rigorous ablation study comparing three pre-training paradigms on over 21,000 hours of unlabeled EEG from the Temple University EEG Corpus (TUEG).

Their findings reveal a fascinating complementarity between reconstruction-based and predictive objectives. Masked reconstruction alone produces embeddings with pronounced clustering structure yet limited cross-montage generalization. Conversely, LeJEPA alone, which regularizes embeddings toward