Why Tokenization, Not the Objective, May Be the Real Bottleneck in Wearable SSL

For the past several years, the wearable sensing community has treated self-supervised learning as a contest of objectives. Should we use contrastive learning, masked reconstruction, or augmentation prediction? Each year brings a new variation, a new loss function, a new claim of state-of-the-art transfer. A recent paper, Bio-Inspired Self-Supervised Learning for Wrist-worn IMU Signals by Tarale and colleagues, makes an argument I find more interesting than yet another loss function: the problem is not which objective we optimize, but what we optimize over. The bottleneck, they contend, is tokenization.

I want to unpack why this claim is more than a clever framing, what evidence the paper offers, and where I think the idea runs into limits.

The problem with treating IMU streams as arbitrary time series

Wrist-worn accelerometers produce a continuous stream of three-axis acceleration values sampled at some fixed rate. The dominant approach in wearable SSL is to slide a fixed-length window across this stream, then apply some pretext task: predict which augmentation was applied, reconstruct masked samples, or align contrastive views.

The hidden assumption here is that the sample grid is the natural unit of analysis. But there is nothing biologically or physically meaningful about a sample at index 137 versus index 138. The grid is an artifact of the analog-to-digital converter, not of the movement itself. When you ask a model to reconstruct masked samples on this grid, it can often succeed through local interpolation and sensor statistics, exactly the shortcut the authors flag in their discussion of masked reconstruction. The model becomes very good at predicting the next wiggle in a waveform, and very poor at understanding that a sequence of wiggles constitutes reaching for a cup.

This is the core insight. Capacity spent memorizing local waveform morphology is capacity not spent on temporal structure. And it is the temporal structure, the way movement units compose into activities, that we actually want to transfer to downstream Human Activity Recognition.

The submovement theory as a tokenizer

The paper's solution is grounded in the submovement theory of motor control (Rohrer et al., 2004). This theory holds that continuous wrist motion is not arbitrary but is built from the superposition of elementary, stereotyped units called submovements. A reach, a grasp, a turn of the wrist: each decomposes into a small number of these basis functions.

Tarale and colleagues build their token, which they call the movement segment, on this foundation. A movement segment is a sub-second unit composed of a finite sequence of submovements, and crucially, it is extractable directly from the accelerometer signal without labels. The analogy they draw is explicit and worth taking seriously:

• Submovements are like characters or phonemes, the irreducible primitives.
• Movement segments are like words, meaningful composite units.
• Activities are like sentences, structured arrangements of words.

Once you have this tokenization, you can do what language modeling does: pretrain a Transformer encoder via masked movement-segment reconstruction. Instead of imputing missing samples on an arbitrary grid, the model imputes missing movement units from their context. This forces it to learn how segments relate to one another over time, which is precisely the compositional structure that should transfer.

The resulting model is called Bio-PM (Bio-inspired Tokenization-based Pretrained Model). It is pretrained on the NHANES corpus, roughly 28,000 hours from 11,000 participants and about 10 million windows. The scale matters here because the central claim is about representation quality under transfer, and you need a large, diverse pretraining corpus to make that argument credibly.

What the evidence shows

The experimental design is more careful than the headline numbers. The authors pretrain all SSL baselines on the same NHANES corpus and evaluate everything under the same subject-disjoint transfer protocol across six public HAR benchmarks. This is the right way to isolate the contribution of tokenization. If you let baselines pretrain on different data or evaluate on overlapping subjects, you cannot attribute gains to the tokenization strategy rather than to data or leakage.

The reported result is an average 6 percent improvement in macro-F1, with a range of 4 to 12 percent, over the controlled baselines. More compelling to me than the average improvement is the claim of stronger data efficiency in low-label regimes. This is exactly what you would predict if the hypothesis is correct. A model that has learned compositional movement structure during pretraining should need fewer labeled examples to map that structure onto activity classes, because the hard representational work is already done. A model that learned only local waveform shapes has to learn the composition from scratch using the scarce labels.

This data-efficiency result is, in my view, the most important piece of evidence. Average macro-F1 gains can come from many sources. But a consistent advantage that grows as labels become scarce is the signature of genuinely better representations rather than better fitting.

My take: the language-modeling lesson is real but incomplete

I am sympathetic to the central argument, and I think it connects to a broader and underappreciated lesson. In natural language processing, the move from character-level to subword tokenization (BPE, WordPiece, SentencePiece) was not a minor engineering detail. It materially shaped what models could learn efficiently. The wearable community has, by contrast, largely defaulted to the raw sample grid because it requires no domain commitment. Bio-PM is making a domain commitment, and that is the source of its leverage.

That said, I want to push on a few points the paper's framing glosses over.

First, the tokenizer is itself a model with assumptions. Submovement decomposition is not a settled, unique operation. Different extraction procedures will yield different segment boundaries, and the choice of how to chunk continuous motion into segments is a hyperparameter that carries inductive bias. If the segmentation is wrong or noisy, you have simply moved the arbitrariness from the sample grid to the segmentation algorithm. The paper would be stronger with an ablation that varies the tokenizer and shows the gains are robust to reasonable variations rather than tuned to one specific extraction.

Second, the word analogy may be too clean. Submovements superpose. They overlap in time, blending into one another in a way words in a sentence do not. A genuinely faithful tokenizer might need to represent overlapping, continuous units rather than discrete, non-overlapping segments. The discreteness that makes the language analogy attractive is also where the analogy is most likely to leak.

Third, there is a generalization question across body locations and sensing modalities. Submovement theory is well developed for wrist and upper-limb motion. It is less obvious how this transfers to hip-worn or ankle-worn devices, or to gyroscope and magnetometer channels. The wrist may be a particularly favorable case precisely because the motor-control literature is richest there.

Closing thoughts

What I appreciate about this work is that it reframes a stagnant debate. The field has been iterating on objectives while holding the input representation fixed. Bio-Inspired Self-Supervised Learning for Wrist-worn IMU Signals asks whether the fixed input representation was the constraint all along, and provides a controlled comparison suggesting it might be.

The open questions I would want answered next are: How sensitive are the gains to the specific submovement extraction procedure? Can the tokenizer be learned end-to-end rather than imposed, so that the model discovers its own movement vocabulary? And does the compositional structure learned over movement segments support genuinely generative or predictive tasks, the way language tokens support generation, or only discriminative transfer?

If the answer to that last question is yes, then we may be looking at the early outline of a foundation-model recipe for human motion. If tokens really do matter more than we admit, the most valuable work in wearable SSL over the next few years may be in defining them well, not in inventing the next loss.