Sensor Foundation Model Scales Across Wearable Health Tasks

Wearable sensors produce long, noisy records of physiology and behavior, but the labels needed to turn those signals into clinical or behavioral endpoints are scarce. This paper argues that the constraint should be attacked with scale: pretrain a general embedding model on unlabeled wearable streams, then adapt that representation to many smaller labeled studies. The result is SensorFM, a foundation model trained on more than one trillion minutes of Fitbit and Pixel Watch sensor data from five million participants and evaluated on 13,985 independent downstream participants.

Core Contribution

The main claim is not that a wearable model solves diagnosis. It is that a single reconstruction-trained representation can carry enough information about daily physiology to support many later tasks without endpoint-specific architectures. The evaluation spans 35 discriminative tasks across demographics, lifestyle, cardiovascular risk, metabolic status, mental health, and sleep. That breadth matters because many wearable models are validated on one endpoint, one cohort, or one engineered feature set; here the paper tests whether scale produces a reusable substrate.

Figure 1 summarizes the design: population-scale pretraining, a bottlenecked reconstruction objective over five sensor modalities, downstream predictive tasks, generative infilling tasks, and a scaling plot linking pretraining performance to downstream performance.

Technical Approach

SensorFM is trained with a generative reconstruction objective rather than supervised health labels. The model receives one-minute aggregate features derived from wearable modalities and learns through a latent bottleneck, forcing the representation to retain information useful for reconstructing missing or held-out signal structure. The paper then post-trains or probes those embeddings on downstream tasks using reduced embeddings; several tables report PCA-50 reduced SensorFM embeddings with five-fold cross-validation.

The paper also introduces an agentic adaptation workflow. A “classroom” of LLM agents searches over downstream predictive heads built on SensorFM embeddings, rather than relying only on a fixed linear probe. The reported example solution uses demographic interactions, standard scaling, PCA over embeddings, and a random forest classifier. This is a pragmatic contribution: the LLM is not treated as the health model, but as a search tool for constructing better heads over a pretrained sensor representation.

A second interface experiment uses SensorFM-derived predictions as context for a Personal Health Agent. Figure 3 shows the pipeline from raw wearable data to SensorFM predictions to an LLM-generated health summary, then clinician grading across harm, context, personalization, justifiability, and relevance.

Results and Analysis

The discriminative results support the scaling claim. The paper evaluates model variants from XXS with 10⁵ parameters through B with 10⁸ parameters, co-scaling data volume and model capacity. In the extended tables, the B model is generally strongest: Table ED.7 shows lower regression error or higher F1 across most tasks, and Table ED.9 reports that SensorFM, with or without demographics, achieves the best performance on 31 of 35 tasks against supervised baselines using engineered features. Concrete examples include age prediction at r=0.920, BMI at r=0.809, current working status at ROC AUC 0.912, smoking at 0.867, hypertension diagnosis at 0.786, and mental-health-medication use at 0.826.

The generative tests are also favorable. For 80% random imputation, SensorFM-B reports MSE 0.215, compared with 0.915 for mean fill, 1.020 for nearest-neighbor fill, and 0.854 for linear interpolation. In signal imputation, SensorFM-B reports MSE 0.122 when 2 of 26 signals are missing, compared with about 1.016 for the simple baselines. The recovered daily summaries under simulated one-hour data loss are close to ground truth: steps are 6208.41 versus 6227.49, light sleep is 25.43 minutes versus 25.64, and high SpO2 minutes are 233.40 versus 233.28. These results suggest the representation is learning cross-signal and temporal structure, not only static population correlations.

The Personal Health Agent evidence is narrower but relevant. The paper reports 31 real patient profiles and 1,860 clinician ratings. The text states that adding SensorFM predictions improves responses over baselines without that context, while model-derived diagnostic inferences compared with clinical ground truth show no statistically significant difference in interaction quality in the static evaluation. The cautious reading is that SensorFM can improve grounding for a health-summary interface, but the experiment does not establish safe real-world conversational use.

Limitations

The paper is direct about several constraints. The pretraining and evaluation data come from Fitbit and Pixel Watch users, so transfer to other devices is not guaranteed. The one-minute aggregate input discards waveform-level information such as beat-to-beat variability and sub-second motion. Many downstream labels are self-reported or screening-derived rather than clinically verified. The agent evaluation is single-turn and static, with clinicians restricted to the provided context, which limits claims about deployment in real consultations.