Layer-wise Representation Analysis and Information Fusion in EEG Foundation Models
Télécom Paris, IPP. February 2026 - June 2026.
Abstract
EEG Foundation Models (EFMs) learn general-purpose brain representations from large EEG corpora and are typically reused for downstream tasks by reading out only their final transformer layer. We test whether that default is justified. Across five EEG datasets and two backbones, CBraMod [1] and CSBrain [2], both 12-layer transformers, we probe every layer independently and compare against three multi-layer fusion strategies (average, concatenation, gating). My part of the project focused on FACED, a 9-class emotion-recognition dataset. On FACED, the best intermediate layer beat the final layer by +5.7 (CBraMod, layer 8) to +10.6 F1-macro points (CSBrain, layer 5), the largest final-layer gap of any dataset in the study.
1. Motivation
CBraMod and CSBrain both stack 12 transformer blocks and, following common practice, are evaluated by attaching a classification head to the last one. This assumes the final layer is where task-relevant information concentrates, an assumption rarely tested directly. We formed three hypotheses:
- H1: task-relevant information is not distributed uniformly across the 12 layers; the most discriminative depth depends on the task and the architecture.
- H2: the final layer is not the optimal readout point; intermediate layers can match or exceed it.
- H3: fusing multiple layers (average, concatenation, gating) captures complementary information the single-layer readout misses.
2. Method
A three-phase pipeline was applied to every dataset/model pair:
- Embedding extraction: PyTorch forward hooks capture the output of all 12 transformer layers.
- Layer-wise probing: a lightweight classification head is trained independently on each frozen layer's embeddings.
- Multi-layer fusion: the 12 layers are combined via average (ê = mean of all layers, a static baseline), concatenation (ê = all layers stacked, brute-force), and gating (ê = input-dependent weighted sum, learned per sample).
Multi-class tasks (FACED's 9 emotion classes) were scored with F1-macro; binary tasks used ROC-AUC.
3. Dataset: FACED
FACED is a 9-class emotion-recognition EEG dataset: the highest class count and, on this benchmark, the hardest task in the study. It was my dataset within the group project.
| Model | Best layer (F1-macro) | Last layer L11 | Δ | Best fusion |
|---|---|---|---|---|
| CBraMod | L8: 0.438 | 0.381 | +0.057 | Gating: 0.428 |
| CSBrain | L5: 0.402 | 0.296 | +0.106 | Gating: 0.384 |
Both backbones are clearly better read from a mid-network layer than from their last one on this task, CSBrain especially, where the final layer loses over 10 F1 points relative to its own best layer. Gating fusion recovers most, but not all, of that gap: it improves substantially on the final-layer baseline but still trails the single best layer in both models. This matches the pattern found across the other four datasets studied by the team (SHU-MI, PhysioNet-MI, MentalArithmetic, Mumtaz-2016): the final layer under-performs everywhere, but its readout gap on FACED was the largest observed, and simple averaging of layers was consistently the weakest fusion strategy, on FACED and elsewhere.
4. Discussion
The results support H1 and H2: informative depth is not fixed, and reading out only the last layer discards representations that a lightweight probe can exploit better. On FACED, CBraMod's best layer (L8) sits later than CSBrain's (L5), consistent with a broader pattern across the five datasets, where CBraMod tends to peak mid-to-late and CSBrain earlier. For H3, gating fusion is an input-dependent, learned improvement over the final-layer default, and its per-sample weights are interpretable (e.g. which layers a given trial relies on), but it did not surpass the single best-probed layer on FACED, suggesting that for this task, locating the right depth mattered more than combining several.
5. Conclusion
Treating the final transformer layer as the default readout point is not supported on FACED, or on any of the four other EEG tasks evaluated by the team. Systematically probing every layer is cheap relative to pretraining and can recover several points of task performance that the default discards. Next steps include a deeper per-layer analysis of what FACED's best layers encode (e.g. correlating activations with classical EEG biomarkers) and testing whether gating weights transfer across subjects.
Team: M. Bajjou, C. Hosken, A. Munier, T. Revol, O. Silva, supervised by Tuan-Kiet Doan
(tuan.doan@ip-paris.fr).
[1] Wang, Jiquan, et al. "CBraMod: A criss-cross brain foundation model for EEG decoding." arXiv:2412.07236 (2024).
[2] Zhou, Yuchen, et al. "CSBrain: A cross-scale spatiotemporal brain foundation model for EEG decoding." arXiv:2506.23075 (2025).
[3] Nguyen et al. "ECG-Soup." arXiv:2509.00102 (2025).
Code: github.com/HubGeth/eeg-representation-learning