PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence

PanLUNA extends LUNA's channel-unification mechanism from topology invariance to cross-modal fusion, jointly processing EEG, ECG and PPG within a single shared encoder via sensor-type embeddings -- no modality specific backbones, no paired multimodal data required during pretraining.

---

🔒 License & Usage Policy (Weights)

Weights license: The released model weights are licensed under Creative Commons Attribution–NoDerivatives 4.0 (CC BY-ND 4.0). This section summarizes the practical implications for users. This is not legal advice; please read the full license text.

✅ You may

• Use and redistribute the unmodified LUNA weights (including in commercial settings) with proper attribution to the LUNA authors.
• Fine-tune / adapt the weights for your internal use (research or production) without redistributing the modified weights.
• Publish your code, configs, logs, and papers describing experiments with LUNA (please cite the paper).

🚫 You may not

• Share, host, or redistribute any modified weights (including LoRA/adapter/delta checkpoints or pruned/quantized variants). Any parameter set that encodes an adaptation is considered a derivative and cannot be shared under CC BY-ND 4.0.
• Imply endorsement by the LUNA authors for any derivative or evaluation without our written permission.
• Use the LUNA name in a way that suggests your modified model is an official LUNA release.

🤝 How to contribute improvements (PR-gated releases)

We welcome community improvements via a pull-request (PR) workflow. If you believe your improvements should become an official LUNA release:

1. Open a PR in the BioFoundation repository describing the change (architecture/head/training recipe, datasets, preprocessing, compute).
2. Include reproducibility artifacts: configs, seeds, scripts, environment details, training/validation logs, and the evaluation protocol (e.g., TUAB/TUAR/TUSL) with exact splits.
3. Provide comprehensive results (AUROC/AUPR/BA, FLOPs, memory) vs. the baselines reported in the PanLUNA paper.
4. After maintainer review, approved changes will be retrained/validated and, if accepted, released by the maintainers as a new official PanLUNA checkpoint under CC BY-ND 4.0.

Rationale: CC BY-ND protects users from fragmented, lower-quality “PanLUNA variants,” while still enabling internal fine-tuning and a path for the community to upstream improvements through review.

---

🔎 Model Summary

• Goal: Compact pan-modal biosignal foundation model for EEG, ECG, and PPG within one shared encoder, with robustnes to missing modalities and feasibility for ultra-low-power edge deployment.
• Core idea: PanLUNA extends LUNA’s Channel-Unification Module from EEG channel/topology unification to cross-modal fusion: EEG, ECG, and PPG channels are treated as entries in a unified query set, augmented with sensor-type embeddings, and fused through cross-attention with learned latent queries inside a single encoder.
• Pre-training data: Approximately 40,000 hours of heterogeneous biosignal data from five public datasets: TUEG and Siena for EEG, MIMIC-IV and CODE-15% for ECG, and PulseDB for synchronized ECG+PPG.
• Downstream tasks: TUAB EEG abnormality detection; PTB-XL ECG Superclass (5-class), Subclass (23-class), Form (19-class), and Rhythm (12-class) classification; CSN ECG (38-class) classification; and HMC five-class sleep staging using EEG+ECG to evaluate cross-modal fusion and missing-modality robustness.

---

🚀 Model Variants

Aimed for feasible ultra-low-power edge deployment, PanLUNA exists in a Tiny Variant, with the following parameters:

Variant	Parameters	PanLUNA parameters
PanLUNA	5.4M	(num_queries = 4, embed_dim = 64)

To obtain the model of different size, consider scaling these parameters accordingly (e.g. num_queries=6 leads to 12M parameters).

---

📊 Results

• TUAB (Abnormal EEG Detection, bipolar montage, 22-channels): 81.21% Balanced Accuracy, 0.899 AUROC, 0.893 AUPR.
• HMC (EEG Sleep Staging Detection, 4-channels): 74.16% Balanced Accuracy, 0.695 Cohen's Kappa, 0.765 Weighted F1.
• ECG PTB-XL Super Class: 0.908 AUROC
• ECG PTB-XL Sub Class: 0.888 AUROC
• ECG PTB-XL Form: 0.833 AUROC
• ECG PTB-XL Rhythm: 0.964 AUROC
• ECG CSN: 0.964 AUROC

---

🧠 Intended Use & Limitations

Intended use. Research on biosignal (EEG, ECG, PPG) representation learning & classification (abnormality, sleep stages, waveform and rhythm irregularities in ECG), especially when aspiring for robustness in available channels and modalities.

Limitations.

• Not a medical device. Do not use for clinical decisions without proper validation & regulatory clearance.
• Unseen topologies: Zero-shot transfer to very different/dense layouts (e.g., SEED-V) can underperform SOTA despite positive scaling; consider augmenting pre-training montage diversity and spatial encodings.
• Distribution shifts: Performance varies across cohorts, devices, and label protocols; validate locally and consider domain adaptation.

---

🏗️ Architecture & Training

PanLUNA Tokenizer & features: Biosignals (EEG, ECG, PPG) are patch-segmented using shared convolution-based feature extractor; temporal features via 1D conv w/ GroupNorm+GELU; frequency features (FFT mag/phase → MLP) are added; 3D electrode coordinates encoded via NeRF-style sinusoids → MLP (positional enc).

PanLUNA Channel-Modality-Unification Module: Q learned queries cross-attend to channel-wise patch features from different modalities to produce a fixed Q×E latent per patch; FFN + Transformer layers refine the query tokens. Complexity is O(Q·C) (linear in channels).

Temporal encoder: Patch-wise Transformer with RoPE operates on the latent sequence (length = #patches), not on channels×patches, reducing sequence length and cost substantially.

No Modality Specific Encoders: All modalities are processed within a single shared encoder, embedded only with modality-type embedding.

Pre-training objective: Masked-patch reconstruction with Smooth-L1; decoder uses channel-indexed queries to reconstruct masked tokens. Query specialization loss encourages diverse query–channel affinities.

---

🔧 How to Use

We provide PanLUNA.safetensors, weights for the PanLUNA model pretrained on the 40,000 hours of heterogenous biosignal data.

PanLUNA experiments can be viewed through two Hydra configurations in BioFoundation/config/experiments:

• PanLUNA_finetune.yaml → configuration for fine-tuning experiments.
• PanLUNA_pretrain.yaml → configuration for pre-training experiments.

---

🔧 Fine-tuning — General Checklist

0. Install & read data prep: clone the BioFoundation repo, set up the environment as described there, then open make_datasets/README.md for dataset-specific notes (naming, expected folder layout, and common pitfalls).
1. Point to weights: set pretrained_safetensors_path: /path/to/PanLUNA.safetensors in the experiment YAML.
2. Preprocess data: acquire fine-tuning dataset and follow preprocessing protocol (see guide in /make_datasets/README.md) to generate train/test/val.h5 files.
3. Update data module of PanLUNA_finetune.yaml config:
   • Unimodal Experiments:
     • Change override /data_module to finetune_data_module_unimodal_PanLUNA.
     • Check out config/data_module/dataset_types.yaml to learn about parameters for each dataset.
     • Keep /data_module: _target_ to datasets.finetuning_unimodal_datasets_PanLUNA.FinetuningUnimodal_Dataset.
     • HDF5 file location → change /data_module:hdf5_file for train, test, and val with the path to the corresponding HDF5 data split file.
     • Change channels, location_fn and sensor_type for the intended dataset.
   • Multimodal Experiments:
     • Change override /data_module to finetune_data_module_multimodal_PanLUNA.
     • Keep /data_module: _target_ to datasets.finetuning_multimodal_datasets_PanLUNA.FinetuningMultimodal_Dataset.
     • HDF5 file location → change /data_module:hdf5_file for train, test, and val with the path to the corresponding HDF5 data split file.
     • Follow the example and instructions in the config/data_module/finetune_data_module_multimodal_PanLUNA.yaml to adjust channels and slicing for each multimodal dataset.
4. Task settings:
   • Fine-tuning strategy: override finetuning: mode with full, freeze_encoder or lora. Use full if you want to fully update PanLUNA's weights after pre-training. For training only classification head use frozen_encoder. For Low-Rank Adapation on selected layers use lora.
   • Classification type: set classification_type (bc, mcc or mlp) and model.num_classes to match your downstream task. Change model.num_classes to describe the number of features in the output.
   • Configuration file includes further #CHANGEME tags and instructions for a working example.
5. Env vars: export DATA_PATH (dataset root) and CHECKPOINT_DIR (artifacts).
6. Trainer/optimizer: adjust gpus/devices, batch_size, max_epochs, LR/scheduler if needed.
7. I/O: set io.base_output_path and confirm io.checkpoint_dirpath exists.

To launch fine-tuning (Hydra):

python -u run_train.py +experiment=PanLUNA_finetune

---

⚖️ Responsible AI, Risks & Biases

• Clinical safety: research-only; human oversight required.
• Bias & drift: montage/device/population differences can induce shifts; validate and monitor.
• Artifacts & rare events: robustness varies; use QC and task-appropriate preprocessing.

---

🔗 Sources

• Code: https://github.com/pulp-bio/BioFoundation
• Paper: PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence (arxiv:2604.04297)

---

📜 Citation

If you use PanLUNA, please cite:

@misc{zelic2026panluna,
      title={PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence}, 
      author={Marija Zelic and Anna Tegon and Yawei Li and Thorir Mar Ingolfsson},
      year={2026},
      eprint={2604.04297},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2604.04297}, 
}

---

🛠️ Maintenance & Contact

• Issues & support: please open a GitHub issue in the BioFoundation repository.

---

---

🔗 Related Models

• LUNA — Transformer-based topology-agnostic EEG foundation model (NeurIPS 2025). Source of the channel-unification cross-attention module that LuMamba reuses.
• FEMBA — Bidirectional Mamba foundation model for EEG. Source of the linear-complexity temporal backbone that LuMamba reuses.
• TinyMyo — Tiny foundation model for flexible EMG signal processing at the edge.
• LuMamba - Extends LUNA's channel-unification to a linear-complexity Mamba backbone, with systematic analysis of LeJEPA for biosignal SSL.

🗒️ Changelog

• v1.0: Initial release of PanLUNA model card with pretrained checkpoint and instructions.