🏥 ClinFusion-8B

A Vision-Centric Multimodal LLM for Holistic Medical Understanding (8B Parameter Version)

ClinFusion-8B is the highly efficient, lightweight version of the ClinFusion multimodal large language model series. Built upon the Qwen3-VL-8B-Instruct base, ClinFusion-8B integrates a composition and cascaded vision encoder framework (featuring DINOv2 and ConvNeXt) to deliver state-of-the-art medical and clinical understanding.

Despite its compact size, ClinFusion-8B strikes an optimal balance between exceptional clinical reasoning capabilities and manageable resource consumption, making it ideal for edge deployment, real-time clinical assistants, and institutions with limited computational budgets.

---

🌟 Key Features of ClinFusion-8B

• Compositional Vision Architecture: Combines DINOv2 (for dense spatial semantics) and CLIP-ConvNeXt (for structural and holistic features) with Qwen3-VL, optimized via a Cascade Spatial-Aware Locality Fusion operator.
• Native 2D & 3D Multimodal Support: Seamlessly processes standard 2D medical images (e.g., X-ray, Ultrasound, Histology) and native 3D medical volumes (NIfTI .nii.gz CT/MRI files).
• Efficient Clinical Reasoning (8B Backbone): Strikes an optimal balance between exceptional medical reasoning and efficient resource consumption, making it highly accessible and easy to deploy.
• SOTA Benchmarks: Outperforms larger open-source medical models on multiple 2D and 3D VQA benchmarks, medical report generation, and instruction-following tasks (MedIF-Bench).

---

🚀 Quick Start & Usage

To set up the environment, run inference, or evaluate ClinFusion-8B on your own data, please refer directly to our official GitHub Repository.

The GitHub repository provides comprehensive, step-by-step instructions for:

• One-click installation using uv.
• Data preparation (supporting 2D images, text-only, and 3D NIfTI volumes).
• Model inference and evaluation scripts.

---

📊 Evaluation and Performance

On our newly introduced MedIF-Bench (instruction-following evaluation) and clinical report generation benchmarks, ClinFusion-8B outperforms prominent open-source counterparts (e.g., Hulu-Med, Lingshu-8B) and exhibits competitive performance compared to proprietary medical systems.

Refer to the main ClinFusion Paper for complete evaluation metrics.

---

📄 Citation

If you find ClinFusion useful in your research, please consider citing:

@article{yuan2026clinfusion,
  title={ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding},
  author={Yuan, Hangjie and Qian, Yichen and Tang, Zhiwei and Xu, Xianzhe and Wu, Lirong and Yang, Sicheng and Wang, Jinwang and Wang, Pengju and Zeng, Zhitao and Han, Yizeng and Xing, Yan and Luo, Shengxuan and Feng, Tao and Xie, Qing and Yao, Weigen and Yang, Yi and Liu, Zuozhu and Tang, Jiasheng and Wang, Shaocheng and Wang, Jitao and Dong, Jiahong and Chen, Weihua and Xu, Feng and Wang, Fan},
  journal={arXiv preprint arXiv:2607.24743},
  year={2026}
}

---

📞 Contact & Acknowledgements

Built with ❤️ by Alibaba DAMO Academy. We acknowledge the authors of Qwen3-VL, DINOv2, and OpenCLIP for their excellent foundation models.