Brain Tumor Detection & Segmentation — SAM + YOLOv9 (SIYO)
Co-authored with Mridul Mayankeyshwar, Mamata P. Wagh, Swatishree Behuria, and Dev Yadav — C.V. Raman Global University. Published at the 2024 IEEE 1st International Conference on Advances in Signal Processing, Power, Communication, and Computing (ASPCC), IIIT Bhubaneswar, India, pp. 67–72.
94%
Detection Accuracy
0.947
mAP @ IoU 0.50
0.788
Avg AP (0.50–0.95)
The Problem
MRI-based brain tumor detection struggles to distinguish tumor tissue from healthy tissue due to noise, intensity inhomogeneity, and high variability in tumor appearance across patients. Manual radiologist interpretation is slow, subjective, and dependent on the availability of specialists. Existing automated methods often handle detection and segmentation as separate tasks — limiting how tightly the two can reinforce each other — and many lack the segmentation precision needed for accurate surgical planning or treatment response assessment.
What We Built
Our team proposed the SIYO scheme— a hybrid deep learning pipeline that tightly couples Meta's Segment Anything Model (SAM) with YOLOv9 for joint tumor detection and segmentation on MRI scans.
YOLOv9 handles the detection and localization of the tumor region, leveraging its Programmable Gradient Information (PGI) framework and the Generalized Efficient Layer Aggregation Network (GELAN) architecture to extract rich, multi-scale features while preserving gradient signal through deep layers. Once YOLOv9 localizes the tumor region, SAM refines the segmentation mask — using its promptable architecture to delineate tumor boundaries precisely from the bounding box prompts.
The model was trained on the Br35H Brain Tumor Detection dataset (800 MRI images: 500 train / 200 validation / 100 test), with data augmentation applied to the training set to improve generalization. Training ran in Google Colab for 100 epochs over approximately 2.13 hours.
Tech Deep Dive
Programmable Gradient Information (PGI)
Deep networks lose semantic information as gradients flow backward through many layers — a problem that makes it hard to learn reliable object representations at low layers. PGI addresses this by introducing an auxiliary supervision branch that provides complete gradient signals directly to earlier layers, bypassing the information bottleneck. This means the network learns better representations at every depth, not just near the output.
Generalized Efficient Layer Aggregation Network (GELAN)
GELAN is a lightweight, flexible backbone that extends the earlier ELAN architecture, allowing arbitrary computational blocks to be substituted into a consistent aggregation design. The result is a network that achieves strong accuracy while remaining faster and more parameter-efficient than comparably-performing architectures — critical for practical clinical deployment.
Segment Anything Model (SAM)
Meta's SAM is a foundation model for promptable segmentation. Given a bounding box, point, or mask prompt, it produces high-quality segmentation masks without task-specific fine-tuning. In SIYO, YOLOv9's detection output acts as the spatial prompt, letting SAM delineate the tumor boundary with precision that a pure detection model cannot achieve.
What I Learned
Working on SIYO taught me how to combine two architecturally different models — a real-time detector and a promptable foundation model — into a coherent pipeline. Medical imaging introduces constraints that generic computer vision datasets don't: class imbalance between tumor and healthy regions, sensitivity to annotation quality, and the need to handle the high cost of false negatives. Collaborating across a multi-author research paper pushed me to write technically precise prose for peer review, handle version-controlled experiments reproducibly, and present quantitative results clearly for a mixed audience of ML researchers and medical professionals. Getting published at IEEE ASPCC 2024 validated that rigorous evaluation methodology matters as much as model architecture.