SAM-MT: Real-Time Interactive Multi-Target Video Segmentation

Ruiqi Shen¹ · Chang Liu^2✉️ · Henghui Ding^1✉️

¹Fudan University    ²Shanghai University of Finance and Economics  

SAM-MT is an efficient interactive multi-target video segmentation framework that maintains near-single-object efficiency (FPS and VRAM) as target count increases, while maintaining robust video segmentation performance.

✨ Highlights

• Real-time speed: 36+ FPS with 10 targets on a single NVIDIA RTX A6000 GPU.
• Individual-global representation: Models individual targets and global scene within a unified framework.
• Interactive multi-target video segmentation: Simple clicks for target specification.

📋 TODO

• ✅ Release checkpoint of SAM-MT.
• ✅ Release inference code and interactive demo.
• ⬜ Enhance negative-click refinement.
• ⬜ Release training code.

🧠 Checkpoint

We provide the official SAM-MT checkpoint for real-time interactive multi-target video segmentation.

Model	Checkpoint
SAM-MT

By default, place the checkpoint under the checkpoints/ directory.

⚙️ Installation

# clone the repo and enter directory
git clone https://github.com/FudanCVL/SAM-MT.git
cd SAM-MT

# create and activate conda environment
conda create -n sammt python=3.10 -y
conda activate sammt

# install required packages
pip install -r requirements.txt

🧪 Evaluation

The evaluation-related files are organized as follows:

SAM-MT/
├── demo/
│   ├── points/
│   └── synthetic_benchmark/
└── evaluation/
    ├── evaluate_efficiency.py # Efficiency evaluation (FPS & VRAM)
    ├── evaluate_mose.py       # MOSE evaluation
    └── evaluate_lvos.py       # LVOS evaluation (supports targets appearing later)

🚀 Inference

We provide two inference scripts for SAM-MT:

# Basic inference (coordinates required)
python inference.py

# Interactive Gradio demo
python inference_gradio.py

For quick exploration, we recommend the Gradio demo, where users can directly click on the targets and try the model:

📚 Acknowledgements & Citation

We are inspired by the excellent work of SAM2, and many other not listed.

If you find SAM-MT useful in your research, please consider citing:

@inproceedings{SAM-MT,
  title={{SAM-MT}: Real-Time Interactive Multi-Target Video Segmentation},
  author={Shen, Ruiqi and Liu, Chang and Ding, Henghui},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}