Any to Full: Prompting Depth Anything for Depth Completion in One Stage

Zhiyuan Zhou¹ · Ruofeng Liu² · Taichi Liu¹ · Weijian Zuo³ · Shanshan Wang¹ · Zhiqing Hong⁴ · Desheng Zhang^1
1Rutgers Univ.   ²Michigan State Univ.   ³JD Logistics   ⁴HKUST (GZ)

---

News

• [2026.06] 🎉 Any2Full has been accepted to ECCV 2026!
• Training code and details are coming soon. Thank you for your interest and support.

---

Overview

Accurate dense depth is essential for robotics, but commodity RGBD sensors are often sparse or incomplete. Any2Full is a one-stage, domain-general, and pattern-agnostic depth completion framework. It reformulates completion as scale-prompting adaptation of a pretrained monocular depth estimation (MDE) model, so the model keeps strong geometric priors while adapting to diverse sparse depth patterns.

Highlights

• One-stage scale prompting: achieves domain-general depth completion by fusing pretrained MDE priors.
• Scale-Aware Prompt Encoder: strong robustness under different sparsity levels and sampling patterns.
• lightweight design: efficient inference with a single forward pass.

---

Requirements (Minimal for Inference)

• python==3.9.x
• torch==2.1.0
• torchvision==0.16.0
• numpy<2
• pillow
• matplotlib
• scipy
• opencv-python-headless==4.11.0.86

pip install torch==2.1.0 torchvision==0.16.0 \
  "numpy<2" pillow matplotlib scipy opencv-python-headless==4.11.0.86

Optional (CUDA 12.1) xFormers acceleration:

pip install xformers==0.0.25.post1 --index-url https://download.pytorch.org/whl/cu121

Notes:

• If xFormers is not installed, inference still runs normally; only memory/performance optimization is disabled.

---

Model Usage

1) Quick Inference (Single or Batch RGBD)

Use run_any2full.py for single RGBD pairs or batch folders (matched by filename stem).

Example inputs are provided under assets/: assets/rgb and assets/depth can be used as inputs, and assets/output shows the corresponding outputs.

# Single pair
python run_any2full.py \
  --rgb /path/to/rgb.png \
  --depth /path/to/depth.png \
  --checkpoint /path/to/Any2Full_vitl.pth.tar \
  --out_dir ./outputs

# Batch (match by basename)
python run_any2full.py \
  --rgb_dir /path/to/rgb_dir \
  --depth_dir /path/to/depth_dir \
  --checkpoint /path/to/Any2Full_vitl.pth.tar \
  --out_dir ./outputs

Optional denoise (from utils/denoise.py): Any2Full relies on accurate sparse depth as an anchor, so cleaner raw depth generally yields better results. We provide a simple denoising pre-processing step for convenience.

python run_any2full.py \
  --rgb /path/to/rgb.png \
  --depth /path/to/depth.png \
  --checkpoint /path/to/Any2Full_vitl.pth.tar \
  --out_dir ./outputs \
  --denoise \
  --denoise_threshold 2 \
  --denoise_kernel_size 9

---

Inference Parameters (Detailed)

• --rgb: RGB image path (single mode).
• --depth: Sparse depth path (.png or .npy) (single mode).
• --rgb_dir: RGB directory (batch mode, filename stem matched).
• --depth_dir: Depth directory (batch mode, filename stem matched).
• --checkpoint: Any2Full checkpoint path (required).
• --da_ckpt_path: Optional backbone MDE checkpoint (for encoder init).
• --encoder: Backbone variant (vits, vitb, vitl).
• --depth_scale: Scale factor for depth PNGs (depth = img / scale).
• --denoise: Enable sparse depth outlier removal before inference.
• --denoise_threshold: Outlier threshold (std multiplier).
• --denoise_kernel_size: Neighborhood size (odd int) for denoise; None auto-estimates.
• --denoise_min_valid: Minimum valid neighbors for denoise.

Model Weights

• Any2Full weights: https://huggingface.co/zhiyuandaily/Any2Full/tree/main/checkpoints

---

Citation

If you find our work useful, please consider citing:

@article{zhou2026any2full,
  title={Any to Full: Prompting Depth Anything for Depth Completion in One Stage},
  author={Zhou, Zhiyuan and Liu, Ruofeng and Liu, Taichi and Zuo, Weijian and Wang, Shanshan and Hong, Zhiqing and Zhang, Desheng},
  journal={arXiv:2603.05711},
  year={2026}
}