Instructions to use MSALab/PerceptionDLM-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MSALab/PerceptionDLM-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="MSALab/PerceptionDLM-Base", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("MSALab/PerceptionDLM-Base", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MSALab/PerceptionDLM-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MSALab/PerceptionDLM-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MSALab/PerceptionDLM-Base",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/MSALab/PerceptionDLM-Base

SGLang

How to use MSALab/PerceptionDLM-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MSALab/PerceptionDLM-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MSALab/PerceptionDLM-Base",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MSALab/PerceptionDLM-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MSALab/PerceptionDLM-Base",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use MSALab/PerceptionDLM-Base with Docker Model Runner:
```
docker model run hf.co/MSALab/PerceptionDLM-Base
```

PerceptionDLM-Base

PerceptionDLM-Base is a strong open multimodal diffusion language model (DLM) that extends a large language diffusion backbone (LLaDA-8B) to visual instruction tuning. It establishes a new state-of-the-art baseline among open discrete-diffusion VLMs, outperforming LLaDA-V on 15 / 16 standard multimodal benchmarks while remaining competitive with same-scale autoregressive (AR) VLMs.

It serves as the foundation model for PerceptionDLM, our parallel region-perception model.

📄 Paper | 💻 Code | 🤗 Model Collection

Highlights

🧠 Diffusion-based VLM. Non-autoregressive masked-denoising generation with intrinsic token-level parallelism.
🏗️ LLaVA-style architecture. SigLIP-2 vision encoder + 2-layer MLP connector + LLaDA-8B diffusion decoder, with dynamic-resolution tiling for high-resolution inputs.
🏆 Strong baseline. Outperforms LLaDA-V on 15/16 benchmarks; especially strong on fine-grained perception and hallucination robustness.

Model Details


Vision encoder	`google/siglip2-so400m-patch16-512` (frozen)
Connector	2-layer MLP with GELU
Language backbone	LLaDA-Instruct-8B (diffusion)
Parameters	~8B
Training	4-stage visual instruction tuning, 32× H100 (~3 weeks)
Precision	bfloat16

Results

PerceptionDLM-Base vs. open diffusion / AR VLMs (selected benchmarks):

Benchmark	PerceptionDLM-Base	LLaDA-V	Qwen2.5-VL-7B	InternVL3-8B
MMBench	85.0	82.9	83.5	83.4
SeedBench	78.9	74.8	77.0	77.1
ChartQA	91.6	78.3	86.2	86.6
MMVP	82.0	76.7	73.3	80.0
BLINK	60.3	50.9	55.3	55.5
RealWorldQA	73.7	63.2	68.4	70.8
HallusionBench	58.4	50.9	51.9	49.9

See the paper for the full 16-benchmark comparison.

Usage

Full inference scripts are provided in the GitHub repository.

python demo/infer_dmllm.py \
  --model-path MSALab/PerceptionDLM-Base \
  --image assets/demo.jpg \
  --prompt "What color shirt is the man in the picture wearing?" \
  --gen-length 64 --block-length 64 --steps 64

import torch
from transformers import AutoModel, AutoProcessor

model_path = "MSALab/PerceptionDLM-Base"
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_path, torch_dtype=torch.bfloat16, trust_remote_code=True
).cuda().eval()
# See demo/infer_dmllm.py for the full preprocessing + generation pipeline.

Citation

@article{sun2026perceptiondlm,
  title   = {PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models},
  author  = {Sun, Yueyi and Wang, Yuhao and Li, Jason and Tian, Ye and Zhang, Tao and Mai, Jacky and Wang, Yihan and Wang, Haochen and Bai, Jinbin and Yang, Ling and Tong, Yunhai},
  journal = {arXiv preprint arXiv:2606.19534},
  year    = {2026}
}