Access Embedl SAM 3D Body (Quantized)

To access this model, please review and accept the terms below. Your contact information is collected solely to manage access and, with your explicit consent, to notify you about updated or new optimized models from Embedl. You can withdraw consent at any time by contacting us (see Contact section below). See our license for full terms.

By requesting access you agree to the Embedl Models Community Licence and the upstream SAM 3D Body License. You must also have access to the gated upstream model facebook/sam-3d-body-dinov3.

Embedl SAM 3D Body (Quantized)

Deployable version of facebook/sam-3d-body-dinov3, Meta's single-image full-body human-mesh recovery model. The repository ships both stages of the pipeline as ONNX models for low-latency NVIDIA TensorRT inference:

Encoder — the 840.6 M-parameter DINOv3 ViT-H+/16 vision backbone (the dominant compute cost), mixed-precision INT8/FP16-quantized with hardware-aware optimizations from embedl-deploy.
Decoder — the SAM-3D-Body promptable decoder + MHR parametric mesh head (body-only inference path), exported to ONNX; it maps backbone features directly to the 3D mesh vertices and camera.

Image → 3D body mesh runs end-to-end in TensorRT in ~42 ms on an NVIDIA L4 (see Performance).

3D Human Mesh Recovery

Below is our INT8-quantized backbone applied to the Gothenburg Poseidon (Carl Milles): input → mesh overlay → ¾ view → side view. The INT8 mesh stays within ~0.5 % (mean ≈ 10 mm) of the FP32 result. Reproduce it with demo3d_trt.py (all-TensorRT) or demo_3d.py (PyTorch pipeline).

Rotating recovered mesh

Backbone features (PCA)

Nvidia L4

Open facebook/sam-3d-body-dinov3 in hfviewer

Highlights

End-to-end TensorRT: image → mesh with both models as TRT engines, 42 ms engine time on an L4 (encoder 26 ms + decoder 16 ms), no PyTorch and no upstream checkpoint needed at inference time.
Formats: encoder as ONNX with external weights (embedl_sam3dbody_int8.onnx + .onnx.data) and as a torch.export graph (embedl_sam3dbody_int8.pt2); decoder as ONNX (sam3dbody_decoder.onnx).
Precision: encoder INT8 with sensitive layers (the 3-channel patch-embed conv and LayerNorms) kept in FP16; decoder FP16 with the MHR mesh-rig math kept in FP32. Both ONNX files ship in fp32 — the demo casts them at engine-build time.
Input: a single (1, 3, 512, 512) ImageNet-normalized person crop → (1, 1280, 32, 32) feature map → mesh (18 439 vertices) + camera.
5.1× faster and 3.3× smaller encoder than FP32 on an NVIDIA L4; recovered 3D mesh within ~0.5 % of the FP32 result.

Quick Start — end-to-end TensorRT demo

demo3d_trt.py recovers and renders the 3D body from a photo, running both models as TensorRT engines. It only needs numpy/OpenCV-class dependencies — no PyTorch, no upstream repo, no upstream checkpoint.

hf download embedl/sam-3d-body \
  embedl_sam3dbody_int8.onnx embedl_sam3dbody_int8.onnx.data \
  sam3dbody_decoder.onnx sam3dbody_faces.npy \
  demo3d_trt.py sample_input.png --local-dir .

pip install tensorrt pycuda onnx numpy opencv-python-headless pillow matplotlib imageio
# on CUDA-12 drivers install `tensorrt-cu12` instead of `tensorrt` (see Performance notes)

python demo3d_trt.py --image sample_input.png --bbox 180 210 700 950
# add --bbox x1 y1 x2 y2 to crop one person out of a wider scene (defaults to the full image)

The first run casts both ONNX models to FP16 (the decoder's mesh-rig section stays FP32 for accuracy) and builds the engines (a few minutes, cached next to the ONNX files). Afterwards:

recovered mesh: 18439 vertices (TensorRT end to end)
inference latency (engine execution only):
  encoder  mean  26.56 ms  p95  27.63 ms
  decoder  mean  15.85 ms  p95  15.96 ms
  total    mean  42.41 ms
wrote mesh_demo_trt.png and mesh_demo_trt_spin.gif

The decoder engine runs the SAM-3D-Body body inference path (equivalent to upstream inference_type="body"): full-body mesh without the optional hand-refinement passes.

Backbone only

infer_trt.py (TensorRT) and infer_pt2.py (torch.export) run just the encoder on an image, report feature statistics + latency, and save a PCA visualization of the patch features — the classic DINOv3 "what the backbone sees" image:

pip install pillow numpy onnx           # + tensorrt/pycuda (trt) or torch (pt2)
python infer_trt.py --image sample_input.png --save-pca features_pca.png
python infer_pt2.py --image sample_input.png --save-pca features_pca.png

PyTorch reference pipeline (`demo_3d.py`)

demo_3d.py runs the upstream eager-PyTorch pipeline with this INT8 backbone swapped in — useful as a reference, or when you need the full inference mode with hand refinement. It requires the upstream code and gated checkpoint:

pip install torch matplotlib pillow numpy imageio opencv-python huggingface_hub

# upstream SAM-3D-Body pipeline (not a pip package - use via PYTHONPATH)
git clone https://github.com/facebookresearch/sam-3d-body
# its runtime deps (see sam-3d-body/INSTALL.md):
pip install pytorch-lightning yacs scikit-image einops timm dill hydra-core hydra-colorlog \
  pyrootutils roma loguru optree fvcore trimesh braceexpand webdataset "networkx==3.2.1" \
  chump jsonlines joblib pandas rich smplx torchvision

# gated upstream checkpoint (accept the licence at facebook/sam-3d-body-dinov3 first)
hf download facebook/sam-3d-body-dinov3 model.ckpt model_config.yaml assets/mhr_model.pt --local-dir sam3d_ckpt
hf download embedl/sam-3d-body embedl_sam3dbody_int8.pt2 demo_3d.py sample_input.png --local-dir .

PYTHONPATH=sam-3d-body python demo_3d.py \
  --image sample_input.png \
  --ckpt-dir sam3d_ckpt \
  --pt2 embedl_sam3dbody_int8.pt2 \
  --out mesh_demo.png

Files

File	Description
`embedl_sam3dbody_int8.onnx`	Encoder — quantized ONNX with precalibrated Q/DQ operations
`embedl_sam3dbody_int8.onnx.data`	Encoder external weights (~3.4 GB)
`embedl_sam3dbody_int8.pt2`	Encoder as INT8 `torch.export` ExportedProgram
`sam3dbody_decoder.onnx`	Decoder — SAM-3D-Body promptable decoder + MHR mesh head (body path), features → mesh (fp32; cast to fp16 at engine build)
`sam3dbody_faces.npy`	Mesh triangle indices (36 874 × 3), used for rendering
`demo3d_trt.py`	End-to-end TensorRT demo — image → mesh with both engines, reports engine-only latency
`infer_trt.py`	TensorRT encoder inference + latency + PCA-feature demo
`infer_pt2.py`	`torch.export` encoder inference + PCA-feature demo
`demo_3d.py`	PyTorch reference pipeline demo (upstream code + checkpoint required)
`sample_input.png`	Example person crop (from the SAM-3D-Body qualitative gallery)

Performance

End-to-end pipeline comparison (NVIDIA L4, TensorRT 11.1)

Environment: NVIDIA L4 · driver 595.71 / CUDA 13.2 · TensorRT 11.1 (tensorrt cu13 build) · batch 1, 512×512 input · engine execution time only (no host↔device copies), 10-iteration warmup, 50 timed iterations, all rows measured in a single run. The FP32/FP16 baselines use the same encoder with Q/DQ stripped (the underlying fp32 weights); "FP32" is TensorRT's default mode for fp32 ONNX (TF32 enabled).

Pipeline (image → mesh)	Encoder	Decoder	End-to-end	Throughput	Engine size	Speedup
FP32 (TRT default, TF32)	144.2 ms	25.7 ms	169.9 ms	5.9 fps	4105 MiB	1.0×
FP16	45.0 ms	16.2 ms	61.2 ms	16.3 fps	2063 MiB	2.8×
Embedl INT8+FP16 (this model)	27.2 ms	16.3 ms	43.4 ms	23.0 fps	1425 MiB	3.9×

With strict IEEE FP32 (TF32 disabled) the baseline is 269.3 ms end-to-end (encoder 241.7 ms), making the quantized pipeline 6.3× faster. Run-to-run variance is ~±2 ms. The standalone encoder benchmark (infer_trt.py, 2 s warmup, 300 iterations) measures 26.2 ms mean / 26.8 ms p95 / 38.2 qps on the same stack.

Encoder precision ladder (NVIDIA L4, TensorRT 11.1)

Same environment and timing method as above. Every row is the identical Q/DQ-stripped fp32 encoder rebuilt at a different precision (Q/DQ kept for the INT8 row); "strict FP32" has TF32 disabled, "TF32" is TensorRT's default mode for fp32 ONNX.

Encoder precision	Mean latency	p95	Speedup	What changes
FP32 (strict, TF32 off)	241.7 ms	244.8 ms	1.0×	CUDA cores, 3.4 GB weights
TF32 (TRT default "fp32")	138.6 ms	140.2 ms	1.7×	GEMMs on tensor cores, weights still fp32
FP16	43.0 ms	45.1 ms	5.6×	faster tensor cores + half the weight traffic
Embedl Deploy INT8+FP16 (this model)	27.2 ms	28.1 ms	8.9×	quantized GEMMs, ~0.9 GB weights

At batch 1 this model is substantially weight-bandwidth-bound, which is why TF32 (compute-only change) gains less than FP16/INT8 (which also shrink the weights). INT8 buys 1.6× over the best non-quantized deployment (FP16) with the mesh-accuracy trade-off quantified under Fidelity.

Encoder precision comparison (NVIDIA L4, TensorRT 10.16, historical)

Environment: NVIDIA L4 · driver 570.211 / CUDA 12.8 · TensorRT 10.16 (tensorrt-cu12 build) · single-image (1, 3, 512, 512) input, GPU compute time only (CUDA events, 2 s warmup, 300-iteration timing).

Configuration	Mean latency	p95	Throughput	Engine size	Speedup
FP32 (TensorRT)	145.1 ms	147.8 ms	6.9 qps	3210 MiB	1.00×
FP16 (TensorRT)	44.7 ms	45.7 ms	22.4 qps	1606 MiB	3.24×
Embedl Deploy INT8+FP16 (this model)	28.3 ms	29.0 ms	35.3 qps	968 MiB	5.12×

Version notes:

TensorRT 11.x (CUDA 13): TRT 11 removed the weakly-typed FP16/INT8 builder flags — engines must be built strongly typed, which requires casting the fp32-declared ONNX to FP16 first (otherwise all non-quantized layers run in FP32, ~3.6× slower). infer_trt.py and demo3d_trt.py handle this cast automatically for both models and reach the numbers above.

TensorRT 10.x (CUDA 12): use TensorRT ≥ 10.16 with the CUDA 12 build (pip install tensorrt-cu12). The INT8 path is highly sensitive to the TRT version's kernel selection on Ada (sm_89): TensorRT 10.16 reaches 28.3 ms, whereas TensorRT 10.1 builds the same engine at ~73 ms and with slightly degraded features.

Match the wheel to your driver: a CUDA 13 build fails with cudaErrorInsufficientDriver on CUDA 12.x drivers — install tensorrt-cu12 there instead.

Fidelity

Because the backbone feeds a 3D-mesh pipeline, the metric that matters is the recovered mesh, not raw features. Running the full SAM-3D-Body pipeline with the INT8 TensorRT engine vs. the FP32 reference on a real person image:

Configuration	Recovered-mesh deviation vs FP32	Backbone-feature rel. diff
FP16 (TensorRT)	negligible	0.36 %
Embedl Deploy INT8+FP16 (this model)	mean ≈ 10 mm (~0.5 %), max ≈ 67 mm	(calibration-dependent)

The decoder + MHR head are robust to the backbone's INT8 quantization noise, so the 3D body is preserved (the patch-embed conv and LayerNorms are kept in FP16). Mesh deviation is on a ~1.8 m body. Re-calibrating on a larger, more diverse set of person crops tightens this further.

The decoder ONNX itself is numerically faithful: as an FP32 TensorRT engine it matches the upstream PyTorch decoder to ≤ 1e-4 relative on all outputs, and the default FP16 engine (mesh-rig math kept in FP32) adds only ~2 mm mean vertex deviation while running 1.7× faster. End-to-end TensorRT mesh deviation stays dominated by the encoder INT8 kernels (mean ≈ 9 mm vs. the quantized PyTorch pipeline on the shipped sample).

Quantization sensitivity is pose-dependent: comparing the full quantized pipeline against the full FP32 pipeline (Q/DQ-stripped encoder + fp32 decoder) gives mean ≈ 10 mm on a standing figure and mean ≈ 28 mm / max ≈ 63 mm on the shipped sample's unusual lying pose.

Deployment boundary

The encoder is the natural torch.export-friendly deployment boundary and is what embedl-deploy quantizes; the decoder + MHR head (~5 % of the compute) are shipped as a separate FP32 ONNX export covering the body inference path. Feed (1, 3, 512, 512) normalized crops to the encoder and its (1, 1280, 32, 32) features to the decoder (plus the camera/bbox conditioning inputs — see demo3d_trt.py for the exact preprocessing) to get mesh vertices, 3D/2D keypoints and camera translation.

You must have accepted the upstream gated license at facebook/sam-3d-body-dinov3 to use this derivative.

Creating Your Own Optimized Models

Deployment-ready models can be created from any supported base model using embedl-deploy, available on PyPI. This artifact follows the SAM3 tutorial workflow applied to the SAM-3D-Body backbone.

License

This model is a derivative of facebook/sam-3d-body-dinov3.

Component	License
Upstream (Meta SAM 3D Body / DINOv3)	SAM 3D Body License
Optimized components	Embedl Models Community Licence v1.0 (no redistribution as a hosted service)

Contact

Enterprise & commercial inquiries: models@embedl.com
Technical issues & early access: github.com/embedl/embedl-deploy

We offer engineering support for on-prem/edge deployments and partner co-marketing opportunities.

Downloads last month: -

Inference Providers NEW

Image Feature Extraction

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for embedl/sam-3d-body

Base model

facebook/sam-3d-body-dinov3

Quantized

(2)

this model