CLiFT: Compressive Light-Field Tokens for Compute-Efficient and Adaptive Neural Rendering

Pretrained checkpoints for CLiFT (NeurIPS 2025 spotlight; arXiv, project page, code).

CLiFT represents a scene as compressed light-field tokens (CLiFTs): an encoder tokenizes the input views, latent K-means selects centroid tokens, and a neural condenser aggregates each cluster into its centroid. A single trained model supports compute-adaptive rendering — fewer tokens for lower storage/compute, more tokens for higher quality.

Checkpoints

File	Description	Place at (in the code repo)
re10k/first_stage.ckpt	RealEstate10K encoder-decoder (LVSM-style, random token-drop)	output/re10k_first_stage/training/last.ckpt
re10k/second_stage.ckpt	RealEstate10K condenser (full CLiFT model)	output/re10k_second_stage/training/last.ckpt
dl3dv/first_stage.ckpt	DL3DV encoder-decoder, fine-tuned from the RE10K model	output/dl3dv_first_stage/training/last.ckpt
dl3dv/second_stage.ckpt	DL3DV condenser (full CLiFT model)	output/dl3dv_second_stage/training/last.ckpt

The second-stage checkpoints are the full CLiFT models used for the paper's main results. The first-stage checkpoints are used by the random / K-means selection baselines, for K-means annotation, and as the initialization for condenser training.

Precomputed K-means assignments (for condenser training)

File	Description
re10k/kmeans_faiss_no_features_merged.zip	Per-scene K-means assignments for RealEstate10K second-stage training (extract to re10k_data/kmeans_faiss_no_features_merged/)
dl3dv/dl3dv_kmeans_faiss_merged.tar.zst	Per-scene K-means assignments for DL3DV second-stage training (tar --zstd -xf ... -C Dataset/)

Only needed for training the condenser; evaluation just needs the checkpoints.

Usage

git clone https://github.com/eric-zqwang/CLiFT.git
cd CLiFT
# download the checkpoints to the paths above, prepare data (see docs/), then e.g.
bash script/eval/eval_clift.sh       # RE10K
bash script/eval/eval_dl3dv.sh 6     # DL3DV, 6 context views

See the test guide for evaluation and the training guide for the two-stage training pipeline.

Citation

@inproceedings{Wang2025CLiFT,
  author    = {Wang, Zhengqing and Wu, Yuefan and Chen, Jiacheng and Zhang, Fuyang and Furukawa, Yasutaka},
  title     = {CLiFT: Compressive Light-Field Tokens for Compute Efficient and Adaptive Neural Rendering},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2025},
}