Arxiv paper - ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis

In this episode, we discuss ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis by Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, Yonghong Tian. ViewCrafter introduces a new method for synthesizing high-fidelity novel views from single or sparse images, using video diffusion models enhanced with sparse 3D information. It incorporates an iterative synthesis and camera trajectory planning approach to expand 3D clues and novel view areas for applications such as immersive experiences and text-to-3D scene generation. The method shows superior performance in generating consistent views from limited data, and related resources are available online.