Geometrically Consistent Light Field Synthesis Using Repaint Video Diffusion Model

We propose to repaint an image-to-video diffusion model to synthesize light fields that are geometrically consistent. Despite significant advancements in diffusion models for novel view synthesis, applying these models to generate a light field, i.e., fronto-parallel multiple views, has been challenging because of persistent visual and geometric consistency issues. By utilizing advances in video diffusion, we extend the temporal consistency of video diffusion to the geometric consistency of multi-view settings. We fine-tune the image-to-video diffusion model framework for optimized multi-view diffusion by incorporating multi-view data with camera parameters. Furthermore, we propose integrating a repaint method during the sampling (denoising process) to achieve enhanced accurate camera control in multi-view diffusion, improving consistency by maintaining the known region in the input image. This approach enables the application of light field synthesis that requires precise camera control and demonstrates the ability of diffusion models to generate light fields with wide baselines, leveraging their unique generative power.