Meta and Oxford University researchers have unveiled VFusion3D, a groundbreaking AI tool that generates high-quality 3D models from a single 2D image in seconds. This innovation aims to transform industries such as virtual reality, gaming, and digital design by addressing the limited availability of 3D data crucial for AI training and content generation. VFusion3D relies on a unique approach, utilizing text, images, and videos to train the AI model instead of depending on existing 3D models.
The research team, led by Junlin Han, Filippos Kokkinos, and Philip Torr, claims that this method can produce highly accurate 3D assets from just one image, potentially reducing the workload in industries that heavily rely on 3D models. The research outlines a specialized pipeline designed for VFusion3D, which uses a minimal amount of 3D data to fine-tune a video diffusion model. Videos, particularly those capturing objects from multiple angles, serve as essential sources for creating faithful 3D reproductions.
A key component of their approach is the EMU Video model, trained with diverse video footage, including panning shots and drone footage. This model harnesses inherent 3D cues within the videos to generate accurate 3D assets from single images, regardless of the viewing angle.
Meta’s VFusion3D AI tool debuts
A user study conducted as part of the research supports the effectiveness and quality of VFusion3D. The researchers validated their model by comparing VFusion3D against competing distillation-based and feed-forward 3D generative models. The comparative quality and performance of VFusion3D have been showcased on a GitHub project page, where users can explore animated objects generated by VFusion3D and its rivals.
For those interested in testing VFusion3D, an online demo is available. Users can generate and download 3D models from provided example images or upload their own source images. However, due to high demand, the demo may occasionally be unresponsive.
This innovative development promises to be a game-changer in fields requiring rapid and accurate 3D asset generation, pushing the boundaries of what’s possible with AI-driven technology in digital design and beyond. Meta and Oxford University’s ultimate goal with VFusion3D is to provide entertainment companies with a powerful tool to simplify and revolutionize 3D model creation, leading to significant productivity improvements and cost reductions without compromising on the quality of the generated models.
