Instant NeRF can turn 2D images into a 3D scene in milliseconds
This is truly the era of modern technology where things can be done in seconds. This is proven by Instant NeRF, which was able to render a 3D scene with a mannequin dressed as Andy Warhol in a demo made by NVIDIA The research team. It only took tens of milliseconds for the process to do this using still photos taken from different angles.
According to Nvidiathe concept of Instant NeRF is quite simple: use inverse rendering and apply it to neural radiation fields, or NeRF, one can transform a set of 2D images into a 3D work in the blink of an eye. This is a far cry from traditional methods of rendering a 3D scene, which typically take hours to days depending on how much detail you want to include. Early NeRF models that use AI somehow shorten the render period, but not that significantly. Then here is Instant NeRF, the method that can do it in milliseconds.
“While traditional 3D representations like polygon meshes are akin to vector images, NeRFs are like bitmap images: they densely capture the way light radiates from an object or within it. scene,” said David Luebke, vice president of NVIDIA Graphics Research. “In this sense, Instant NeRF could be as important to 3D as digital cameras and JPEG compression have been to 2D photography, dramatically increasing the speed, ease and reach of 3D capture and sharing. “
The Instant NeRF includes a fast 2D image capture process to capture still images, as subject movements can result in a blurry 3D scene. After that, the NeRF will recreate the scene providing the missing information and predicting the color of light radiating in any direction in 3D space.
“It relies on a technique developed by NVIDIA called Multi-Resolution Hash Grid Coding, which is optimized to run efficiently on NVIDIA GPUs,” writes NVIDIA Corporate Communications Team Member Isha Salian. in one blog post. “Using a new input coding method, researchers can achieve high-quality results using a tiny neural network that runs quickly… The model was developed using the NVIDIA CUDA toolkit and the Tiny CUDA Neural Networks library. Since it’s a lightweight neural network, it can be trained and run on a single NVIDIA GPU – running faster on boards with NVIDIA Tensor Cores.
According to NVIDIA, Instant NeRF can be useful for a wide variety of purposes, including creating avatars and virtual worlds or reconstructing scenes and events in 3D form. It can also be used to help and train robots to learn the real size, shape and dimensions of objects.